Frame rate control method and related apparatus

Abstract

The application provides a frame rate control method and related device. When a first application is running on an electronic device, the screen display content is refreshed at a first frame rate. When the first application is switched to a second application, frame rate switching is triggered. In the frame rate switching process, two steps are executed: ① the frequency of a TE signal is adjusted first. ② after a frame switching instruction is cached, the number of image frames received by HWC is counted. When the number reaches a first number (which can be determined in advance and written into the storage space of the electronic device), the period of a MIPI signal is adjusted, so that the frequency of the TE signal and the period of the MIPI signal take effect in the same screen refresh period, thereby solving the display abnormality problem caused by the unsynchronized taking effect of the TE frequency and the MIPI signal in the frame switching process, and improving the viewing experience of the user in the frame rate switching process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display, in particular to a frame rate control method and related device. BACKGROUND

[0002] With the rapid development of display technology, more and more display screens can support dynamic adjustment of refresh rate. For example, when running an application with high frame rate requirement, the refresh rate of the display screen is increased to improve the smoothness of the picture; when running an application with low frame rate requirement, the refresh rate of the display screen is decreased to reduce the power consumption of the electronic device. However, the current refresh rate adjustment scheme has display abnormality phenomenon. SUMMARY

[0003] Therefore, the present application provides a frame rate control method and related device to solve at least part of the above problems, and the disclosed technical scheme is as follows:

[0004] In a first aspect, the present application provides a frame rate control method applied to an electronic device with an LTPS screen, which comprises: receiving a first operation of a user; starting a first application in response to the first operation; transmitting an image frame based on a first MIPI signal and displaying the image frame based on a first TE signal during the running of the first application, wherein the frequency of the first TE signal is the same as a first frame rate; receiving a second operation of the user; starting a second application in response to the second operation and generating a frame cutting instruction; buffering the frame cutting instruction; generating a second TE signal based on the first TE signal; generating a second MIPI signal based on the first MIPI signal in the case that the number of image frames received by a hardware compositor of the electronic device reaches a first number after buffering the frame cutting instruction; transmitting an image frame based on the second MIPI signal and displaying the image frame based on the second TE signal, wherein the frequency of the second TE signal is the same as a second frame rate, and the second frame rate is different from the first frame rate. It can be seen that, after generating the frame cutting instruction, the present application buffers the frame cutting instruction first, and then executes the frame cutting instruction in two steps: ① first adjust the frequency of the TE signal; ② after buffering the frame cutting instruction, in the case that the number of image frames received by the HWC reaches the first number (the first number can be determined in advance and written into the storage space of the electronic device), adjust the period of the MIPI signal again, so that the adjustment of the frequency of the TE signal and the adjustment of the period of the MIPI signal take effect in the same screen refresh period. Therefore, the present application avoids the display abnormality problem caused by the unsynchronized taking effect of the TE frequency and the MIPI signal in the frame cutting process, and improves the viewing experience of the user in the frame rate switching process.

[0005] In a possible implementation manner of the first aspect, before the second MIPI signal is generated based on the first MIPI signal when the number of image frames received by the hardware compositor of the electronic device after the cut-frame instruction is cached reaches the first number, the method further includes: counting the number of image frames received by the hardware compositor of the electronic device after the cut-frame instruction is cached; and determining that the number of image frames received by the hardware compositor of the electronic device after the cut-frame instruction is cached reaches the first number when the counted value reaches the first number. It can be seen that the number of image frames received by the HWC after the cut-frame instruction is cached is counted, so that the number of caching periods (or the number of cached frames) of the cut-frame instruction is counted, and the method is more convenient to implement.

[0006] In a possible implementation manner of the first aspect, before the second operation of the user is received, the method further includes: receiving a third operation of the user; and in response to the third operation, closing the first application or returning to a desktop.

[0007] In a possible implementation manner of the first aspect, the first application is a video application, and the second application is an application for displaying a graphic text. For example, the second application can be a browser, a Zhihu APP, a Baidu APP, or the like.

[0008] In a possible implementation manner of the first aspect, before the cut-frame instruction is cached, the method further includes: determining that the first frame rate is greater than the second frame rate. It can be seen that when the high frame rate is switched to the low frame rate, the frame rate control method provided in the application is triggered to be executed, the frequency of the MIPI signal is adaptively reduced after the high frame rate is switched to the low frame rate, the overall power consumption of the electronic device is reduced, and at the same time, the frequency of the TE signal and the period of the MIPI signal are adjusted to take effect in the same screen refresh period, so that the display abnormality problem caused by the unsynchronized taking effect of the TE frequency and the MIPI signal in the cut-frame process is avoided, and the user experience is improved.

[0009] In a possible implementation manner of the first aspect, when the number of image frames received by the hardware compositor of the electronic device after the cut-frame instruction is cached reaches the first number, the second MIPI signal is generated based on the first MIPI signal, including: when the number of image frames received by the hardware compositor after the cut-frame instruction is cached reaches the first number, the cut-frame instruction is parsed to obtain a second frame rate; based on a mapping relationship between the frame rate and the period of the MIPI signal, a first period matched with the second frame rate is obtained; and based on the first period, the second MIPI signal is generated based on the first MIPI signal, the period of the second MIPI signal is equal to the first period, and the first period is greater than the period of the first MIPI signal.

[0010] ​In one possible implementation of the first aspect, generating a second TE signal based on a first TE signal includes: parsing a frame-cutting instruction to obtain a second frame rate; generating a TE frequency adjustment instruction based on the second frame rate; and generating a second TE signal based on the first TE signal in response to the TE frequency adjustment instruction.

[0011] Secondly, this application also provides a frame rate control method applied to an electronic device employing an LTPS screen. The method includes: receiving a first operation from a user; responding to the first operation by launching a first application; transmitting image frames based on a first MIPI signal and displaying image frames based on a first TE signal during the operation of the first application, thereby causing the electronic device to refresh the screen display content at a first frame rate. The frequency of the first TE signal is the same as the first frame rate; receiving a second operation from a user; responding to the second operation by launching a second application; and generating a second TE signal based on the first TE signal in a first period (e.g., as shown in the image). Figure 8 The example shown adjusts the frequency of the TE signal in the B frame; in the second cycle, a second MIPI signal is generated based on the first MIPI signal (e.g., as shown in the example). Figure 8 In the example shown, the period of the MIPI signal is adjusted (triggered in frame D). The second period follows the first period and is spaced a preset number of times the first period interval is applied, where the size of the first period interval is the reciprocal of the first frame rate. In the third period, image frames are transmitted based on the second MIPI signal and displayed based on the second TE signal (e.g., as shown in the example). Figure 8 The example shown illustrates that adjusting the MIPI signal period and the TE signal frequency takes effect synchronously within the E-frame. The third period follows the second period, the second TE signal frequency is the same as the second frame rate, and the second frame rate differs from the first frame rate. This scheme first triggers the adjustment of the TE signal frequency, and then triggers the adjustment of the MIPI signal period after a first number of screen refresh cycles (i.e., after receiving a first number of image frames). This ensures that the TE frequency and MIPI signal take effect within the same screen refresh cycle, avoiding display anomalies caused by the TE frequency and MIPI signal not being synchronized during frame switching, thus improving the user's viewing experience during frame rate changes.

[0012] In a third aspect, the present application also provides a frame rate control method applied to an electronic device with an LTPO screen, the method comprising: receiving a first operation of a user; in response to the first operation, starting a first application; displaying image frames at a first frame rate during running of the first application, the first frame rate being 90 Hz; receiving a second operation of the user; in response to the second operation, starting a second application and generating an original frame-cutting instruction, the original frame-cutting instruction being used to make the electronic device display image frames at a second frame rate, the second frame rate being less than the first frame rate; generating a first frame-cutting instruction and adjusting the first frame rate to a third frame rate in response to the first frame-cutting instruction, the third frame rate being 120 Hz; generating a second frame-cutting instruction in a next screen refresh cycle of generating the original frame-cutting instruction; and adjusting the third frame rate to the second frame rate in response to the second frame-cutting instruction. It can be seen that in the process of switching from 90 Hz to other lower frame rates (which can be referred to as target frame rates), the scheme is not directly switched from 90 Hz to the target frame rate, but the frame-cutting process is divided into two steps: ① first switch from 90 Hz to 120 Hz; ② switch from 120 Hz to the target frame rate. Switching to 120 Hz first makes the next frame still maintain a frame rate of 90 Hz, and then switching to the target frame rate after 1 frame, avoiding the frame loss phenomenon that occurs when directly switching from 90 Hz to 60 Hz, thereby improving the user viewing experience in the process of switching from 90 Hz to other lower frame rates of the LTPO screen.

[0013] In a possible implementation manner of the third aspect, generating the second frame-cutting instruction in the next screen refresh cycle of generating the original frame-cutting instruction comprises: after generating the original frame-cutting instruction, buffering the original frame-cutting instruction; and generating the second frame-cutting instruction in a case where the number of image frames received by a hardware compositor of the electronic device after buffering the original frame-cutting instruction reaches 1 frame.

[0014] In a possible implementation manner of the third aspect, before receiving the second operation of the user, the method further comprises: receiving a third operation of the user; and in response to the third operation, closing the first application or returning to a desktop.

[0015] In a possible implementation manner of the third aspect, the first application is a video application, and the second application is an application for displaying images and texts.

[0016] In a fourth aspect, the present application provides a frame rate control method applied to an electronic device. The method comprises: receiving a first operation of a user; starting a first application in response to the first operation; transmitting an image frame based on a first MIPI signal and displaying the image frame based on a first TE signal during running of the first application, wherein a frequency of the first TE signal is the same as a first frame rate; receiving a second operation of the user; starting a second application and generating a frame cutting instruction in response to the second operation; if it is determined that a screen type of the electronic device is an LTPS screen, buffering the frame cutting instruction; generating a second TE signal based on the first TE signal; generating a second MIPI signal based on the first MIPI signal if a number of image frames received by a hardware synthesizer of the electronic device reaches a first number after buffering the frame cutting instruction; transmitting an image frame based on the second MIPI signal and displaying the image frame based on the second TE signal, wherein a frequency of the second TE signal is the same as a second frame rate, and the second frame rate is different from the first frame rate; if it is determined that the screen type is an LTPO screen and the first frame rate is 90 Hz, and the second frame rate is less than the first frame rate, generating a first frame cutting instruction; adjusting the first frame rate to a third frame rate in response to the first frame cutting instruction, wherein the third frame rate is 120 Hz; generating a second frame cutting instruction in a next screen refresh cycle after generating the frame cutting instruction; and adjusting the third frame rate to the second frame rate in response to the second frame cutting instruction. It can be seen that the scheme does not need to develop different frame rate switching processing logics for different screen types, and the electronic device using the LTPS screen and the LTPO screen can use the same frame rate switching processing logic, thereby improving the application range of the frame rate switching control program and reducing the development workload of the developer.

[0017] In a fifth aspect, the present application provides an electronic device. The electronic device comprises: one or more processors, a memory and a touch screen; the memory is configured to store program code; and the processor is configured to execute the program code, so that the electronic device implements the frame rate control method according to any possible implementation manner of the first aspect to the fourth aspect.

[0018] In a sixth aspect, the present application provides a computer readable storage medium having instructions stored thereon. When the instructions are executed on an electronic device, the electronic device performs the frame rate control method according to any possible implementation manner of the first aspect to the fourth aspect.

[0019] In a seventh aspect, the present application provides a computer program product having instructions stored thereon. When the computer program product is executed on an electronic device, the electronic device implements the network service optimization method according to any possible implementation manner of the first aspect to the fourth aspect.

[0020] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be achieved in any single embodiment. On the contrary, it can be understood that the description of a feature or a beneficial effect means that the specific technical feature, technical solution or beneficial effect is included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Further, the technical features, technical solutions and beneficial effects described in this embodiment can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings from these drawings without creating labor.

[0022] Figure 1 is a schematic diagram of a service scenario switching provided by an embodiment of the present application;

[0023] Figure 2 is a frame rate switching schematic diagram of a traditional LTPS screen provided by an embodiment of the present application;

[0024] Figure 3 is another frame rate switching schematic diagram of a traditional LTPS screen provided by an embodiment of the present application;

[0025] Figure 4 is a frame rate switching schematic diagram of a traditional LTPO screen provided by an embodiment of the present application;

[0026] Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0027] Figure 6 is a software architecture schematic diagram of an electronic device using an LTPS screen provided by an embodiment of the present application;

[0028] Figure 7 is a software architecture schematic diagram of an electronic device using an LTPO screen provided by an embodiment of the present application;

[0029] Figure 8 is a frame rate switching schematic diagram of an LTPS screen provided by an embodiment of the present application;

[0030] Figure 9 is a frame rate switching schematic diagram of an LTPO screen provided by an embodiment of the present application;

[0031] Figure 10 is a software architecture schematic diagram of an electronic device provided by another embodiment of the present application using an LTPS screen;

[0032] Figure 11 is a software architecture schematic diagram of an electronic device provided by another embodiment of the present application using an LTPO screen;

[0033] Figure 12 is a flowchart of a frame rate control method provided by an embodiment of the present application;

[0034] Figure 13 is a flowchart of another frame rate control method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The terms "first", "second", and "third" and the like in the specification and claims of the present application and the description of the drawings are used to distinguish different objects, and are not used to limit a specific order.

[0036] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, in no way limiting. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of the words "exemplary" or "for example" is intended to present relevant concepts in a specific way.

[0037] For the sake of clear and concise description of each of the following embodiments, first, a brief introduction of related technologies is given:

[0038] Frame rate is the frequency of continuous appearance of images in frames on a display screen, which refers to the number of pictures transmitted per unit time (1 second).

[0039] Refresh rate refers to the number of times of repeated scanning of images on a screen by an electron beam, and the higher the refresh rate, the better the stability of the displayed image.

[0040] A display driver integrated circuit (DDIC) is a chip used to drive a display screen for image display. It controls the display panel for self-illumination and image display. During image display, the DDIC receives image data from the application processor (AP) through the Mobile Industry Processor Interface (MIPI), and then drives the display panel to scan the image based on this data, thereby achieving image display. (The following text...) Figure 6 The software architecture of the electronic device shown runs within the application processor (AP), meaning that the application layer, application framework layer, system library layer, hardware abstraction layer, and kernel layer all run within the application processor.

[0041] The Tearing Effect (TE) signal is a signal generated by the DDIC to prevent screen tearing during image refresh. The DDIC generates the TE signal when it has finished refreshing one frame of the image and is ready to refresh the next frame.

[0042] Electronic devices can dynamically adjust the frame rate according to different business scenarios. Furthermore, DDIC dynamically adjusts the refresh rate of the display screen based on the frame rate.

[0043] For example, such as Figure 1 As shown in (1), after the mobile phone receives the user's click on the video application, it jumps to the following... Figure 1 The video application page shown in (2) is the one that launches the video application. The mobile phone receives the user's... Figure 1 After swiping up from the bottom of the page as shown in (2), exit the video application or go back directly to the previous page. Figure 1 The desktop shown in (3) is shown. After the phone receives the user's click on the reading application, it jumps to... Figure 1 The page of the reading application shown in (4) is used to launch the reading application.

[0044] When a video application starts playing a video, it requires a high frame rate, such as 120Hz or 90Hz. The user then exits the video application and launches a reading application, which requires a lower frame rate, such as 60Hz, 40Hz, or 30Hz. When the electronic device switches from a video scenario to a reading scenario, the frame rate can be switched from 120Hz to 60Hz to reduce power consumption.

[0045] However, the current frame rate switching scheme has the phenomenon of screen flashing or frame loss. Current display screens include two types of Low Temperature Poly-silicon (LTPS) and Low Temperature Polycrystalline Oxide (LTPO). For different frame rates, LTPS screens usually use different hardware clocks (i.e. the clock of MIPI transmitting image data, also known as MIPI CLK) to transmit image data, and the hardware clock is low when the frame rate is low, and the hardware clock is high when the frame rate is high. The implementation principle of LTPO screen is different from that of LTPS screen, and LTPO screen needs to use the same hardware clock to transmit image data for different frame rates. Both types of screens have problems when switching frame rates, which will be described below.

[0046] 1. The current LTPS screen has a screen flashing problem when switching frames

[0047] For LTPS screens, the hardware clock and TE frequency (i.e. the frequency of the TE signal) are usually changed when switching frame rates, and the hardware clock change and TE frequency modification are usually completed in the same frame. However, after the hardware clock is changed, the next image transmission period takes effect immediately, while the TE frequency may not take effect. Some DDICs take effect in the next frame, while some DDICs need to be delayed for 1-2 frames to take effect. This 1-2 frames will cause the hardware clock and TE frequency to be mismatched, resulting in screen flashing abnormalities.

[0048] The AP side performs layer rendering through APP and layer composition through SurfaceFlinger to obtain image data, and finally sends the image data to the display through the MIPI interface, i.e. writes the image data to the DDIC, and the DDIC controls the display panel to refresh and display the image. After the AP completes the preparation of image data, it does not immediately send the display, but sends the display when detecting the TE signal output by the DDIC, i.e. the AP detects the rising edge of the TE signal and then starts the transmission of the next frame of image data.

[0049] As shown in Figure 2 , the frame rates of A frame and B frame are both 120Hz. During the display process of B frame image, SurfaceFlinger initiates a frame switching from 120Hz to 60Hz, and when the next TE signal arrives, the modification of MIPI CLK takes effect, i.e. the modification of MIPI CLK takes effect immediately in C frame. In this example, the TE frequency adjustment is delayed by 2 frames compared with the time when the frame switching is initiated, i.e. the modification of TE frequency takes effect in E frame. This will cause the MIPI CLK modification of C frame and D frame to take effect while the TE frequency does not take effect.

[0050] Switching from 120Hz to 60Hz lowers the MIPI CLK frequency, thus slowing down the transmission of image data. This means that transmitting the same amount of image data using the lowered MIPI CLK frequency takes longer. Since the TE frequency modification hasn't taken effect (i.e., the TE frequency is still output at 120Hz), there will be instances where the C-frame image data isn't fully transmitted before the D-frame TE signal is output. As mentioned earlier, the AP will start transmitting the next frame of image data after detecting the rising edge of the TE signal. This means that the D-frame image data transmission starts before the C-frame image data is fully transmitted, causing abnormal C-frame display and screen flickering. For the same reason, the E-frame transmission starts before the D-frame is fully transmitted, also causing abnormal D-frame display. After the TE frequency modification takes effect in the E-frame, the MIPI CLK and TE frequencies synchronize, and the image display is normal. Therefore, it is clear that LTPS screens will experience screen flickering when switching from a high frame rate to a low frame rate.

[0051] In order to solve Figure 2 The LTPS screen shown exhibits display abnormalities when switching from a high frame rate to a low frame rate. Typically, after sending the frame-switching command to the DDIC, a delay occurs, and image compositing is not performed until the DDIC's TE cycle modification takes effect before compositing the next frame. For example, as... Figure 3 As shown, after the frame-cutting command is issued in frame B, frames C and D are not combined into images. The next frame is combined in frame E. As a result, frames C and D will be dropped.

[0052] 2. Current LTPO screens experience frame drops during frame switching.

[0053] LTPO screens use 120Hz as the base frequency, and other frame rates are obtained through a frameskip mechanism. The frequencies obtained in this way are all divisible by 120Hz, such as 60Hz, 40Hz, 30Hz, 20Hz, and 10Hz. However, in practical applications, 90Hz is also used. This 90Hz is simulated using the LongV method, which involves lengthening the high-level portion of the TE signal to 2.8ms on top of 120Hz (8.3ms), i.e., 2.8ms + 8.3ms = 11.1ms. This causes inaccurate TE periods in the first frame when switching from 90Hz to other frame rates, affecting image synthesis in the SF (Frame Startup) and leading to frame drops.

[0054] For example, such as Figure 4As shown, the A frame and the B frame run at a frame rate of 90Hz, and the B frame undergoes a frame rate switching from 90Hz to 60Hz. However, when the next period (i.e., the C frame) arrives, the TE signal cannot be modified in time, and the high level part thereof remains 2.8ms, and the low level part thereof becomes a period corresponding to 60Hz, i.e., 16.6ms. That is, the period of the C frame is 16.6ms+2.8ms=19.4ms, which is inaccurate and longer than the period corresponding to 60Hz by 2.8ms.

[0055] The AP transmits image data to the DDIC starting from the rising edge of the TE signal, i.e., the high level part 2.8ms is counted into the image sending period, and the period of the C frame is lengthened by 2.8ms, which causes the presentFence (indicating that the display of the previous frame is completed) to be released late by 2.8ms. The SF synthesizes an image according to the period of the VSYNC signal, and if the presentFence is not released after the VSYNC signal is received, the SF does not synthesize the next frame image.

[0056] After the 90Hz-to-60Hz switching frame instruction is issued to the HWC, the abnormal period 19.4ms of the TE signal of the DDIC is synchronized to the VSYNC, causing the VSYNC modeling of the SF to be disordered, so that the VSYNC of the SF and the TE signal are not aligned according to the offset (i.e., the agreed offset between the VSYNC signal and the TE signal), causing the work rhythm of the SF (the SF works according to the period of the VSYNC signal) and the work rhythm of the display driver (the display driver works according to the period of the TE signal) to be inconsistent, so that the release time of the presentFence and the rising edge time of the VSYNC signal of the SF deviate from each other, and finally the SF does not synthesize the next frame image because the release of the presentFence is overdue. As shown, Figure 4 As shown, the release of the presentFence signal of the C frame is not in time, causing the SF to fail to synthesize the D frame image, and causing the D frame to be lost.

[0057] After the inventors found the above problems in the current frame rate control scheme, the frame rate control method of the present application is proposed, which is applied to Figure 5 the electronic device as shown.

[0058] The electronic device can be an electronic device provided with a display screen, such as a mobile phone, a tablet computer, a desktop / laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), and a wearable electronic device. The specific form of the electronic device is not specially limited in the present application.

[0059] As shown,Figure 5 As shown, the electronic device can include a processor, a display screen, a touch sensor, a memory, a communication module, a USB interface, a charging management module, a power management module, a battery, and the like.

[0060] It can be understood that the structure shown in the embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor is the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching and executing instructions.

[0062] The processor can include one or more processing units. For example, the processor can include an application processor, a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0063] The memory can be used to store computer executable program codes, which can include an operating system, at least one application required for a function (such as a sound playing function, an image / video playing function, etc.), and the like. The executable program codes include instructions, and the processor runs the instructions stored in the memory, so that the electronic device performs various function applications and data processing. For example, in the present application, the processor runs the instructions stored in the memory, so that the electronic device performs the frame rate control method provided in the present application.

[0064] The display screen is used to display images, videos, a series of graphical user interfaces (GUIs), and the like. The electronic device can include 1 or N display screens, N being a positive integer greater than 1.

[0065] The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oled, a quantum dot light emitting diodes (QLED), or the like.

[0066] The touch sensor, also referred to as a "touch device". The touch sensor can be disposed on the display screen, and the touch sensor and the display screen form a touch screen, also referred to as a "touch screen". The touch sensor is used to detect a touch operation acting on or near the touch sensor. The touch sensor can transmit the detected touch operation to the application processor to determine the touch event type. The visual output related to the touch operation can be provided through the display screen. In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device, which is different from the position of the display screen.

[0067] The communication module can include at least one of a mobile communication module and a wireless communication module.

[0068] The wireless communication function of the electronic device can be realized through an antenna, a mobile communication module, a wireless communication module, a modem processor, and a baseband processor, etc.

[0069] The mobile communication module can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device.

[0070] The wireless communication module can provide a solution for wireless communication including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device. The wireless communication module can be one or more devices integrating at least one communication processing module.

[0071] The USB interface is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge the electronic device, and can also be used to transmit data between the electronic device and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0072] The charging management module is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module can receive charging input from a wired charger through the USB interface. In some wireless charging embodiments, the charging management module can receive wireless charging input through the wireless charging coil of the electronic device. The charging management module can charge the battery while also providing power to the electronic device through the power management module.

[0073] The power management module is used to connect the battery, the charging management module, and the processor. The power management module receives input from the battery and / or the charging management module to provide power to various devices / modules within the electronic device.

[0074] In addition, an operating system runs on the above-mentioned components. For example an operating system, an open-source operating system, an operating system, etc. Application programs can be installed and run on the operating system.

[0075] The operating system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The present embodiment takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device.

[0076] Figure 6 is a software structure block diagram of an electronic device with an LTPS screen according to an embodiment of the present application, Figure 7 is a software structure block diagram of an electronic device with an LTPO screen according to an embodiment of the present application.

[0077] The layered architecture divides the software into several layers, each layer having a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom, including an application layer, an application framework layer (Framework), an Android runtime and a system library, a hardware abstraction layer (HAL), and a kernel layer (Kernel).

[0078] The application layer can include a series of application packages. As shown in Figure 6 and Figure 7 , the application package can include video, game, reading, etc. applications.

[0079] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0080] In the embodiments of the present application, as shown in Figure 6 and Figure 7 , the application framework layer includes a window manager service (WMS). The window manager is used to manage the window program.

[0081] The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, etc. The window manager is an interface for managing the Android window mechanism, and is the lowest layer for displaying the View.

[0082] Android runtime and system library (Native), wherein the Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0083] The core library includes two parts: one part is the function function called by the java language, and the other part is the core library of Android.

[0084] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc.

[0085] The system library can include a plurality of functional modules. For example, in the embodiments of the present application, the system library can include: a display synthesizer (SurfaceFlinger), a frame rate management module, etc.

[0086] The function of SurfaceFlinger is to accept multiple sources of graphical display data, synthesize them and send them to the hardware display device, such as DDIC.

[0087] The frame rate management module is used to dynamically adjust the current running frame rate of the application according to the business scenario of the electronic device, and trigger the SurfaceFlinger to initiate the frame rate switching.

[0088] The hardware abstraction layer (HAL) serves as a bridge between software and hardware, encapsulates the underlying hardware driver, and provides a general interface for the Framework layer to call the driver. For example, in the embodiments of the present application, the hardware abstraction layer can include a hardware composer (HWC), which is used to complete the functions of image data composition and display by hardware.

[0089] In some embodiments of the present application, in addition to including a function module for completing the functions of image data composition and display by hardware, the HWC can also include a screen type judgment module, a cache module, and a counting module.

[0090] The screen type judgment module is used to determine whether the screen type of the electronic device is an LTPS screen or an LTPO screen. If it is an LTPS screen, it continues to determine whether the current frame cutting is from a high frame rate to a low frame rate. If so, the frame rate control method of the present application is executed. If the current screen is an LTPO screen and the current frame cutting is from 90Hz to 60Hz (or other lower frame rates such as 40Hz, 30Hz, etc.), the frame rate control method of the present application is triggered to be executed.

[0091] In an exemplary embodiment, during the initialization stage of the operating system of the electronic device, the HWC can read the screen type of the current electronic device and the number of lag frames (i.e., the preset number of frames) corresponding to the effective TE frequency of the screen from the display driver through the channel 3 (or third channel) as shown in the figure. Figure 6

[0092] The cache module is used to cache the frame cutting instructions sent by the SurfaceFlinger to the HWC.

[0093] The counting module is used to count the image frames received by the HWC after caching the frame cutting instructions.

[0094] The kernel layer is a layer between the hardware and the software. In the embodiments of the present application, the kernel layer at least contains a display driver, etc.

[0095] In the embodiments of the present application, the display control module and the cache frame number configuration module are set in the display driver.

[0096] The cache frame number configuration module stores the number of lag frames (i.e., the first number described above) corresponding to the TE frequency of the DDIC currently used by the electronic device, for example, 1 frame, 2 frames. This parameter is directly written into the code of the display driver.

[0097] In the embodiments of the present application, the channel between the HWC and the display driver can be a communication interface, and adding a communication channel between the HWC and the display driver can set a new interface for the HWC and the display driver.

[0098] ​The display control module is configured to respond to the instruction issued by the HWC.

[0099] In the embodiments of the present application, the hardware layer can include a data processing unit (DPU), a DDIC, and a display panel (e.g., an OLED, an LCD, etc.).

[0100] As shown in Figure 6 , the APP currently running on the electronic device performs layer rendering, and the window manager is used to manage the underlying View of the APP currently running on the electronic device. Then, the layer data is transmitted to the SurfaceFlinger for layer composition to obtain image data.

[0101] When the APP currently running on the electronic device changes, for example, from a video application to a reading application, the frame rate management module can perceive that the business scenario of the electronic device changes, i.e., from a high frame rate scenario to a low frame rate scenario. The SurfaceFlinger is triggered to initiate frame rate switching, for example, from 120Hz to 60Hz. The SurfaceFlinger issues a frame switching instruction while issuing the image data. The frame switching instruction carries a target frame rate, for example, 60Hz.

[0102] In the embodiments, after the HWC receives the frame switching instruction, the screen type of the electronic device is first determined.

[0103] 1) LTPS screen

[0104] If it is an LTPS screen, it is further determined whether the current frame switching instruction is from a high frame rate to a low frame rate. If it is from a high frame rate to a low frame rate, the frame rate control method provided in the present application is triggered. If it is from a low frame rate to a high frame rate, the original frame rate control method is executed, i.e., the frame switching instruction is directly issued to the display driver.

[0105] For example, the APP running on the electronic device is switched from a video APP to a reading APP. The frame rate required by the video APP is 120Hz, and the frame rate required by the reading APP is 60Hz. That is, the frame rate is switched from 120Hz to 60Hz. Moreover, the TE frequency of the DDIC used by the electronic device lags behind by 2 frames to take effect.

[0106] The process of switching from 120Hz to 60Hz of the electronic device with an LTPS screen will be described below in conjunction with Figure 6 and Figure 8 .

[0107] For example, as shown in Figure 8 , the frame rate of the A frame is 120Hz, and the SurfaceFlinger initiates frame rate switching from 120Hz to 60Hz after the image data of the B frame is transmitted. As shown in Figure 6As shown, the buffer module in the HWC buffers the received frame-splitting command and triggers the counting module to count the number of frames of image data received by the HWC. Simultaneously, in the B frame, the HWC sends a TE frequency adjustment command to the display driver through channel 1 (or the first channel) between the HWC and the display driver. After receiving the TE frequency adjustment command, the display driver forwards it to the data processor (DPU). Further, the DPU sends the TE frequency adjustment command to the DDIC via MIPI, and the DDIC adjusts the TE frequency in response. The TE frequency is the frequency of the TE signal output by the DDIC, which is also the frame rate. The TE frequency adjustment takes effect two frames after the command is sent, meaning the TE frequency modification takes effect in the E frame.

[0108] And, such as Figure 6 As shown, when the counter value of the counting module reaches 2 frames, a MIPI signal adjustment command (or frame-switching command) is sent through channel 2 (or the second channel) between the HWC and the display driver. That is, the HWC will not send a frame-switching command in frames B and C until it receives the image data of frame D, after which it sends the frame-switching command to the display driver. After receiving the frame-switching command, the display driver determines the target period of the MIPI signal that matches the target frame rate and adjusts the MIPI signal period of the data processor to the target period of the MIPI signal. The modified MIPI signal period takes effect immediately in the next period, such as... Figure 8 As shown, the MIPI signal modification takes effect in E-frame.

[0109] like Figure 8 As shown, in the E-frame, the period of the TE signal becomes 16.6ms, and simultaneously, the period of the MIPI signal lengthens, meaning the time for transmitting image data via MIPI within one period increases. It is evident that the TE frequency and MIPI signal are synchronized in the E-frame, ultimately completing the frame rate switch from 120Hz to 60Hz. Furthermore, due to... Figure 8 It can be seen that the frame rate of C-frames and D-frames is still 120Hz, and there is no phenomenon of frame dropping or failure to synthesize.

[0110] 2) LTPO screen

[0111] Based on the aforementioned frame rate adjustment principle of LTPO screens, it can be seen that when switching frame rates, LTPO screens do not need to modify the MIPI signal period, but only the TE frequency.

[0112] If it is an LTPO screen, then it continues to determine whether the current frame switching command is switching from 90Hz to 60Hz. If so, the frame rate control method provided in this application is triggered; otherwise, the original frame rate control method is executed, that is, the frame switching command is directly sent to the display driver.

[0113] The following is combined with Figure 7 and Figure 9introducing the process of the electronic device with LTPO screen from 90Hz to 60Hz:

[0114] As shown in Figure 9 , the A frame is 90Hz, and the HWC receives the 90Hz to 60Hz frame cutting instruction issued by the SurfaceFlinger after the image data transmission of the B frame is completed. As shown in Figure 7 , the HWC first issues the frame cutting instruction (or referred to as the first frame cutting instruction) to the display driver through channel 1 to cut to 120Hz. After receiving the first frame cutting instruction, the display driver issues the first frame cutting instruction to the DPU, and the DPU further issues the first frame cutting instruction to the DDIC through MIPI. The DDIC adjusts the TE frequency in response to the first frame cutting instruction. However, the high level part of the TE cannot be modified in time, so the high level part of the next period of TE is still 2.8ms, and the low level part is modified to 8.3ms corresponding to the period of 120Hz. As shown in Figure 9 , the next period (i.e. the C frame) after receiving the first frame cutting instruction is still 2.8+8.3=11.1ms, i.e. 90Hz.

[0115] As shown in Figure 7 , the HWC issues the frame cutting instruction (or referred to as the second frame cutting instruction) to the display driver through channel 2 to cut to 60Hz in the next period (i.e. the C frame) of receiving the first frame cutting instruction. After receiving the second frame cutting instruction, the display driver issues the second frame cutting instruction to the DPU, and the DPU further issues the second frame cutting instruction to the DDIC through MIPI. The DDIC adjusts the TE frequency in response to the second frame cutting instruction. The next period (i.e. the D frame) after issuing the second frame cutting instruction, the high level part of the TE signal has been modified to the narrow level corresponding to 120Hz, and the low level part is modified to 16.6ms corresponding to the period of 60Hz, i.e. the TE frequency of the D frame is switched to 60Hz.

[0116] From the above frame rate control process, it can be known that when the LTPO screen is switched from 90Hz to 60Hz, the HWC receives the frame cutting instruction issued by the SurfaceFlinger, and first issues the first frame cutting instruction to the display driver to cut to 120Hz, and then issues the frame cutting instruction to cut to 60Hz after 1 frame.

[0117] In another embodiment of the present application, Figure 6 and Figure 7 the execution logic of the HWC shown in the drawings can also be executed by the display driver module. Figure 10 is another software architecture diagram of an electronic device with LTPS screen provided by an embodiment of the present application; Figure 11 is another software architecture diagram of an electronic device with LTPO screen provided by an embodiment of the present application.

[0118] Figure 10-11The software architecture of the embodiment shown is different from that of the prior art in that the processing logic of the cache and counting of the frame switching instruction is executed by the display driver instead of the HWC. As shown in Figure 6-7 , the display driver module includes a screen type judgment module, a cache module, a counting module, and a cache frame number configuration module, wherein the functions of the respective modules are the same as those of the modules in Figure 10-11 . The HWC includes a frame switching control module. Figure 6-7

[0119] When it is necessary to switch the frame rate, the SurfaceFlinger issues the frame switching instruction to the HWC at the same time as the image data. The HWC directly issues the frame switching instruction to the display driver. After receiving the frame switching instruction, the display driver first uses the screen type judgment module to determine whether the screen of the electronic device is LTPS or LTPO.

[0120] As shown in Figure 10 , if it is an LTPS screen, the cache module in the display driver caches the frame switching instruction and records the frame number of the image data issued by the HWC to the display driver. At the same time, the display driver can issue a TE frequency adjustment instruction to the DPU through channel 1. Further, the DPU issues a TE frequency adjustment instruction to the DDIC, and the DDIC adjusts the TE frequency in response to the TE frequency adjustment instruction. When the cache frame number of the frame switching instruction reaches 2 frames, the display driver triggers the DPU to modify the MIPI signal period.

[0121] As shown in Figure 11 , if it is an LTPO screen, after the display driver receives the frame switching instruction issued by the HWC to switch from 90Hz to 60Hz, it first issues a frame switching instruction to switch from 90Hz to 120Hz to the DPU through channel 1, and then issues a frame switching instruction to switch from 120Hz to 60Hz to the DPU through channel 3 after one frame.

[0122] The process of the frame rate control method provided in the present application will be described in detail below with reference to the flowchart of the frame rate control method shown in Figure 12 . The present embodiment only takes the scenario of switching from a video APP to a reading APP shown in Figure 1 as an example to illustrate the frame rate control process. The reading APP can also be other applications for displaying text and images, such as APP, Zhihu APP, browser, etc., and the video APP can also be other applications for playing videos, such as other video players or video platform APPs, etc.

[0123] As shown in Figure 12 , the frame rate control process can include:

[0124] S100, in the initialization phase, the HWC issues a frame switching instruction to the display driver through the channel between the display driver and the HWC (such as channel 1 shown in Figure 7 ​The screen type of the electronic device and the preset frame number are obtained through channel 3) in the display driver.

[0125] The preset frame number is the frame number of the TE frequency change lagging behind the DDIC receiving the frame switching instruction. For example, some DDICs lag behind by 2 frames, and some DDICs lag behind by 1 frame. On the premise that the DDIC has been determined, the frame number of the TE frequency lagging behind can be determined, and the lagging frame number is usually taken as a performance parameter of the DDIC.

[0126] In an example embodiment, the screen type of the electronic device and the preset frame number are written in the code of the display driver, and the HWC reads the two information from the display driver in the operating system initialization stage of the electronic device.

[0127] In S101, the frame rate management module determines a first target frame rate after detecting that the video APP is started, so that the electronic device displays images at the first target frame rate.

[0128] After detecting that the video APP is started, the frame rate management module determines a target frame rate matching the video APP, i.e., the first target frame rate. Generally, the frame rate required by the video APP is higher, such as 120 Hz or 90 Hz, etc.

[0129] The business scenario can refer to the business of the APP currently running in the foreground of the electronic device.

[0130] For example, when the electronic device switches from the APP requiring the first frame rate (120 Hz) to the APP requiring the second frame rate (90 Hz), it is determined that the frame rate requirement of the business scenario changes.

[0131] In S102, the frame rate management module determines a second target frame rate matching the reading APP after detecting that the reading APP is started.

[0132] After the business scenario of the electronic device changes, the frame rate management module can analyze the frame rate requirement corresponding to the current business scenario, i.e., determine the target frame rate corresponding to the current business scenario. For example, the current business scenario is a graphic-text reading scenario, and the second target frame rate matching the reading scenario is determined.

[0133] In S103, the frame rate management module detects that the reading APP is started, and triggers the SurfaceFlinger to initiate frame rate switching.

[0134] In an example, the electronic device can trigger the SurfaceFlinger to initiate frame rate switching in the process of starting the reading APP.

[0135] In another example, the electronic device can trigger the SurfaceFlinger to initiate frame rate switching after starting the reading APP.

[0136] After the frame rate management module detects that the frame rate of the reading APP is different from the first target frame rate, the second target frame rate is sent to the SurfaceFlinger, and the SurfaceFlinger is triggered to initiate frame rate switching.

[0137] In S104, the SurfaceFlinger issues a frame switching instruction to the HWC, and the frame switching instruction includes the second target frame rate.

[0138] In S105, the HWC determines whether the screen of the electronic device is an LTPS screen or an LTPO screen; if it is an LTPS screen, S106 is executed; if it is an LTPO screen, S115 is executed.

[0139] After adding the step of determining the screen type in the frame rate switching processing logic, different frame rate switching processing logics do not need to be developed for different screen types, and the electronic device using the LTPS screen and the LTPO screen can use the same frame rate switching processing logic, thereby improving the application range of the frame rate switching control program and reducing the development workload of the developer.

[0140] Of course, in other embodiments of the present application, the screen type currently used by the electronic device does not need to be determined, and the screen type of the electronic device is determined when it is shipped, so that only the logical processing flow corresponding to the screen type can be executed. For example, for the electronic device with the LTPS screen, S106-S116 can be directly executed. For the electronic device with the LTPO screen, S117-S124 can be directly executed. In this way, only the program code part corresponding to the LTPS (or LTPO) screen needs to be installed in the electronic device with the LTPS screen, and the program code part corresponding to the LTPO (or LTPS) screen does not need to be installed, thereby saving the storage space occupied by the program code.

[0141] In S106, the HWC determines whether the current frame switching is from a high frame rate to a low frame rate; if yes, S107 is executed; if no, S113 is executed.

[0142] The HWC can obtain the running frame rate (i.e., the first target frame rate) corresponding to the previous service scene from the system, and can also obtain the second target frame rate from the frame switching instruction issued by the SurfaceFlinger. The HWC can determine whether the current frame switching instruction is from a high frame rate to a low frame rate or from a low frame rate to a high frame rate.

[0143] In S107, the HWC caches the frame switching instruction issued by the SurfaceFlinger, and counts the frame number of the image data issued by the SurfaceFlinger.

[0144] The HWC does not issue a frame cutting instruction after receiving the frame cutting instruction issued by the SurfaceFlinger, but buffers the frame cutting instruction. For example, the HWC stores the received frame cutting instruction in a memory space allocated for the HWC.

[0145] In S108, the HWC issues a TE frequency adjustment instruction to the display driver through the first channel, and the display driver issues the TE frequency adjustment instruction to the DDIC.

[0146] In an exemplary embodiment, S107 and S108 can be executed synchronously. Alternatively, S107 can be executed first and then S108 can be executed. For example, S107 is executed at time t1, and S108 is executed at time t2. t1 and t2 are different time points in the same period.

[0147] The first channel in the present embodiment refers to a newly added channel between the HWC and the display driver. The purpose of adding the first channel between the HWC and the display driver is to not affect the original frame cutting instruction processing logic. In this way, in the case where the current frame cutting is not from a high frame rate to a low frame rate, the original frame cutting processing logic can be directly used.

[0148] The HWC analyzes the received frame cutting instruction to obtain a target frame rate (i.e., a second target frame rate), which is a target frequency of the TE signal output by the DDIC, and issues a TE frequency adjustment instruction to the display driver through the first channel. The TE frequency adjustment instruction contains the target frequency.

[0149] The display driver continues to issue the received TE frequency adjustment instruction to the data processor (DPU), and the DPU continues to issue the TE frequency adjustment instruction to the DDIC.

[0150] In S109, the DDIC adjusts the frequency of the TE signal to a target frequency matching the target frame rate in response to the TE frequency adjustment instruction.

[0151] For example, as shown in FIG. 6, after the display driver issues the instruction to modify the TE frequency in the B frame, the DDIC modifies the TE frequency, and the modification of the TE frequency takes effect 2 frames later, i.e., the modification of the TE frequency takes effect in the E frame. Figure 8

[0152] In S110, the number of buffered frame cutting instructions reaches a preset number of frames, and a MIPI signal adjustment instruction is issued to the display driver through the second channel.

[0153] The number of buffered frame cutting instructions in the present embodiment refers to the number of image frames received by the HWC since the frame cutting instruction is buffered. The preset number of frames is the first number of frames described above.

[0154] ​After the HWC caches the frame-cut instruction, the counter is incremented by 1 each time image data sent by the SurfaceFlinger is received, and after the count value reaches the preset frame number, the HWC sends a MIPI signal adjustment instruction to the display driver.

[0155] For example, the preset frame number corresponding to the DDIC is 2 frames, as shown in the figure, the HWC caches the frame-cut instruction sent by the SurfaceFlinger, and when the next TE signal rising edge arrives (i.e. the TE signal of the C frame), the SurfaceFlinger sends image data to the HWC, at which time the counter is incremented by 1. Similarly, when the TE signal of the D frame arrives, the counter is incremented by 1 again, at which time the count value is 2, reaching the preset frame number, and a MIPI signal adjustment instruction is sent to the display driver through the second channel. Figure 8

[0156] In the embodiment, the second channel can be the original channel between the HWC and the display driver.

[0157] The function of the MIPI signal adjustment instruction is to make the display driver adjust the MIPI signal period of the data processor.

[0158] In an exemplary embodiment, the HWC sends an instruction (or notification) to the display driver to adjust the MIPI signal, and the MIPI signal adjustment instruction includes a target frame rate. The display driver can determine a target period of the MIPI signal that matches the target frame rate according to the target frame rate. Further, the period of the MIPI signal of the data processor is configured to be the target period. For example, the display driver can configure the period of the MIPI signal through a MIPI signal configuration interface.

[0159] In other embodiments of the present application, the HWC can also send the received frame-cut instruction directly to the display driver through the second channel.

[0160] In other embodiments of the present application, the TE frequency adjustment instruction and the MIPI signal adjustment instruction can also be sent to the display driver through the original channel (i.e. the second channel) between the HWC and the display driver, so that the processing logic of the display driver needs to be modified. For example, the display driver needs to distinguish whether the instruction received through the second channel is a TE frequency adjustment instruction or a MIPI signal adjustment instruction. For the TE frequency adjustment instruction, the display driver sends the instruction to the DDIC, so that the DDIC adjusts the frequency of the TE signal according to the instruction. For the MIPI signal adjustment instruction, the display driver reconfigures the MIPI signal period of the DPU.

[0161] S111, the display driver responds to the MIPI signal adjustment instruction and triggers the DPU to modify the MIPI signal period.

[0162] ​After the display driver receives the MIPI signal adjustment instruction, the MIPI signal period of the DPU needs to be reconfigured.

[0163] In S112, the MIPI signal modification and the TE frequency modification take effect synchronously.

[0164] The synchronous effect here means that the MIPI signal modification and the TE frequency modification take effect in the same period.

[0165] Since the TE frequency adjustment instruction is sent to the DDIC immediately after the HWC receives the frame cutting instruction, and the MIPI signal adjustment instruction is sent after the TE frequency adjustment instruction is sent for a preset number of frames (e.g., 2 frames), and the TE frequency adjustment instruction takes effect after 2 frames, and the MIPI signal takes effect immediately in the next screen refresh period (i.e., the next frame) after the MIPI signal adjustment instruction is sent, the TE frequency modification and the MIPI signal modification take effect in the same period.

[0166] As an example, as shown in Figure 8 the TE frequency adjustment instruction is sent in B frame, and the TE frequency adjustment takes effect in E frame. The frame cutting instruction is buffered in B frame, and the frame cutting instruction (or the MIPI signal adjustment instruction) is sent to the display driver after 2 frames, i.e., the MIPI signal adjustment instruction is sent in D frame. Moreover, the MIPI signal adjustment takes effect in the next frame after the MIPI signal adjustment instruction is sent, i.e., the MIPI signal adjustment in E frame takes effect. As can be seen, the TE frequency and the MIPI signal take effect synchronously in E frame.

[0167] In S113, the HWC sends the frame cutting instruction to the display driver through the second channel.

[0168] If the HWC determines that the current frame cutting instruction is to switch from a low frame rate to a high frame rate, the original frame cutting processing logic is executed, i.e., the frame cutting instruction is sent to the display driver through the original second channel between the HWC and the display driver.

[0169] In S114, the display driver adjusts the MIPI signal period of the DPU in response to the frame cutting instruction, and sends the frame cutting instruction to the DPU.

[0170] After the display driver receives the frame cutting instruction sent by the HWC, the frame cutting instruction is sent to the DDIC (e.g., the display driver sends the frame cutting instruction to the DPU, and the DPU further sends the frame cutting instruction to the DDIC through MIPI), and the MIPI signal period of the DPU is adjusted.

[0171] In S115, the DPU sends the frame cutting instruction to the DDIC through MIPI.

[0172] In S116, the DDIC adjusts the frequency of the TE signal in response to the frame cutting instruction to complete the frame rate switching.

[0173] For example, the current frame cutting is switched from 90Hz to 120Hz, and the DDIC adjusts the period of the TE signal to 8.3ms after receiving the frame cutting instruction, that is, the frame rate is adjusted to 120Hz.

[0174] In S117, the HWC determines whether the current frame cutting is switched from 90Hz to 60Hz, and if yes, S118 is executed; if not, S123 is executed.

[0175] If the HWC determines that the screen of the electronic device is an LTPO screen, it is further determined whether the current frame cutting is switched from 90Hz to 60Hz or 40Hz, 30Hz, etc., and if yes, S116-S120 are executed; if not, S121-S122 are executed.

[0176] In S118, the HWC caches the frame cutting instruction of switching from 90Hz to 60Hz, and counts the number of frames of the received image data.

[0177] After the HWC receives the frame cutting instruction of switching to 60Hz (i.e., the original frame cutting instruction) issued by the SurfaceFlinger, it does not send the frame cutting instruction to the display driver directly, but caches the received frame cutting instruction. In addition, the counter is incremented by 1 after receiving the image data issued by the SurfaceFlinger.

[0178] For example, as shown in FIG. 6, the HWC caches the received frame cutting instruction in B frame, and the counter is incremented by 1 after the next period TE signal arrives (i.e., the TE signal in C frame arrives). Figure 9

[0179] In S119, the HWC issues the first frame cutting instruction of switching to 120Hz to the display driver through the first channel, and the display driver issues the first frame cutting instruction to the DDIC.

[0180] While the HWC caches the original frame cutting instruction, the frame cutting instruction of switching to 120Hz is issued to the display driver through the first channel. Further, the display driver issues the first frame cutting instruction to the DPU through the channel between the display driver and the DPU, and the DPU sends the first frame cutting instruction to the DDIC through the MIPI.

[0181] In the embodiments of the present application, the first channel can be a newly added channel between the HWC and the display driver, so that the original frame rate control processing logic does not need to be modified.

[0182] In S120, the DDIC adjusts the frequency of the TE signal in response to the first frame cutting instruction.

[0183] As shown in FIG. 7, the HWC caches the received frame cutting instruction in B frame, and the counter is incremented by 1 after the next period TE signal arrives (i.e., the TE signal in C frame arrives). Figure 9 ​As shown, after HWC sends the first frame-switching command to the display driver to switch to 120Hz in frame B, the high-level portion of the TE signal in the next frame (i.e., frame C) cannot be adjusted in time and remains at 2.8ms. The low-level portion of the TE signal is adjusted to the period of 8.3ms corresponding to 120Hz. Therefore, frame C is still 90Hz. This continues until the high-level portion of the TE signal in frame D is adjusted to a narrow level.

[0184] S121, after buffering one frame of the 90Hz to 60Hz frame switching instruction, sends a second frame switching instruction to 60Hz to the display driver through the second channel, and the display driver sends a second frame switching instruction to the DDIC.

[0185] For example, such as Figure 9 As shown, after HWC issues a frame-switching command to 120Hz in frame B, it triggers a counter to increment by 1 after receiving image data in frame C. At this point, the counter value reaches 1, and HWC issues a buffered frame-switching command to 60Hz to the display driver. The display driver then further issues a frame-switching command to DDIC to 60Hz.

[0186] In other embodiments of this application, the first frame-cutting instruction and the second frame-cutting instruction can both be sent to the display driver through the existing channel between HWC and the display driver, and this application does not limit this.

[0187] In other embodiments of this application, the 90Hz to 60Hz frame switching command may not be cached. Instead, it is sufficient to record the screen refresh cycle in which the HWC receives the 90Hz to 60Hz frame switching command, and to count the number of screen refresh cycles after receiving the command. Once the counted number of screen refresh cycles is 1, a second frame switching command is generated and sent to the display driver.

[0188] S122, DDIC responds to the second frame-switching command and adjusts the TE signal frequency.

[0189] After receiving the second frame-splitting command, the DDIC adjusts the frequency of the TE signal. For example, as... Figure 9 As shown, in frame B, the DDIC was triggered to adjust the TE frequency from 90Hz to 120Hz. This continued until frame D, when the high level of the TE signal was adjusted to a narrow level, further responding to the second frame-cutting command. This adjusted the duration of the low-level portion of the TE signal in frame D to the period corresponding to 60Hz, i.e., 16.6ms. Therefore, after the frame-cutting command to 60Hz is issued in frame C, the frame rate of the next frame (i.e., frame D) can be adjusted to 60Hz.

[0190] Moreover, this scheme avoids the frame loss phenomenon caused by the failure of the presentFence of the C frame to be released in time, which prevents the synthesis of D frame image data.

[0191] S123, the HWC sends a frame switching instruction to the display driver through the second channel, and the display driver sends the frame switching instruction to the DDIC.

[0192] If the current frame switching is from 120Hz to 60Hz, the HWC determines that the current frame switching is not from 90Hz to 60Hz, and executes the original frame switching process, that is, the HWC sends a frame switching instruction to the display driver through the original channel between the HWC and the display driver. Further, the display driver sends a frame switching instruction to the DPU, and finally the DPU sends the frame switching instruction to the DDIC through the MIPI interface.

[0193] S124, the DDIC adjusts the frequency of the TE signal to the target frame rate in response to the frame switching instruction.

[0194] Still taking the current frame switching from 120Hz to 90Hz as an example, after the DDIC receives the frame switching instruction, it determines that the target frame rate is 90Hz, and then lengthens the duration of the high level part of the TE signal to 2.8ms, and the duration of the low level part remains unchanged at 8.3ms, and the total duration is 2.8+8.3=11.1ms, that is, the period of the TE signal is adjusted to the duration corresponding to 90Hz.

[0195] Of course, the frame rate control method provided in the present application is also applicable to other frame rate switching scenarios. For example, a mobile phone displays different content types of pages in the same application running process, such as a page displaying a video and a page displaying a text. For another example, a mobile phone displays a same page at different sliding speeds in the same application running process, for example, the frame rate is 60Hz when sliding the first page at a first sliding speed, and the frame rate is 10Hz when the first page is static.

[0196] The frame rate control method provided in the present embodiment, after the HWC receives the frame switching instruction, first determines the screen type of the electronic device, if it is an LTPS screen, then further judges whether the current frame switching is from a high frame rate to a low frame rate, if it is, then buffers the received frame switching instruction, and counts the frame number of the image data received by the SurfaceFlinger. Further, the frame switching instruction is divided into two steps for execution: first, send a TE frequency adjustment instruction to adjust the frequency of the TE signal in time. When the count value reaches the preset frame number, send the MIPI signal adjustment instruction to adjust the MIPI signal period. The TE frequency adjustment instruction takes effect with a lag, while the MIPI signal adjustment instruction takes effect in the next frame. Therefore, the TE frequency adjustment instruction is sent first, and the MIPI signal adjustment instruction is sent after the preset frame, which can ensure that the TE frequency and the MIPI signal take effect in the same screen refresh period, and finally avoids the display abnormal problem caused by the unsynchronized taking effect of the TE frequency and the MIPI signal in the frame switching process, and improves the user's viewing experience in the frame rate switching process

[0197] If it is an LTPO screen, and it is determined that the current frame cutting is switched from 90Hz to 60Hz (or 40Hz, 30Hz, etc. lower frame rate), the frame cutting instruction is split into two steps: first, issue a frame cutting instruction to switch to 120Hz, and then issue a frame cutting instruction to switch to 60Hz after 1 frame. For example, the frame cutting instruction to switch to 120Hz is issued at the 1st frame, but the duration of the high level part of the TE signal cannot be adjusted in time, so the period of the 2nd frame is still 2.8+8.3=11.1ms, that is, the frame rate is still 90Hz. Until the high level part of the TE signal of the next frame (i.e. the 3rd frame) is adjusted to be negligible. After issuing the frame cutting instruction to switch to 60Hz at the 2nd frame, the high level part of the TE signal of the 3rd frame has returned to the narrow level, and only needs to calculate the arrival time of the high level of the TE signal of the next period according to 60Hz, so the TE signal period of the 3rd frame is adjusted to 16.6ms, which is the period corresponding to 60Hz. This scheme first issues a frame cutting instruction to switch to 120Hz, so that the next frame still maintains a frame rate of 90Hz, avoiding the frame loss phenomenon when directly switching from 90Hz to 60Hz. After 1 frame, the frame cutting instruction to switch to 60Hz is issued again, so that the frame rate is successfully switched to 60Hz.

[0198] As shown in Figure 13 , the embodiment of the present application also provides a flowchart of another frame rate control method. The difference between this method and the embodiment shown in Figure 12 is that the frame rate control processing flow provided by the present embodiment is mainly executed by the display driver.

[0199] As shown in Figure 13 , the method can include the following steps:

[0200] S200, in the initialization stage, the display driver obtains a preset frame number and a screen type of the electronic device.

[0201] S201, after the frame rate management module detects that the video APP is started, a first target frame rate is determined, so that the electronic device runs at the first target frame rate.

[0202] S202, after the frame rate management module detects that the reading APP is started, a second target frame rate matched with the reading APP is determined.

[0203] S203, the frame rate management module detects that the business scenario is switched from the video scenario to the reading scenario, and triggers the SurfaceFlinger to initiate frame rate switching.

[0204] S204, the SurfaceFlinger issues a frame cutting instruction to the HWC, and the frame cutting instruction includes the second target frame rate.

[0205] In the present embodiment, the second target frame rate is the frame rate corresponding to the reading scenario.

[0206] The implementation process of S200-S204 of this embodiment is the same as that of S100-S104 in Figure 12 , which will not be described here again.

[0207] S205, the HWC issues a frame cutting instruction to the display driver.

[0208] The frame cutting processing logic of the HWC and the channel between the HWC and the display driver in this embodiment are not modified. After receiving the frame cutting instruction, the HWC directly issues it to the display driver through the original channel.

[0209] S206, the display driver determines whether the screen of the electronic device is an LTPS screen or an LTPO screen; if it is an LTPS screen, S207 is executed; if it is an LTPO screen, S216 is executed.

[0210] S207, the display driver determines whether the current frame rate switching is from a high frame rate to a low frame rate; if yes, S208 is executed; if no, S213 is executed.

[0211] S208, the display driver caches the frame cutting instruction issued by the HWC and counts the frame number of the image data issued by the HWC.

[0212] S209, the display driver issues a TE frequency adjustment instruction to the DPU through the first channel, and the DPU issues a TE frequency adjustment instruction to the DDIC.

[0213] In this embodiment, the first channel (i.e., channel 1 in Figure 10 ) is a newly added communication channel between the display driver and the DPU. Any communication channel in this embodiment can be a software interface.

[0214] S210, the DDIC adjusts the frequency of the TE signal to a target frequency matching the target frame rate in response to the TE frequency adjustment instruction.

[0215] S211, the cached frame number of the frame cutting instruction reaches a preset frame number, and the MIPI signal period of the DPU is configured.

[0216] In this embodiment, the display driver configures the MIPI signal period of the DPU through the second channel between itself and the DPU. The second channel (i.e., channel 2 in Figure 10 ) is the original channel between the display driver and the DPU, which is used to issue the MIPI signal period of the DPU.

[0217] S212, the MIPI signal modification and the TE frequency modification are synchronized to take effect.

[0218] S206-S212 in this embodiment are the same as S106-S112 in Figure 12The implementation processes of S105-S112 are the same, and the difference is that S206-S209 and S211 are executed by the display driver instead of the HWC.

[0219] S213, the display driver modifies the MIPI signal period in response to the frame cutting instruction, and issues the frame cutting instruction to the DPU.

[0220] In this embodiment, the display driver can issue the frame cutting instruction to the DPU through the third channel (i.e., channel 3 in Figure 10 ).

[0221] The third channel can be the original channel between the display driver and the DPU, which is used to transmit the frame cutting instruction.

[0222] S214, the DPU issues the frame cutting instruction to the DDIC.

[0223] S215, the DDIC adjusts the frequency of the TE signal in response to the frame cutting instruction to complete the frame rate switching.

[0224] The implementation processes of S213-S215 in this embodiment are the same as those of S113-S116 in Figure 12 , which will not be described here.

[0225] S216, the display driver determines whether the current frame cutting is from 90Hz to 60Hz; if so, S217 is executed; if not, S222 is executed.

[0226] If the current screen is an LTPO screen, it is further determined whether the current frame rate switching is from 90Hz to 60Hz (or 40Hz, 30Hz, 20Hz).

[0227] S217, the display driver caches the original frame cutting instruction, and counts the number of frames of the received image data.

[0228] S218, the display driver issues a first frame cutting instruction to cut to 120Hz to the DPU, and the DPU issues the first frame cutting instruction to the DDIC.

[0229] In this embodiment, the display driver can issue the frame cutting instruction to cut to 120Hz to the DPU through the first channel, which is channel 1 between the display driver and the DPU as shown in Figure 11 .

[0230] S219, the DDIC adjusts the frequency of the TE signal in response to the first frame cutting instruction.

[0231] S220, after caching one frame of the frame cutting instruction, the display driver issues a second frame cutting instruction to cut to 60Hz to the DPU, and the DPU issues the second frame cutting instruction to the DDIC.

[0232] In the embodiment, the display driver issues the frame switching instruction to the DPU through channel 3 between the display driver and the DPU. Figure 11 The display driver issues the frame switching instruction to the DPU through channel 3 between the display driver and the DPU.

[0233] S221, the DDIC adjusts the frequency of the TE signal in response to the second frame switching instruction.

[0234] S222, the display driver issues the frame switching instruction to the DPU.

[0235] In the embodiment, the display driver issues the frame switching instruction to the DPU through channel 3 between the display driver and the DPU. Figure 11 The display driver issues the frame switching instruction to the DPU through channel 3 between the display driver and the DPU.

[0236] S223, the DPU issues the frame switching instruction to the DDIC.

[0237] S224, the DDIC adjusts the frequency of the TE signal to the target frame rate in response to the frame switching instruction.

[0238] S216-S224 of the embodiment are the same as the implementation process of S117-S124 of the embodiment, except that S216 and S217 are performed by the display driver instead of the HWC, which will not be described herein. Figure 12

[0239] The frame rate control method provided in the embodiment, after the display driver receives the frame switching instruction, first determines the screen type of the electronic device, if it is an LTPS screen, further judges whether this frame switching is from high frame rate to low frame rate, if yes, caches the received frame switching instruction, and counts the image data frame sent by the HWC. The display driver divides the frame switching process into two steps, that is, first issues the TE frequency adjustment instruction to adjust the frequency of the TE signal. When the count value reaches the preset frame number, the MIPI signal adjustment instruction is issued to adjust the MIPI signal period. In this way, the TE frequency and the MIPI signal can be effective in the same period, which finally avoids the display abnormal problem caused by the unsynchronized effect of the TE frequency and the MIPI signal in the frame switching process.

[0240] If it is an LTPO screen, it is judged whether this frame switching is from 90Hz to 60Hz, if yes, the frame switching instruction is also divided into two steps: first issue the frame switching instruction to switch to 120Hz, and then issue the frame switching instruction to switch to 60Hz after 1 frame. By first issuing the frame switching instruction to switch to 120Hz, the next frame still maintains the frame rate of 90Hz, avoiding the frame loss phenomenon when directly switching from 90Hz to 60Hz. The frame switching instruction to switch to 60Hz is issued after 1 frame, so that the frame rate is successfully switched to 60Hz.

[0241] ​Those skilled in the art can clearly understand the technical solutions of the present application from the above description of the embodiments, and the above-described system, device and unit can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0242] In several embodiments provided in the present embodiment, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, and the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0243] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0244] In addition, each functional unit in each embodiment of the present embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0245] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present embodiment essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0246] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A frame rate control method, characterized in that, Applied to electronic devices employing LTPS screens, the method includes: Receive the user's first action; In response to the first operation, the first application is launched; During the operation of the first application, image frames are transmitted based on the first MIPI signal and displayed based on the first TE signal, wherein the frequency of the first TE signal is the same as the first frame rate. Receive the user's second operation; In response to the second operation, a second application is launched, and a frame-slicing instruction is generated; Cache the frame-slicing instructions; A second TE signal is generated based on the first TE signal; When the number of image frames received by the hardware synthesizer of the electronic device that caches the frame-cutting instruction reaches a first number, a second MIPI signal is generated based on the first MIPI signal; Image frames are transmitted based on the second MIPI signal and displayed based on the second TE signal. The frequency of the second TE signal is the same as the second frame rate. The second frame rate is different from the first frame rate, and the first frame rate is greater than the second frame rate.

2. The method according to claim 1, characterized in that, If the number of image frames received by the hardware synthesizer of the electronic device, which caches the frame-cutting instructions, reaches a first number, the method further includes, before generating a second MIPI signal based on the first MIPI signal: After caching the frame-cutting instruction, the number of image frames received by the hardware synthesizer of the electronic device is counted; When the count value reaches the first number, it is determined that the number of image frames received by the hardware synthesizer of the electronic device that caches the frame-cutting instruction has reached the first number.

3. The method according to claim 1 or 2, characterized in that, Before receiving the user's second action, the method further includes: Receive the user's third operation; In response to the third operation, close the first application or return to the desktop.

4. The method according to claim 1 or 2, characterized in that, The first application is a video application, and the second application is an application that displays images and text.

5. The method according to claim 1, characterized in that, When the number of image frames received by the hardware synthesizer of the electronic device and cached with the frame-cutting instruction reaches a first number, a second MIPI signal is generated based on the first MIPI signal, including: When the number of image frames received by the hardware synthesizer and cached frame-cutting instructions reaches a first number, the frame-cutting instructions are parsed to obtain the second frame rate. Based on the mapping relationship between frame rate and MIPI signal period, a first period matching the second frame rate is obtained; Based on the first period, a second MIPI signal is generated using the first MIPI signal. The period of the second MIPI signal is equal to the first period, and the first period is greater than the period of the first MIPI signal.

6. The method according to claim 1, 2, or 5, characterized in that, Generating a second TE signal based on the first TE signal includes: The second frame rate is obtained by parsing the frame-slicing command; Based on the second frame rate, a TE frequency adjustment command is generated, and in response to the TE frequency adjustment command, a second TE signal is generated based on the first TE signal.

7. A frame rate control method, characterized in that, Applied to electronic devices employing LTPS screens, the method includes: Receive the user's first action; In response to the first operation, the first application is launched; During the operation of the first application, image frames are transmitted based on the first MIPI signal and displayed based on the first TE signal, wherein the frequency of the first TE signal is the same as the first frame rate. Receive the user's second operation; In response to the second operation, the second application is launched; In the first cycle, a second TE signal is generated based on the first TE signal; In the second cycle, a second MIPI signal is generated based on the first MIPI signal. The second cycle is located after the first cycle and is spaced apart by a preset number of first cycle intervals, wherein the size of the first cycle interval is the reciprocal of the first frame rate. In the third cycle, image frames are transmitted based on the second MIPI signal and displayed based on the second TE signal. The third cycle is located after the second cycle. The frequency of the second TE signal is the same as the second frame rate. The second frame rate is different from the first frame rate, and the first frame rate is greater than the second frame rate.

8. A frame rate control method, characterized in that, Applied to electronic devices employing LTPO screens, the method includes: Receive the user's first action; In response to the first operation, the first application is launched; During the operation of the first application, image frames are displayed at a first frame rate of 90Hz. Receive the user's second operation; In response to the second operation, a second application is launched and an original frame-cutting instruction is generated, the original frame-cutting instruction being used to cause the electronic device to display image frames at a second frame rate, the second frame rate being less than the first frame rate; Generate a first frame-cutting instruction, and in response to the first frame-cutting instruction, adjust the first frame rate to a third frame rate, wherein the third frame rate is 120Hz; A second frame-slicing instruction is generated in the next screen refresh cycle following the generation of the original frame-slicing instruction; In response to the second frame-slicing command, the third frame rate is adjusted to the second frame rate.

9. The method according to claim 8, characterized in that, The step of generating a second frame-slicing instruction in the next screen refresh cycle after generating the original frame-slicing instruction includes: After generating the original frame-cutting instruction, the original frame-cutting instruction is cached; When the number of image frames received by the hardware synthesizer of the electronic device that caches the original frame-cutting instruction reaches 1 frame, a second frame-cutting instruction is generated.

10. The method according to claim 8 or 9, characterized in that, Before receiving the user's second action, the method further includes: Receive the user's third operation; In response to the third operation, close the first application or return to the desktop.

11. The method according to claim 8 or 9, characterized in that, The first application is a video application, and the second application is an application that displays images and text.

12. An electronic device, characterized in that, The electronic device includes: one or more processors, a memory, and a touch screen; the memory is used to store program code; the processor is used to run the program code, causing the electronic device to implement the frame rate control method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, It stores instructions that, when executed on an electronic device, cause the electronic device to perform the frame rate control method as described in any one of claims 1 to 11.

Images