Display method, electronic equipment and computer readable storage medium

Abstract

The invention provides a display method, electronic equipment for realizing the display method and a computer readable storage medium. In the method, after it is detected that an SOC sends image data of a first picture to a display screen, the electronic device starts to execute first frame reduction based on a first frame reduction sequence. The first frame reduction comprises n times of screen refresh, and the equivalent refresh rate of the ith screen refresh is equal to the ith refresh rate in the frame reduction sequence. N screen refresh rates in the first reduced frame sequence are arranged in a descending order, the first screen refresh rate can be determined according to a time interval between a first refresh frame and a previous refresh frame, and the first refresh frame is the time when the display screen refreshes the first picture for the first time; the last refresh frame is the last screen refresh time before the first refresh frame. In this way, the power consumption of the display screen can be reduced through the self-adaptive frame reduction process of the frame reduction starting point, and meanwhile the problems of screen flicker and the like can be solved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a display method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of display technology, in order to display the picture smoothly, the screen refresh rate supported by electronic devices such as mobile phones and tablets has been continuously improved. However, the power consumption of the display screen poses a great challenge to the power saving or battery life of the device. In recent years, display screens that support refresh rates reduced to low refresh rates or even ultra-low refresh rates have gradually been introduced into the market. For example, low-temperature polycrystalline oxide (LTPO) display screens support refresh rates reduced to 1Hz. In order to save power consumption, electronic devices using such displays can adopt dynamic refresh rates (or variable refresh rates), allowing the device to reduce the refresh rate in usage scenarios such as when the user does not click or slide the screen. However, user usage scenarios are complex and changeable, and difficult to predict. The solution of dynamically adjusting the refresh rate based on user usage scenarios has great limitations. Summary of the invention

[0003] In a first aspect, an embodiment of the present application provides a display method, which is applied to an electronic device, and the electronic device includes a picture sending unit and a display screen; the method may include: the electronic device detects that the picture sending unit sends image data of a first picture to the display screen; the electronic device starts to execute the first frame reduction based on the first frame reduction sequence. Wherein, the first frame reduction sequence is composed of n screen refresh rates arranged in descending order, n≥2, and n is a positive integer; the first screen refresh rate in the first frame reduction sequence is determined according to the interval frame number K from the first refresh frame to the previous refresh frame of the first refresh frame, and K is a positive integer; the first refresh frame refers to the frame in which the display screen refreshes the first picture for the first time. Wherein, the first frame reduction includes refreshing the first picture n times using the image data of the first picture, and the n refreshes are executed according to the n screen refresh rates, and the time interval between two adjacent refreshes becomes larger and larger as the first frame reduction proceeds.

[0004] In the first aspect, the image sending unit can be a system on chip SOC (integrated with CPU, GPU, etc.), or it can be other processors or processor systems, chips or chip systems that can generate image data for one picture after another through rendering, synthesis and other operations, and is responsible for sending images to the display screen.

[0005] By implementing the method of the first aspect, the electronic device can gradually reduce the screen refresh rate while improving problems such as image retention.

[0006] In combination with the first aspect, in some embodiments, the equivalent refresh rate of the i-th screen refresh in the first frame drop is equal to the i-th refresh rate f in the first frame drop sequence.i , i is a positive integer, i≤n. The equivalent refresh rate of the i-th screen refresh is related to the time interval from the i-th screen refresh to the i+1-th screen refresh. The larger the time interval, the smaller the equivalent refresh rate of the i-th screen refresh; the smaller the time interval, the larger the equivalent refresh rate of the i-th screen refresh.

[0007] In combination with the first aspect, in some embodiments, the equivalent refresh rate of the i-th screen refresh may be defined as: Among them, f base The reciprocal of the single frame duration (e.g., 1 / 120 second); m represents the number of frames between the i-th screen refresh and the i+1-th screen refresh. In this article, a frame is a time concept, for example, 1 / 120 second is a frame.

[0008] In combination with the first aspect, in some embodiments, n refreshes are performed according to n screen refresh rates, which may specifically include: the time T of the i-th refresh in the first drop frame i for: f i is the i-th refresh rate in the first down-frame sequence, f base is the reciprocal of the single frame duration, T i-1 is the time of the i-1th refresh; T i , T i-1 The time unit is frame; n≥i≥2, i is a positive integer. In this way, as i increases, the time interval between two adjacent screen refreshes increases, thereby gradually reducing the screen refresh rate.

[0009] In combination with the first aspect, in some embodiments, before the electronic device starts to execute the first frame reduction based on the first frame reduction sequence, the display method may also include: the electronic device determines whether there is a second frame reduction, the second frame reduction is an ongoing frame reduction, and if so, stops the second frame reduction.

[0010] In combination with the first aspect, in some embodiments, the display method may further include: during the first frame reduction process, the electronic device detects whether the image sending unit has a second image sent, and if so, stops the first frame reduction and starts the third frame reduction. In this way, the screen refresh rate can respond to the image sending of the SOC in a timely manner, ensuring the normal use of the electronic device by the user.

[0011] The third frame reduction is performed based on the second frame reduction sequence, the second frame reduction sequence is composed of m screen refresh rates arranged in descending order, m≥2, and m is a positive integer; the first screen refresh rate in the second frame reduction sequence is determined according to the interval frame number from the second refresh frame to the previous refresh frame of the second refresh frame; the second refresh frame refers to the frame in which the display screen refreshes the second picture for the first time. The third frame reduction may include refreshing the second picture m times using the image data of the second picture, the m refreshes are performed according to the m screen refresh rates, and the time interval between two adjacent refreshes becomes larger and larger as the third frame reduction proceeds.

[0012] In combination with the first aspect, in some embodiments, a parameter that can reflect the size of the interval frame number K is also introduced, which is the equivalent refresh rate f(j-1) of the last refresh before the frame reduction starts, and the determination rule of the frame reduction starting point is designed using it. Specifically, the first screen refresh rate in the first frame reduction sequence is determined according to the interval frame number K from the first refresh frame to the previous refresh frame of the first refresh frame, which may specifically include: the first screen refresh rate in the first frame reduction sequence is determined according to the equivalent refresh rate f(j-1) when the display screen is refreshed in the previous refresh frame of the first refresh frame, and f(j-1) is calculated and determined by the following formula: Among them, f base It is the reciprocal of the duration of a single frame.

[0013] Compared with designing the frame drop starting point determination rule directly based on the interval frame number K, using f(j-1) to design the rule can be applied to more scenarios, such as f base The scene changes.

[0014] In combination with the first aspect, in some embodiments, the first screen refresh rate in the first frame reduction sequence is determined according to the equivalent refresh rate f(j-1) when the display screen is refreshed in the previous refresh frame of the first refresh frame, which may specifically include:

[0015] If f(j-1) falls within the first segmentation interval, determining the first screen refresh rate in the first frame reduction sequence to be a first value, and the first value is set to be less than or equal to the minimum value of the first segmentation interval;

[0016] If f(j-1) falls within the second segmentation interval, the first screen refresh rate in the first frame reduction sequence is determined to be a second value, and the second value is set to be greater than or equal to the maximum value of the second segmentation interval;

[0017] The value range of f(j-1) is divided into one or more first segmented intervals and one or more second segmented intervals, and the refresh rate of the first segmented intervals is lower than the refresh rate of the second segmented intervals.

[0018] In the embodiment of the present application, the frame reduction starting point corresponding to the first segmented interval can ensure that the refresh rate of the display screen will not be too high when the SOC continuously sends images in this interval, so that the SOC image sending rate is basically consistent with the refresh rate of the display screen; and the refresh rate will not jump during the frame reduction process, which can improve the screen flickering problem. This will be explained later with examples.

[0019] In the embodiment of the present application, the frame reduction starting point corresponding to the second segmentation interval can prevent the display screen from being at a low refresh rate for a long time, improve the screen retention problem, and enable the display screen to quickly adapt to the high SOC image delivery rate that may occur at any time, prepare for restoring the high refresh rate, reduce the jump from low refresh rate to high refresh rate, and improve the screen flicker problem. This will be explained later with examples.

[0020] In combination with the first aspect, in some embodiments, the value range of f(j-1) may be [0,f max ], where f max is the maximum refresh rate of the display, for example 120Hz.

[0021] In combination with the first aspect, in some embodiments, the second segmentation interval may be [0,f thre ), where f thre is the refresh rate threshold of the display screen; if the display screen continues to work for a certain period of time (such as 5 seconds) at the refresh rate threshold, image retention will occur. thre (such as 30Hz) as the dividing point between the first segmentation interval and the second segmentation interval can further help improve the image retention problem.

[0022] In combination with the first aspect, in some embodiments, fthre may be equal to 30 Hz. The value range of f(j-1) is divided into the following segmented intervals: [0,30), 30, (30,60), 60, (60,120), 120, wherein [0,30) is the second segmented interval, and 30, (30,60), 60, (60,120), 120 are the first segmented intervals.

[0023] In combination with the first aspect, in some embodiments, the first value is set to 60; if f(j-1) falls in one of the second segmentation intervals of 30, (30, 60), and 60, the second value is set to 30; if f(j-1) falls in one of the second segmentation intervals of (60, 120), and 120, the second value is set to 60. In this way, by uniformly setting the frame drop starting points of the second segmentation intervals of 30, (30, 60), and 60 to 30, and uniformly setting the frame drop starting points of the second segmentation intervals of (60, 120), and 120 to 60, it is more convenient to determine the frame drop starting point by looking up a table.

[0024] In combination with the first aspect, in some embodiments, the image sending unit may be a system on chip SOC. The interval frame number K may be determined by the following steps: the SOC records the time when a refresh instruction is sent to a display driver integrated circuit DDIC connected to the display screen, and determines that the interval frame number K is equal to the interval frame number from the first frame to the second frame, the first frame is the time when the SOC sends a refresh instruction for refreshing the first screen for the first time, and the second frame is the time when the SOC sends a refresh instruction for the last time before the first frame.

[0025] In combination with the first aspect, in some embodiments, the interval frame number K can be determined by the following steps: the display driver integrated circuit DDIC connected to the display screen records the time when the display screen performs screen refresh, and determines that the interval frame number K is equal to the interval frame number from the third frame to the fourth frame, the third frame is the refresh time when the display screen refreshes the first picture for the first time, and the fourth frame is the time when the display screen performs the last refresh before the third frame.

[0026] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories; wherein the memories are coupled to the processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect, or any possible implementation method of the first aspect.

[0027] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions so that the electronic device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0029] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 An electronic device provided in an embodiment of the present application is shown;

[0031] Figure 2 A subsystem for realizing a display function in an electronic device is shown;

[0032] Figure 3An exemplary process of a frame reduction sequence being fully executed is shown;

[0033] Figure 4 A schematic diagram showing the overall process of the display method provided in an embodiment of the present application;

[0034] Figure 5A It shows the situation where the first down frame is not interrupted by the SOC image sending;

[0035] Figure 5B It shows the situation where the first down frame is interrupted by the SOC sending picture;

[0036] Fig. 6A A frame reduction process using a fixed frame reduction starting point is shown;

[0037] Figure 6B Another frame reduction process using a fixed frame reduction starting point is shown;

[0038] Figure 7 A graphical representation corresponding to the frame drop start point lookup table is shown;

[0039] Figure 8 Another graphical representation corresponding to the frame drop start point lookup table is shown;

[0040] Fig. 9 The equivalent refresh rate of the last refresh falls on Figure 7 The frame drop process during the middle B segment;

[0041] Fig.10 The equivalent refresh rate of the last refresh falls on Figure 7 The frame drop process during the middle A segment;

[0042] Fig.11 The equivalent refresh rate of the last refresh falls on Figure 7 The frame drop process during the middle C segment;

[0043] Fig.12 The process of frame downgrading and SOC image delivery rate adaptation with adaptive frame downgrading starting point is shown;

[0044] Fig.13 A plurality of functional units in an electronic device for implementing the display method provided in an embodiment of the present application are shown;

[0045] Fig.14 Shows Fig.13 An integrated architecture of the functional units shown;

[0046] Fig.15 Shows Fig.13 Another integrated architecture of the functional units shown;

[0047] Fig.16An example of image refresh without frame buffer is shown. DETAILED DESCRIPTION

[0048] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0049] The embodiment of the present application also provides an electronic device for executing the display method.

[0050] Figure 1 An electronic device 100 provided in an embodiment of the present application is shown.

[0051] In the embodiment of the present application, the device type of the electronic device 100 can be a mobile phone, a tablet computer, a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), as well as smart home devices such as smart large screens and smart speakers, wearable devices such as smart bracelets, smart watches, and smart glasses, extended reality (extended reality, XR) devices such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), vehicle-mounted devices or smart city devices, and the like.

[0052] like Figure 1 As shown, the electronic device 100 may include: a processor 110, a memory 120, a display screen 130, a display driver integrated circuit (DDIC) 140, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a gyroscope sensor 180B, an acceleration sensor 180E, and a touch sensor 180K, etc. The various components in the electronic device 100 may be connected through a bus.

[0053] The processor 110 may be one or more, and they may be integrated into an integrated circuit of a system on chip (SOC). SOC is a system-on-chip. The processor 110 may include a central processing unit (CPU) and a graphic processing unit (GPU). The CPU may be an application processor (AP). The CPU and GPU may be used to render and synthesize images to be sent to the display screen 130. The processor 110 may also include a neural-network processing unit (NPU), a modem processor, and the like.

[0054] The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input and output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0055] The processor 110 may be provided with a cache memory for storing instructions or data just used or cyclically used by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the cache memory, which can reduce the waiting time of the processor 110 and improve the program running efficiency.

[0056] The memory 120 may include a program storage area and a user data storage area, wherein the program storage area may store an operating system and one or more application programs (such as game applications), and the data storage area may store data (such as photos, contacts) created by the user when using the electronic device 100. The memory 120 may be a high-speed random access memory or a non-volatile memory, such as a disk, a flash memory, a universal flash storage (UFS), etc. The memory 120 may also be an external memory card, such as a Micro SD card.

[0057] The memory 120 may also store code instructions of the display method provided in the embodiment of the present application. When the processor 110 reads the code instructions from the memory 120 and runs the code instructions, the electronic device 100 may execute the display method.

[0058] The memory 120 may also be integrated with the processor 110 in an integrated circuit of a SOC.

[0059] like Figure 2 As shown, the electronic device 100 can realize the display function through the SOC, the DDIC 140, and the display screen 130.

[0060] The display screen 130 has multiple refresh rates. The refresh rate indicates the number of times the display screen refreshes the display screen in 1 second. For example, a refresh rate of 60 Hz indicates that the display screen refreshes the display screen 60 times in 1 second. The display screen 130 may use an LTPO display panel, allowing the refresh rate to be reduced to a low refresh rate, such as 10 Hz or 1 Hz, thereby supporting the reduction of the power consumption of the display screen.

[0061] Among them, the display driver integrated circuit (DDIC) 140 can be used as the control core of the display screen 130, driving the display screen 130 to work, and receiving data from the SOC (processor 110), such as image data and some instructions. The DDIC 140 can send driving signals and data to the display panel of the display screen 130 in the form of electrical signals, and then realize the control of the screen brightness and color, so that image information such as letters and pictures can be displayed on the screen, completing the screen refresh.

[0062] The image data of the screen to be displayed sent by the SOC to the DDIC 140 can be sent to the frame buffer for storage to complete the display (or image sending). Then, the DDIC 140 takes out the image data from the frame buffer and drives the display screen 130 to display.

[0063] Some instructions sent by the SOC to the DDIC 140 may include picture sending refresh instructions, non-picture sending refresh instructions, etc. Among them, the picture sending refresh instruction can be used to trigger the DDIC 140 to drive the display screen 130 to perform picture sending refresh. The non-picture sending refresh instruction can be used to trigger the DDIC 140 to drive the display screen 130 to perform non-picture sending refresh. Picture sending refresh means that after receiving the image data of the picture to be displayed from the SOC, the DDIC 140 drives the display screen 130 to refresh and display the picture. Non-picture sending refresh means that the SOC does not send the picture, but the DDIC controls the display screen to perform refresh according to the screen refresh rate, and usually requires the old picture in the frame buffer to perform frame insertion refresh. Non-picture sending refresh generally occurs when the screen refresh rate is higher than the SOC picture sending rate.

[0064] The SOC image delivery rate is also called the image delivery frame rate, which is the number of frames that the SOC sends to display the screen in 1 second. For example, 60fps means that the SOC sends 60 pictures in 1 second. In order to facilitate the size comparison with the screen refresh rate, the definition of the SOC image delivery rate can be transformed into the number of times the SOC sends the picture in 1 second. For example, 60 hertz (Hz) means that the SOC sends 60 times in 1 second, one picture at a time. The SOC image delivery rate can be changed. For example, when user interactions such as user clicks or slides the screen occur, the SOC image delivery rate is large to ensure timely response to user operations. For another example, when the user stays on a static display content, such as a page of an e-book, the SOC image delivery rate is small. For another example, when the user plays a video, the SOC image delivery rate is set to be consistent with the video frame rate. DDIC 140 can control the refresh rate of the display screen 130 to adapt to the SOC image delivery rate. That is, the SOC image delivery rate can change based on changes in user usage scenarios.

[0065] The screen refresh rate is the number of times the display refreshes the display screen in 1 second. For example, a 60Hz refresh rate means that the display screen refreshes the display screen 60 times in 1 second. For a display screen with multiple screen refresh rates and supporting a low refresh rate (such as 10Hz), switching the screen refresh rate from high to low can reduce screen power consumption. How to reduce the screen refresh rate will be described in detail in subsequent embodiments, which will not be expanded here.

[0066] Generally, the frame buffer can be integrated into the DDIC. In order to miniaturize the IC, the frame buffer can also be omitted in the DDIC. In this way, the SOC does not send images to the display screen through the frame buffer, and does not send image refresh instructions or non-image refresh instructions to the DDIC. When the DDIC receives the image sent by the SOC through the data transmission interface such as the MIPI interface, it immediately drives the display screen to display. Generally, the response time of this type of display screen is very short. For example, the time to refresh a frame of the picture is as low as 1 / 120 second.

[0067] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0068] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0069] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0070] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 130. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0071] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0072] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0073] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 130 and application processor.

[0074] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0075] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0076] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0077] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0078] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0079] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0080] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0081] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0082] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0083] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0084] The earphone interface 170D is used to connect a wired earphone and can be a USB interface, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0085] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The electronic device 100 may receive a button input and generate a key signal input related to the user settings and function control of the electronic device 100. The motor 191 may generate a vibration prompt. The SIM card interface 195 is used to connect a SIM card. The SIM card may be connected to or disconnected from the electronic device 100 by inserting the SIM card interface 195 or pulling the SIM card interface 195 out.

[0086] Figure 1 The illustrated structure does not constitute a specific limitation on the electronic device 100. The electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] The embodiment of the present application provides a display method, which reduces the power consumption of the display screen through a frame reduction process with an adaptive frame reduction starting point, while also improving problems such as screen flickering.

[0088] In the embodiment of the present application, the process of reducing the screen refresh rate may be referred to as frame reduction. Frame reduction may be performed based on a frame reduction sequence. The frame reduction sequence may be composed of n screen refresh rates arranged in descending order, n ≥ 2, and n is a positive integer. For example, "60-30-10" represents a frame reduction sequence composed of three screen refresh rates of 60 Hz, 30 Hz, and 10 Hz, where 60 Hz is the starting point of frame reduction.

[0089] In the frame-dropping sequence, the refresh rates of several consecutive screens can be the same, and it is not required that the refresh rates of n screens are different. For example, in the frame-dropping sequence "60-60-30-30-10", the refresh rates of the first and second screens are both 60 Hz, and the refresh rates of the third and fourth screens are both 30 Hz.

[0090] In the embodiment of the present application, the specific implementation of frame reduction is: perform n screen refreshes in succession, wherein the equivalent refresh rate of the i-th screen refresh is equal to the i-th refresh rate f in the frame reduction sequence. i , i is a positive integer, i≤n.

[0091] The equivalent refresh rate of the i-th screen refresh is related to the time interval from the i-th screen refresh to the i+1-th screen refresh. The longer the time interval is, the smaller the equivalent refresh rate of the i-th screen refresh is; the smaller the time interval is, the larger the equivalent refresh rate of the i-th screen refresh is.

[0092] Specifically, the equivalent refresh rate of the i-th screen refresh can be defined as: Among them, f base The reciprocal of the single frame duration (e.g., 1 / 120 second); m represents the number of frames between the i-th screen refresh and the i+1-th screen refresh. In this article, a frame is a time concept, for example, 1 / 120 second is a frame.

[0093] By simply converting the expression of the equivalent refresh rate of the i-th screen refresh, the number of frames between the i-th screen refresh and the i+1-th screen refresh can be expressed as: f i is the i-th refresh rate in the frame-dropping sequence. As i increases, the time interval between two adjacent screen refreshes increases, thereby gradually reducing the screen refresh rate.

[0094] The time T of the i-th screen refresh in frame drop i and the time T of the i+1th screen refresh i+1 Has the following relationship (in frames):

[0095] Figure 3 The following example shows a process in which a frame reduction sequence is completely executed. The frame reduction sequence is "60-30-10". Figure 3 As shown in the figure, the display screen has executed three screen refreshes in accordance with the frame reduction sequence. Among them, the interval from the first screen refresh to the second screen refresh is 1 frame, and the equivalent refresh rate of the first screen refresh is 60Hz; the interval from the second screen refresh to the third screen refresh is 3 frames, and the equivalent refresh rate of the second screen refresh is 30Hz; the interval from the third screen refresh to the next screen refresh is 11 frames, and the equivalent refresh rate of the third screen refresh is 10Hz. Figure 3 The frame numbers "1", "2", etc. in the figure only indicate the order of frames, and do not limit them to the first frame, the second frame, etc. during the operation of the display screen. The same applies to the subsequent figures.

[0096] Secondly, Figure 4 The overall flow chart of the display method provided in the embodiment of the present application is shown.

[0097] like Figure 4 As shown, the method may include the following steps:

[0098] S11, the electronic device detects that the SOC sends image data of the first picture to the display screen.

[0099] Not limited to SOC, the image sending device can also be other processors or processor systems, chips or chip systems that can generate image data for one picture after another through rendering, synthesis and other operations. It is also responsible for sending images to the display screen, which is summarized here as an image sending unit.

[0100] S12: If there is a frame drop in progress, the electronic device stops the existing frame drop.

[0101] S13, the electronic device starts to perform new frame reduction based on the first frame reduction sequence.

[0102] Herein, the new downgraded frame may be referred to as a first downgraded frame, and the aforementioned existing downgraded frame may be referred to as a second downgraded frame.

[0103] The first frame reduction sequence may be composed of n screen refresh rates arranged in descending order, n≥2, and n is a positive integer. The first screen refresh rate in the first frame reduction sequence (i.e., the frame reduction starting point) may be determined according to the interval frame number K from the first refresh frame to the previous refresh frame.

[0104] In the embodiment of the present application, the refresh frame refers to the frame in which the display screen performs screen refresh, such as Figure 3 The 1st, 3rd, 7th, and 18th frames in the display are shown in Figure 1. The opposite concept is the non-refresh frame, which means that the display screen does not perform screen refresh, such as Figure 3Specifically, the first refresh frame refers to the refresh frame of the first screen refreshed by the display screen for the first time. It is also the time of the first screen refresh in the first drop frame, and the time when the drop frame starts.

[0105] The last refresh frame of the first refresh frame is the frame of the most recent screen refresh performed by the display screen before the first refresh frame, that is, the time of the last screen refresh performed by the display screen before the first refresh of the first picture. It is also the time of the last screen refresh before the start of the first drop frame.

[0106] The first frame reduction refers to performing n screen refreshes on the first picture. As the frame reduction process progresses, the time interval between two adjacent screen refreshes becomes longer and longer, thereby gradually reducing the screen refresh rate. Specifically, the time T of the i-th (i≥2) screen refresh in the first frame reduction is i is (in frames): f i is the i-th refresh rate in the first down-frame sequence, f base It is the reciprocal of the single frame duration (such as 1 / 120 seconds), T i-1 is the time of the i-1th screen refresh.

[0107] by Figure 3 For example, frame reduction is performed based on the frame reduction sequence "60-30-10". Figure 3 As shown, the first screen refresh in the frame drop occurs in the first frame, that is, T 1 =1; therefore, according to This formula can determine that the second and third screen refresh times in the frame drop are: frame 3 and frame 7 respectively; f base =120.

[0108] During the execution of the first frame reduction, the electronic device can also detect whether the SOC has sent a picture. If the SOC has sent a picture during the first frame reduction process, the electronic device stops executing the first frame reduction. The picture that the SOC newly sends during the frame reduction process can be called the second picture. That is, the electronic device can start to perform frame reduction on the display screen every time the picture is refreshed, but the frame reduction can be interrupted by the new picture sent by the SOC at any time and is no longer executed, but the picture of the new picture sent by the SOC is refreshed, so that the screen refresh rate responds to the picture sent by the SOC in time to ensure the normal use of the electronic device by the user. While refreshing the picture of the new picture sent by the SOC, the electronic device starts to execute a new frame reduction based on a frame reduction sequence, which can be called the third frame reduction. The frame reduction sequence can be called the second frame reduction sequence, which can be composed of m screen refresh rates arranged in descending order, m≥2, m is a positive integer, and the first screen refresh rate (new frame reduction starting point) included in it can be determined according to the time interval between the last refresh before the start of the third frame reduction and the first screen refresh of the second frame reduction. That is, the first screen refresh rate in the second frame reduction sequence is determined according to the interval frame number from the second refresh frame to the previous refresh frame of the second refresh frame. The second refresh frame refers to the frame in which the display screen refreshes the second picture for the first time. Among them, the third frame reduction includes refreshing the second picture m times using the image data of the second picture, and the m refreshes are executed according to the m screen refresh rates, wherein the time interval between two adjacent refreshes becomes larger and larger as the third frame reduction proceeds, thereby achieving a reduction in the screen refresh rate.

[0109] Figure 5A This shows the situation where the frame drop is not interrupted by the SOC sending the picture. Figure 5A As shown in the figure, after receiving the picture h sent by the SOC, the display screen starts to reduce the frame in the 3rd frame, based on the frame reduction sequence of "60-30-10". A complete frame reduction process based on this sequence is from the 3rd frame to the 20th frame. The first refresh in the frame reduction refreshes the picture h, which is a picture-sending refresh; the second and third refreshes thereafter refresh the old picture of the picture h, which are non-picture-sending refreshes. Since the SOC did not send pictures for a long time after sending picture h, the entire frame reduction process was not interrupted and the frame reduction was fully executed.

[0110] Figure 5B It shows the situation where the frame drop is interrupted by the SOC sending picture. Figure 5B As shown, after receiving the picture h sent by the SOC, the display screen starts to reduce the frame at the 3rd frame, based on the frame reduction sequence of "60-30-10". A complete frame reduction process based on this sequence is from the 3rd frame to the 20th frame. However, since the SOC sends the picture h+1 after sending the picture h, the frame reduction process is interrupted and no longer executed. Only the first and second screen refreshes are executed. The first screen refresh refreshes the picture h, and the second refresh refreshes the old picture of the picture h. While refreshing the picture h+1, a new frame reduction starts.

[0111] In the embodiment of the present application, the starting point of a frame drop is set according to the interval frame number K from the last refresh before the start of the frame drop to the first refresh of the frame drop. It changes according to the size of the interval frame number K and is not a fixed value.

[0112] For example, Fig. 6A As shown in the figure, the frame drop sequence is "60-30-10", and the starting point of frame drop is fixed to a higher screen refresh rate, such as 60Hz. In a scenario where the SOC image delivery rate is low (such as 30Hz), if the user stays on a static interface, the screen refresh rate will be higher than the SOC image delivery rate. At the second refresh of each frame drop, the display will use the old image in the frame buffer for interpolation refresh. The actual refresh rate of the display does not drop but increases overall, resulting in increased display power consumption. Therefore, using a fixed frame drop starting point for frame drop will result in increased display power consumption.

[0113] For example, Figure 6B As shown in the figure, the frame drop sequence is "30-10", and the starting point of the frame drop is fixed to a lower screen refresh rate, such as 30Hz. When performing frame drop, the screen refresh rate may jump, such as from 120Hz to 30Hz, causing the display to flicker. 120Hz is the screen refresh rate before frame drop.

[0114] In other words, using a fixed frame rate reduction starting point will not be conducive to improving problems such as reducing screen power consumption and screen flickering.

[0115] To this end, the embodiment of the present application also introduces a parameter that can reflect the size of the interval frame number K, which is the equivalent refresh rate f(j-1) of the last refresh before the start of frame reduction, and uses it to design a rule for determining the starting point of frame reduction.

[0116] Referring to the definition of the equivalent refresh rate of the i-th screen refresh in the previous article, f(j-1) can be expressed as Among them, f base It is the reciprocal of a single frame duration (such as 1 / 120 second).

[0117] For example, Figure 5A As shown in FIG. 1 , the display screen refreshes the picture h in the 3rd frame and starts to drop the frame at the same time. The last refresh before the frame drop occurs in the 1st frame. There is one frame between the 1st frame and the 3rd frame, that is, K=1. Therefore, the equivalent refresh rate of the last refresh is 60Hz. For another example, Figure 5B As shown, the display screen refreshes the picture h in the second frame and starts to drop the frame at the same time. The last refresh before the frame drop occurs in the first frame. There are 0 frames between the first frame and the second frame, that is, K = 0. Therefore, the equivalent refresh rate of the last refresh is 120Hz.

[0118] The key to calculating f(j-1) is to obtain the value of parameter K, which can be recorded by an electronic device, specifically by a SOC or a DDIC.

[0119] The SOC can record the time when the refresh instruction (including non-image sending refresh instruction and image sending refresh instruction) is sent to the DDIC. The time when the SOC sends the refresh instruction is equivalent to the time when the display screen executes the refresh. That is, in which frame does the SOC send the refresh instruction for the first time to refresh the first screen? This frame is the first refresh frame, and its previous refresh frame is the time when the SOC sent the previous refresh instruction before the refresh instruction. The frame interval number between the two can be determined as the value of K. In this way, the value of K is determined according to the interval frame number from the refresh instruction sent by the SOC to the previous refresh instruction. In other words, the SOC can record the time when the refresh instruction is sent to the DDIC, and determine that the interval frame number K is equal to the interval frame number from the first frame to the second frame. The first frame is the time when the SOC sends the refresh instruction for the first time to refresh the first screen, and the second frame is the time when the SOC sends the refresh instruction for the last time before the first frame.

[0120] Since DDIC is responsible for controlling the refresh of the display, DDIC can directly record the refresh time of the first refresh of the first screen and the time of the last refresh, and then directly calculate the value of K, or let SOC calculate the value of K. In other words, DDIC can record the time when the display refreshes the screen, and determine that the interval frame number K is equal to the interval frame number from the third frame to the fourth frame. The third frame is the refresh time when the display refreshes the first screen for the first time, and the fourth frame is the time when the display refreshed the last time before the third frame.

[0121] Compared with designing the frame drop starting point determination rule directly based on the interval frame number K, using f(j-1) to design the rule can be applied to more scenarios, such as f base The scene changes.

[0122] The following will explain in detail the rules for setting the frame drop starting point.

[0123] First, the possible value range of f(j-1) (eg, 0 to 120 Hz) is divided into multiple segments, for example, 6 segments.

[0124] Secondly, divide these multiple intervals into two types, among which the equivalent refresh rate in the second segmented interval is lower than the equivalent refresh rate in the first segmented interval; and determine: in the first segmented interval, the starting point of frame reduction is set to be less than or equal to the minimum value of the interval; in the second segmented interval, the starting point of frame reduction is set to be greater than or equal to the minimum value of the interval.

[0125] Specifically, Table 1 exemplarily shows the setting rules of the frame drop starting point.

[0126]

[0127] Table 1

[0128] As shown in Table 1, the possible value range of f(j-1) [0~n] is divided into 6 segments: 0~c, c, c~f, f, f~n, n. These segments are arranged in ascending order. Among them, c, f, and n are not segments in the strict sense, but a value. When the value of f(j-1) is 0, this situation generally occurs when the display screen is powered on, and no refresh has been performed before. The frame drop starting point is u, and u is greater than or equal to c; when the value of f(j-1) falls between 0 and c, the frame drop starting point is u, and u is greater than or equal to c; when the value of f(j-1) is equal to c, the frame drop starting point is t, and t is less than or equal to c; when the value of f(j-1) falls between c and f, the frame drop starting point is s, and s is less than or equal to c; when the value of f(j-1) is equal to f, the frame drop starting point is r, and r is less than or equal to f; when the value of f(j-1) falls between f and n, the frame drop starting point is q, and q is less than or equal to f; when the value of f(j-1) is equal to n, the frame drop starting point is p, and p is less than or equal to n. In this way, the frame drop starting point can be determined based on the frame drop starting point lookup table shown in Table 1.

[0129] The interval 0 to c is the second type of segmented interval mentioned above, and the interval c, c to f, f, f to n, n is the first type of segmented interval mentioned above.

[0130] That is to say, if f(j-1) falls within the first segmentation interval, the first screen refresh rate in the first frame reduction sequence is determined to be a first value, such as u, and the first value is set to be less than or equal to the minimum value of the first segmentation interval; if f(j-1) falls within the second segmentation interval, the first screen refresh rate in the first frame reduction sequence is determined to be a second value, such as t, s, r or p, q, and the second value is set to be greater than or equal to the maximum value of the second segmentation interval. The value range of f(j-1) is divided into one or more first segmentation intervals and one or more second segmentation intervals, and the refresh rate of the first segmentation interval is less than the refresh rate of the second segmentation interval.

[0131] The frame reduction starting point corresponding to the first segmented interval can ensure that the refresh rate of the display screen will not be too high when the SOC continuously sends images in this interval, so that the SOC image sending rate is basically consistent with the display screen refresh rate; and the refresh rate will not jump during the frame reduction process, which can improve the screen flicker problem. This will be explained with examples later.

[0132] The frame drop starting point corresponding to the second segment interval can prevent the display from being at a low refresh rate for a long time, improve the screen retention problem, and enable the display to quickly adapt to the high SOC image delivery rate that may occur at any time, prepare for restoring the high refresh rate, reduce the jump from low refresh rate to high refresh rate, and improve the screen flicker problem. This will be explained with examples later.

[0133] In addition, the frame reduction starting points corresponding to the two segments f~n and n in Table 1 can be uniformly set to f, satisfying the frame reduction starting point setting rules for these two segments; the frame reduction starting points corresponding to the three segments c, c~f, and f in Table 1 can be uniformly set to c, satisfying the frame reduction starting point setting rules for these three segments. The frame reduction starting point corresponding to the segment 0~c in Table 1 can be set to be greater than or equal to c, such as f. In this way, the six segments in Table 1 can be simplified as Figure 7 The A, B, and C sections in the table are more convenient for table lookup.

[0134] For example, Table 1 can be further implemented as Table 2. The values ​​of c, f, and n in Table 1 can be selected as 30Hz, 60Hz, and 120Hz respectively, the frame drop starting points p and q can be uniformly selected as 60Hz, the frame drop starting points r, s, and t can be uniformly selected as 30Hz, and the frame drop starting point u can be selected as 60Hz. Figure 7 Can be embodied Figure 8 .

[0135]

[0136] Table 2

[0137] Using 30Hz as the dividing point between the first segmentation interval and the second segmentation interval can further help improve the problem of screen retention. 30Hz is generally the refresh rate threshold of the display screen in terms of screen retention. That is, the display screen will have or is prone to screen retention problems only when the refresh rate is lower than 30Hz. In actual applications, depending on the hardware capabilities of the display screen, the value of 30Hz can also be replaced with other refresh rates. For example, a certain display screen will have screen retention problems only when the refresh rate is lower than 10Hz. The refresh rate threshold can be determined as follows: the display screen continues to work at a certain refresh rate for a first period of time (such as 5 seconds) or more. If screen retention occurs, the refresh rate is determined to be the refresh rate threshold of the display screen.

[0138] In conjunction with the frame drop start point lookup table shown in Table 2, Figure 9-12 The example shown illustrates the frame reduction solution provided by the embodiment of the present application.

[0139] Fig. 9 The frame reduction process is shown when the value of f(j-1) falls within the B segment. Fig. 9As shown, the SOC continuously sends images at a rate of 30 Hz. The display screen refreshes the picture h sent by the SOC in the 5th frame. The last refresh before the 5th frame occurred in the 1st frame, and the equivalent refresh rate of the last refresh was 30 Hz. Therefore, the lookup table 2 determines that the starting point of the i-th frame reduction is 30 Hz, and starts the i-th frame reduction in the 5th frame according to the "30-10" frame reduction sequence. However, the i-th frame reduction is interrupted by the picture h+1 sent by the SOC. The display screen refreshes the picture h+1 newly sent by the SOC in the 9th frame, and starts the i+1th frame reduction. The last refresh before the 9th frame occurred in the 5th frame, and the equivalent refresh rate of the last refresh was 30 Hz. Therefore, the lookup table 2 determines that the starting point of the i+1th frame reduction is 30 Hz, and starts the i+1th frame reduction in the "30-10" frame reduction sequence. The i+2th frame reduction is interrupted by the picture h+2 sent by the SOC. The display screen refreshes the picture h+2 newly sent by the SOC in the 9th frame and starts the i+2th frame reduction.

[0140] from Fig. 9 It can be seen that since the starting point of each frame drop is adaptively based on the equivalent refresh rate of the previous refresh, the refresh rate is stabilized at 30Hz, which is consistent with the SOC image delivery rate and will not increase the refresh rate. This minimizes the power consumption of the display while ensuring the normal use of the device by the user. Fig. 6A The disadvantages of the scheme of using a fixed frame drop starting point for frame drop can be more clearly perceived.

[0141] Fig.10 The frame reduction process is shown when the value of f(j-1) initially falls into segment A. Fig.10 As shown, the SOC continuously sends images at a rate of 30Hz. The display screen refreshes the picture h-1 sent by the SOC in the 2nd frame. The last refresh before the 2nd frame occurred in the 1st frame, and the equivalent refresh rate of the last refresh was 120Hz, so the lookup table 2 determines that the starting point of the i-1th frame reduction is 60Hz, and starts the i-th frame reduction according to the frame reduction sequence of "60-30-10". However, the i-1th frame reduction is interrupted by the picture h sent by the SOC. The display screen refreshes the picture h sent by the SOC in the 6th frame, and starts the i-th frame reduction. The last refresh before the 6th frame occurred in the 4th frame, and the equivalent refresh rate of the last refresh was 60Hz, so the lookup table 2 determines that the starting point of the i-th frame reduction is 30Hz, and starts the i-th frame reduction according to the frame reduction sequence of "30-10". The i-th frame reduction is interrupted by the picture h+1 sent by the SOC, and the display screen refreshes the picture h+1 sent by the SOC in the 10th frame, and starts the i+1-th frame reduction. And so on, the subsequent frame reduction process continues.

[0142] from Fig.10It can be seen that when the equivalent refresh rate is very fast at the beginning, the equivalent refresh rate is first reduced to 60Hz and then to 30Hz, and finally stabilized at 30Hz, which is consistent with the SOC image transmission rate. This frame reduction process does not jump from high refresh rate to low refresh rate, avoiding the problem of screen flickering; and because the display refresh rate is consistent with the SOC image transmission rate during the period of continuous SOC image transmission, the refresh rate is not increased, and the display power consumption is minimized on the basis of ensuring the normal use of the user. Regarding these beneficial effects, compared with the previous Fig. 6A and Figure 6B The disadvantages of the scheme of using a fixed frame drop starting point for frame drop can be more clearly perceived.

[0143] Fig.11 The frame reduction process when the value of f(j-1) falls within the C segment is shown. Fig.11 As shown, the SOC continuously sends images at a rate of 30Hz. The display screen refreshes the picture h sent by the SOC in the 13th frame. The last refresh before the 13th frame occurred in the 1st frame, and the equivalent refresh rate of the last refresh was 10Hz, so the lookup table 2 determines that the starting point of the i-th frame reduction is 60Hz, and starts the i-th frame reduction according to the frame reduction sequence of "60-30-10". However, the i-th frame reduction is interrupted by the picture h+1 sent by the SOC. The display screen refreshes the picture h+1 newly sent by the SOC in the 17th frame, and starts the i+1th frame reduction. The last refresh before the 17th frame occurred in the 13th frame, and the equivalent refresh rate of the last refresh was 60Hz, so the lookup table 2 determines that the starting point of the i+1th frame reduction is 30Hz, and starts the i+1th frame reduction according to the frame reduction sequence of "30-10". The i+1th frame reduction is interrupted by the picture h+2 sent by the SOC, and the display screen refreshes the picture h+2 sent by the SOC in the 21st frame, and starts the i+2th frame reduction. And so on, the subsequent frame reduction process continues.

[0144] from Fig.11 It can be seen that when the equivalent refresh rate is very low at the beginning, the refresh rate is first raised to 60Hz and then reduced to 30Hz, and finally stabilized at 30Hz, which is consistent with the SOC image transmission rate. This can prevent the display from being at a low refresh rate for a long time, which can improve the screen retention problem. In addition, there is no jump from low refresh rate to high refresh rate during this frame reduction process, which avoids the problem of screen flickering. In addition, since the refresh rate of the display is consistent with the SOC image transmission rate during the continuous image transmission period of the SOC, the refresh rate is not raised, which minimizes the power consumption of the display while ensuring the normal use of the device by users.

[0145] In addition, if Fig.12As shown, when the equivalent refresh rate is very low at the beginning (as low as 10Hz), the refresh rate is first pulled up to 60Hz and then responds to the SOC image sending rate increased to 120Hz. This allows the display to quickly adapt to the high SOC image sending rate that may occur at any time, such as picture h+1 to picture h+3, to prepare for the restoration of high refresh rate, avoid the jump from low refresh rate to high refresh rate, and improve the screen flickering problem.

[0146] Fig.13 FIG. 1 shows multiple functional units in the electronic device 100 for implementing the display method provided in the embodiment of the present application. Fig.13 As shown, the multiple functional units may include a picture sending unit, a statistics unit, a frame reduction control unit and a storage unit, wherein:

[0147] The image sending unit can be used to generate one image after another through rendering, synthesis and other operations, and send the image data of these images to the DDIC. Finally, the image data is processed by the DDIC and sent to the display screen for display. The mobile industry processor interface (MIPI) signal of the display interface can be used to test the SOC image sending situation, that is, when the SOC sends the image.

[0148] The image sending unit can be integrated into a SOC, such as a GPU or a CPU.

[0149] The statistical unit can be used to count the time it takes for the display screen to perform each refresh, that is, to count the time for each refresh frame; and use the statistical refresh frame time to calculate the equivalent refresh rate of the last refresh before frame reduction, and pass the equivalent refresh rate of the last refresh before frame reduction to the frame reduction control unit. The key to calculating the equivalent refresh rate f(j-1) of the last refresh before frame reduction is to obtain the value of parameter K, which is the number of frames between the first refresh frame of the first refresh of the first screen and its previous refresh frame. Regarding how to use K to calculate the equivalent refresh rate f(j-1) of the last refresh before frame reduction, please refer to the relevant content in the previous article, which will not be repeated here.

[0150] The statistical unit can be integrated in the SOC or in the DDIC.

[0151] In the embodiment where the statistics unit is integrated into the SOC, the statistics unit can record the time when the SOC sends a refresh instruction (including a non-image sending refresh instruction and an image sending refresh instruction) to the DDIC. The time when the SOC sends the refresh instruction is equivalent to the time when the display screen executes the refresh. That is, in which frame does the SOC send the refresh instruction for the first time to refresh the first screen? This frame is the first refresh frame, and its previous refresh frame is the time when the SOC sent the previous refresh instruction before the refresh instruction. The number of frame intervals between the two can be determined as the value of K. In this way, the statistics unit determines the value of K based on the number of interval frames from when the SOC sends the refresh instruction for the first time to when the SOC sends the previous refresh instruction.

[0152] In the embodiment where the statistical unit is integrated in the DDIC, since the DDIC is responsible for controlling the display screen to refresh, the statistical unit can directly record the refresh time of the first refresh of the first screen and the time of the last refresh, and then directly calculate the value of K, or let the SOC calculate the value of K.

[0153] The storage unit can be used to store the frame drop sequence and the frame drop start point lookup table. The frame drop start point lookup table can refer to Table 1, Table 2, Figure 7-Figure 8 The frame reduction sequence can be stored in association with its frame reduction starting point, so that after determining the frame reduction starting point, the frame reduction control unit further finds the frame reduction sequence according to the frame reduction starting point.

[0154] The storage unit can be integrated in the SOC, in the DDIC, or in an independent memory. Here, independent means independent of the SOC and DDIC and outside of them.

[0155] The frame reduction control execution unit can be used to find the frame reduction starting point and frame reduction sequence from the frame reduction starting point lookup table according to the equivalent refresh rate of the last refresh before the frame reduction transmitted by the statistical unit, and then trigger the DDIC to drive the display to execute frame reduction.

[0156] The frame reduction control execution unit may be integrated in the SOC or in the DDIC.

[0157] Fig.14 Shows Fig.13 An integrated architecture of the functional units shown. Fig.14 As shown, the image sending unit, the statistical unit, the frame reduction control unit and the storage unit are all integrated in the SOC. While the SOC is sending the image, the statistical unit in the SOC can calculate the equivalent refresh rate of the last refresh before the frame reduction, and pass the equivalent refresh rate of the last refresh before the frame reduction to the frame reduction control execution unit, which finds the frame reduction starting point from the frame reduction starting point lookup table and obtains the frame reduction sequence, and triggers the display system composed of the DDIC and the display screen to execute the frame reduction process.

[0158] exist Fig.14 In the illustrated architecture, the display system composed of the SOC, the DDIC and the display screen can implement the display method provided in the embodiment of the present application, that is, the solution provided in the embodiment of the present application can be applied to an electronic device including the above-mentioned SOC and the above-mentioned display system.

[0159] Fig.15 Shows Fig.13 Another integrated architecture of the functional units shown. Fig.15As shown, only the image sending unit is integrated in the SOC, and the statistical unit, frame reduction control unit and storage unit are integrated in the DDIC. While the SOC is sending the image, the statistical unit in the DDIC can calculate the equivalent refresh rate of the last refresh before the frame reduction, and pass the equivalent refresh rate of the last refresh before the frame reduction to the frame reduction control execution unit in the DDIC. The frame reduction control execution unit finds the frame reduction starting point from the frame reduction starting point lookup table and obtains the frame reduction sequence, and controls the display screen to perform the frame reduction process.

[0160] exist Fig.15 In the illustrated architecture, the display system composed of the DDIC and the display screen can serve as the executor of the display method provided in the embodiment of the present application, that is, the solution provided in the embodiment of the present application can be applied to the display system.

[0161] As mentioned above, in order to miniaturize the IC, a frame buffer may not be provided in the DDIC, so that the SOC does not send images to the display screen through the frame buffer, nor does it send image refresh instructions or non-image refresh instructions to the DDIC. When the DDIC receives the image sent by the SOC through a data transmission interface such as the MIPI interface, it immediately drives the display screen to display. Generally, the response time of such a display screen is very short. In this embodiment, the SOC needs to understand the refresh rate currently followed by the display screen, and send the image data of the new picture to the display screen when a new picture arrives. When there is no new picture arriving, it determines whether to perform frame refresh according to the refresh rate currently followed by the display screen. If necessary, the image data of the previous picture is sent to the display screen.

[0162] For example, Fig.16 As shown, in the second frame, the SOC sends the image data of the new picture, i.e., picture h, to the display screen, and controls the display screen to start frame reduction. In frame reduction, the refresh frame of the display screen that refreshes picture h for the first time is the second frame, and its previous refresh frame is the first frame. Therefore, the SOC can determine that the starting point of frame reduction is 60Hz based on the interval frame number 0 between the two, and adopt the frame reduction sequence "60-30-10". Moreover, after the second frame, no new picture arrives for a long time. In this way, in the third frame, the SOC does not send the picture; in the fourth frame, the eighth frame, and the nineteenth frame, the SOC still sends the image data of picture h to the display screen (also known as sending the old picture); in the fifth frame to the seventh frame, and the ninth frame to the eighteenth frame, the SOC does not send the picture.

[0163] In the implementation method without setting the frame buffer, the SOC can record the time of each image transmission. The time of each image transmission by the SOC is equivalent to the time of refreshing the screen. Then, the SOC can calculate the value of K based on the time of the first image transmission and the last image transmission time, and then calculate the equivalent refresh rate of the last refresh before the frame reduction, and finally determine the starting point of the frame reduction.

[0164] In the implementation method without setting the frame buffer, the DDIC can record the time of receiving each image data from the SOC. The time of receiving the image data of each picture is equivalent to the time of refreshing the picture on the display screen. Then, the DDIC can calculate the value of K according to the time of first receiving the image data of the first picture and the time of last receiving the image data from the SOC, and then calculate the equivalent refresh rate of the last refresh before the frame drop, and finally determine the starting point of the frame drop.

[0165] The frame reduction scheme provided in the embodiments of the present application can also be applied to various equipment and technical fields that support high-frequency and low-frequency operation. For example, the process of reducing the sampling frequency of near-light ambient light from a high sampling rate to a low sampling rate can also be applied to the frame reduction scheme provided in the embodiments of the application, and the image sending unit in the above text can be replaced with a light test data sending unit in this example. For another example, the process of reducing the sampling and reporting points of a touch device from a high sampling rate to a low sampling rate can also be applied to the frame reduction scheme provided in the embodiments of the application, and the image sending unit in the above text can be replaced with a touch data sending unit in this example.

[0166] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0167] The embodiments of the present application also provide a computer program product. When the computer program product is executed on an electronic device, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0168] The embodiment of the present application also provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0169] The term "user interface (UI), or interface for short) in the specification and drawings of this application refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. A control, also called a widget, is a basic element of a user interface. Typical controls include a toolbar, menu bar, text box, button, scrollbar, picture and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as . A node corresponds to a control or attribute in the interface. After parsing and rendering, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, HTML is defined by 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0170] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0171] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0172] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0173] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A display method, the method is applied to an electronic device, the electronic device comprises an image sending unit and a display screen, characterized in that: include: The electronic device detects that the image sending unit sends image data of the first picture to the display screen; The electronic device starts to perform a first frame reduction based on the first frame reduction sequence; The first frame-decreasing sequence is composed of n screen refresh rates arranged in descending order, n≥2, and n is a positive integer; the first screen refresh rate in the first frame-decreasing sequence is determined according to the interval frame number K from the first refresh frame to the previous refresh frame of the first refresh frame, and K is a positive integer; the first refresh frame refers to the frame in which the display screen refreshes the first picture for the first time; The first frame reduction includes refreshing the first picture n times using the image data of the first picture, wherein the n refreshes are performed according to the n screen refresh rates, wherein the time interval between two adjacent refreshes becomes larger as the first frame reduction proceeds.

2. The method according to claim 1, characterized in that The n refreshes are executed according to the n screen refresh rates, specifically including: the time T of the i-th refresh in the first down-frame i for: f i is the i-th refresh rate in the first frame-down sequence, f base is the reciprocal of the single frame duration, T i-1 is the time of the i-1th refresh; T i , T i-1 The time unit is frame; n≥i≥2, i is a positive integer.

3. The method according to claim 1 or 2, characterized in that Before the electronic device starts to perform the first frame reduction based on the first frame reduction sequence, the electronic device also includes: determining whether there is a second frame reduction, the second frame reduction is an ongoing frame reduction, and if so, stopping the second frame reduction.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: In the process of executing the first frame reduction, the electronic device detects whether the image sending unit has a second picture to send, and if so, stops executing the first frame reduction and starts executing the third frame reduction; the third frame reduction is executed based on a second frame reduction sequence, the second frame reduction sequence is composed of m screen refresh rates arranged in descending order, m≥2, and m is a positive integer; the first screen refresh rate in the second frame reduction sequence is determined according to the interval frame number from the second refresh frame to the previous refresh frame of the second refresh frame; the second refresh frame refers to the frame in which the display screen refreshes the second picture for the first time; The third frame reduction includes refreshing the second picture m times using the image data of the second picture, and the m refreshes are performed according to the m screen refresh rates, wherein the time interval between two adjacent refreshes becomes larger as the third frame reduction proceeds.

5. The method according to any one of claims 1 to 4, characterized in that The first screen refresh rate in the first frame reduction sequence is determined according to the interval frame number K from the first refresh frame to the previous refresh frame of the first refresh frame, specifically including: The first screen refresh rate in the first frame drop sequence is determined according to an equivalent refresh rate f(j-1) of the display screen when it is refreshed in a previous refresh frame of the first refresh frame, and f(j-1) is calculated and determined by the following formula: Among them, f base It is the reciprocal of the duration of a single frame.

6. The method according to claim 5, characterized in that The first screen refresh rate in the first frame reduction sequence is determined according to an equivalent refresh rate f(j-1) when the display screen is refreshed in a previous refresh frame of the first refresh frame, specifically including: If f(j-1) falls within the first segmentation interval, determining the first screen refresh rate in the first frame reduction sequence to be a first value, wherein the first value is set to be less than or equal to a minimum value of the first segmentation interval; If f(j-1) falls within the second segmentation interval, determining the first screen refresh rate in the first frame reduction sequence to be a second value, wherein the second value is set to be greater than or equal to the maximum value of the second segmentation interval; The value range of f(j-1) is divided into one or more of the first segmented intervals and one or more of the second segmented intervals, and the refresh rate of the first segmented intervals is lower than the refresh rate of the second segmented intervals.

7. The method according to claim 6, characterized in that The value range of f(j-1) is [0,f max ], where f max is the maximum refresh rate of the display.

8. The method according to claim 6 or 7, characterized in that The second segmentation interval is [0,f thre ), where f thre is a refresh rate threshold of the display screen; when the display screen continuously operates at the refresh rate threshold for a first period of time or longer, image retention occurs.

9. The method according to any one of claims 6 to 8, characterized in that fthre is equal to 30 Hz; the value range of f(j-1) is divided into the following segmented intervals: [0,30), 30, (30,60), 60, (60,120), 120, among which [0,30) is the second segmented interval, 30, (30,60), 60, (60,120), 120 is the first segmented interval.

10. The method according to claim 9, characterized in that The first value is set to 60; if f(j-1) falls in one of the second-type segmentation intervals of 30, (30, 60), and 60, the second value is set to 30; if f(j-1) falls in one of the second-type segmentation intervals of (60, 120), and 120, the second value is set to 60.

11. The method according to any one of claims 1 to 10, characterized in that The image sending unit is a system on chip (SOC); the interval frame number K is determined by the following steps: The SOC records the time of sending a refresh instruction to the display driver integrated circuit DDIC connected to the display screen, and determines that the interval frame number K is equal to the interval frame number from the first frame to the second frame, the first frame is the time when the SOC sends the refresh instruction for the first refresh of the first screen, and the second frame is the time when the SOC last sent the refresh instruction before the first frame.

12. The method according to any one of claims 1 to 11, characterized in that The interval frame number K is determined by the following steps: The display driver integrated circuit DDIC connected to the display screen records the time when the display screen performs screen refresh, and determines that the interval frame number K is equal to the interval frame number from the third frame to the fourth frame, the third frame is the refresh time when the display screen refreshes the first picture for the first time, and the fourth frame is the time when the display screen performs the last refresh before the third frame.

13. An electronic device, characterized in that: It includes one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the first electronic device executes the method as described in any one of claims 1-12.

14. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes a processing circuit and an interface circuit, the interface circuit is used to receive instructions and transmit them to the processing circuit, and the processing circuit is used to run the instructions to execute the method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 11.

Images