Image sending and displaying method and electronic equipment

Abstract

The invention discloses an image transmitting and displaying method and electronic equipment, and relates to the technical field of terminals. The method comprises the steps of obtaining a to-be-displayed first image; and under the condition that a TE signal does not wait for a waiting duration exceeding a preset duration, the first image is sent and displayed at a first moment, and the first moment is located before a next TE signal arrives. Wherein the frequency of the TE signal is consistent with the application frame rate of the first application. Therefore, the image can be sent and displayed in time, response delay caused by too long time for waiting for TE is avoided, and the response timeliness is improved. Moreover, the frequency of the TE signal is consistent with the application frame rate, so that the image can be refreshed at the time interval consistent with the application frame rate, no obvious frame rate fluctuation occurs, and the visual experience of a user can be improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image display method and electronic device. Background Technology

[0002] Electronic devices can obtain a frame of image by drawing / rendering and compositing, and then send the image to the display screen for display via a display driver. For example, the display driver can send the image to the display screen (also called the screen, screen end, etc.) when it detects a tear effect (TE) signal, such as when it detects the rising edge of the TE signal pulse.

[0003] In gaming scenarios, electronic devices can improve game responsiveness through the following solutions: Upper layers, such as game applications and surface fllingers (SF), can perform drawing / rendering and compositing approximately every 16.6ms to obtain one frame of image, achieving a game frame rate of 60FPS. Simultaneously, the TE signal frequency is 120Hz, and the display driver can send the image to the screen at a 120Hz refresh rate, for example, triggering the display driver to send an image approximately every 8.3ms. In this way, the display driver can detect the TE signal twice during the time it takes to process one frame of image, reducing the waiting time for the TE signal and sending the image to the screen more promptly. This allows for more timely responses to user actions, improving game responsiveness.

[0004] However, practice has shown that using the above-mentioned solutions to improve responsiveness results in significant fluctuations in the game's frame rate and an extremely poor visual experience. Summary of the Invention

[0005] This application provides an image display method and electronic device, which can improve the timeliness of response and reduce the fluctuation of application frame rate, thereby enhancing the visual experience, by controlling the timing of image display and adjusting the refresh rate.

[0006] Firstly, this application provides an image display method. This method can be executed by an electronic device, or by a component of the electronic device, such as a processor, chip, or chip system. It can also be implemented by a logic module or software (such as a display driver) capable of implementing all or part of the electronic device. This application does not specifically limit this method. The following description mainly uses an electronic device as an example, and the foreground application of the electronic device is a first application, such as a game application.

[0007] Specifically, the method includes: obtaining the first image to be displayed, i.e., sending the first image for display (hereinafter also referred to as image sending). If the tearing effect TE signal has not arrived after a preset waiting time, the first image is sent for display at the first moment, which is before the arrival of the next TE signal. For example, the first image is the (k+1)th frame image in the following text, and the first moment can be the moment when the waiting time exceeds the preset time, or the first moment can be the target moment among the preset image sending moments in the following text. Wherein, the frequency of the TE signal is consistent with the application frame rate of the first application, then the display screen can refresh the image at time intervals consistent with the frame rate.

[0008] In summary, by employing this application, when the waiting time for the TE signal is relatively long, image delivery (a process also known as frame tracking) can be performed in advance, ensuring the image is promptly sent to the display screen for display. This avoids response delays caused by excessively long TE waiting times, thereby improving response timeliness. Furthermore, since the frequency of the TE signal is consistent with the application frame rate, there will be no significant frame rate fluctuations, enhancing the user's visual experience.

[0009] In one possible design of the first aspect, before obtaining the first image to be displayed, or after displaying the first image at the first moment, the method further includes: obtaining the second image to be displayed, i.e., needing to display the second image. If the waiting time while waiting for the TE signal does not exceed a preset time, the second image is displayed in response to the arrival of the TE signal. For example, the second image may be the k-th frame image as described below.

[0010] In other words, if the waiting time for the TE signal is short, the display can be executed when the TE signal arrives, thus ensuring that the display is executed at least when the TE signal arrives.

[0011] In one possible design approach of the first aspect, the first moment includes the first timing period of the first timer (hereinafter referred to as Timer 2), which is the adjacent timing period after the waiting time exceeds a preset duration. For example, the first moment is the moment when the target signal is detected after the waiting time exceeds the preset duration, and the target signal is generated at various timing periods of the first timer.

[0012] In one possible design approach of the first aspect, if the tearing effect TE signal is not received after the waiting time exceeds a preset time, the first image is displayed at the first moment, including: if the TE signal is not received after the waiting time exceeds the preset time, and displaying the first image at the first moment will not cause image loss, the first image is displayed at the first moment.

[0013] In other words, images are only sent in advance if it will not result in them being lost, thus ensuring that sending images in advance will not lead to them being lost.

[0014] In one possible design approach of the first aspect, the method further includes: if the TE signal is not received after a waiting period exceeding a preset time, and sending the image at the first moment would result in image loss, as shown in the case of the k+n frame image below, the first image is sent for display in response to the arrival of the TE signal.

[0015] In other words, if sending images in advance would lead to image loss, then images will not be sent in advance, thus avoiding image loss due to sending images in advance.

[0016] In one possible design of the first aspect, after obtaining the second image to be displayed and before sending the first image for display, the method further includes: detecting whether the first moment is within the high-level range of the TE signal. If the first moment is within the high-level range of the TE signal, sending the image for display at the first moment will result in image loss. If the first moment is not within the high-level range of the TE signal, sending the image for display at the first moment will not result in image loss.

[0017] In this way, the frame loss situation can be accurately determined by comparing the time interval of the high and low levels of the TE signal at the first moment.

[0018] In one possible design of the first aspect, after displaying the first image at the first moment, the method further includes: obtaining a third image to be displayed, the third image being an adjacent frame image following the first image. If the tearing effect TE signal is not received after a waiting period exceeding a preset time, the third image is displayed in response to the arrival of the TE signal. For example, the third image is the (k+2)th frame image described below.

[0019] In other words, after the previous frame (e.g., the first image) is sent in advance, the next frame (e.g., the third image) will not be sent in advance even if the waiting time exceeds the preset time. Instead, it will wait until the TE signal arrives before being sent. This avoids large frame rate fluctuations when returning to the normal frame rate after continuous advance image sending, thus reducing frame rate fluctuations and improving the visual experience.

[0020] In one possible design approach of the first aspect, the method further includes: after detecting that sending the image to the display at the first moment would cause frame dropping, switching the frequency of the TE signal from a first frequency (hereinafter referred to as frequency 1) to a second frequency (hereinafter referred to as frequency 2), where the second frequency is higher than the first frequency. That is, if sending the image prematurely at the first frequency of the TE signal would cause frame dropping, then increasing the frequency of the TE signal can reduce the waiting time for the display driver to receive the TE signal, allowing for more timely image sending. Furthermore, after running at the second frequency for at least one frame, the frequency of the TE signal is restored to the first frequency. After restoring to the first frequency, the problem of frame dropping due to premature image sending usually does not continue, thus allowing for premature image sending even with longer waiting times.

[0021] In one possible design of the first aspect, during the operating range at the second frequency, the method further includes: obtaining a fourth image to be displayed. The fourth image is displayed in response to the arrival of the TE signal. For example, the fourth image is the (k+n+1)th frame image described below.

[0022] In other words, after increasing the frequency of the TE signal to the second frequency, image delivery is maintained only when the TE signal arrives. This way, the image display duration only changes from the period corresponding to the first frequency to the period corresponding to the second frequency, without a larger change in frame rate, further enhancing the visual experience.

[0023] In one possible design of the first aspect, after restoring the frequency of the TE signal to the first frequency, the method further includes: obtaining a fifth image to be displayed, the fifth image being an image within a first number (e.g., 1) obtained after restoring to the first frequency. That is, the fifth image is a series of frames after restoring to the first frequency, including the first frame image after restoring to the first frequency, such as the (k+n+2)th frame image mentioned below. In response to the arrival of the TE signal, a fourth image is displayed.

[0024] In this way, the display duration of the image only changes from the period corresponding to the second frequency to the period corresponding to the first frequency, without a larger change in frame rate, which can further improve the visual experience.

[0025] Secondly, this application also provides an electronic device including a display screen, a memory, and one or more processors. The display screen, memory, and processors are coupled. The memory stores computer program code, including computer instructions, which, when executed by the processor, cause the electronic device to perform the methods described in the first aspect and any of its possible designs.

[0026] Thirdly, this application provides a chip system applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the interface circuits are used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect and any of its possible design embodiments.

[0027] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform a method as described in the first aspect and any possible design thereof.

[0028] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method as described in the first aspect and any of its possible design methods.

[0029] Understandably, the beneficial effects that the electronic device of the second aspect, the chip system of the third aspect, the computer storage medium of the fourth aspect, and the computer program product of the fifth aspect can achieve can be referred to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0030] Figure 1 One of the schematic diagrams illustrating the principle of image display provided in the embodiments of this application;

[0031] Figure 2 A second schematic diagram illustrating the principle of image display provided in this application embodiment;

[0032] Figure 3 The third schematic diagram illustrating the principle of image display provided in the embodiments of this application;

[0033] Figure 4 A scene diagram for a game scene image display scheme;

[0034] Figure 5 A hardware structure diagram of the electronic device provided in the embodiments of this application;

[0035] Figure 6 A schematic diagram of the TE signals for the 7T and 8T screens provided in the embodiments of this application;

[0036] Figure 7 Hardware and software architecture diagrams of electronic devices provided in embodiments of this application;

[0037] Figure 8 A timing interaction diagram of the image display method provided in the embodiments of this application;

[0038] Figure 9 One of the scenario diagrams for the image display method provided in the embodiments of this application;

[0039] Figure 10 A schematic diagram illustrating the timing of image submission as provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of a frame dropping scenario;

[0041] Figure 12 A second schematic diagram illustrating a scenario for the image display method provided in this application embodiment;

[0042] Figure 13 A flowchart illustrating the display process of a single frame of image provided in an embodiment of this application;

[0043] Figure 14 This is a schematic diagram of the chip system provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0045] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0046] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0047] Before introducing the embodiments of this application, the following will first combine... Figures 1-3A brief introduction to the principle of image display.

[0048] See Figure 1 Electronic devices include system-on-chips (SoCs) and displays.

[0049] The SoC integrates key chips such as an application processor (AP) and a baseband processor (BP, also known as a modem). Furthermore, the AP may include processing units such as a CPU and a GPU. The display screen further includes a display control module, an image storage unit, and a display panel.

[0050] For example, the display control module may be a display driver integrated circuit (DDIC). It should be noted that the display control module may also be referred to as a controller, driver circuit, etc., and this application embodiment does not specifically limit it in this way.

[0051] For example, the image storage unit can be random access memory (RAM), such as graphics RAM (GRAM).

[0052] The following explanation will use DDIC as an example, where the display control module is a display controller and GRAM is the image storage unit.

[0053] During image processing, electronic devices can execute image processing steps such as rendering and compositing through the CPU and GPU in the SoC. The SoC (such as the display driver in the SoC's software architecture, which will be used as an example below) writes the processed image data into the GRAM of the display screen (this process can also be referred to as image loading). Finally, the DDIC refreshes the image data in the GRAM onto the panel (this process can also be referred to as image refresh), thus realizing image display.

[0054] DDIC can control the writing and reading of image data in GRAM through the Tear Effect (TE) signal.

[0055] See Figure 2 The TE signal can be a periodic pulse signal. After the DDIC completes refreshing a frame of image on the panel, it generates a TE signal, triggering the display driver to write the new frame of image data into the GRAM. For example, the DDIC... Figure 3After the image data of the last line M in the GRAM is refreshed (read) to the panel, the DDIC can generate a TE signal. The display driver detects the rising edge of the TE signal, such as... Figure 2 After TE (up) arrives, it can be Figure 2 During the time period indicated by the "Send Image (short for sending image data from the display driver to the GRAM)" flag X, the image data of a new frame is written into the GRAM.

[0056] Subsequently, after the falling edge of the TE signal arrives, the DDIC can refresh the image data of the new frame in the GRAM onto the panel, allowing the panel to display the new frame of image. See also... Figure 2 and Figure 3 DDIC from Figure 3 As shown, after the last row M in the GRAM returns (along the direction indicated by the dashed arrow in the diagram) to the first row N in the GRAM, just as the falling edge of the TE signal arrives, the DDIC begins reading the image data of the new frame from the GRAM and refreshing it onto the panel, as shown in... Figure 2 During the time period indicated by the refresh flag Y after the arrival of TE (bottom), the image data of the new frame in GRAM is refreshed to the panel.

[0057] The panel can be refreshed via a source signal. See also... Figure 2 After the falling edge of the TE signal, the DDIC reads image data from the GRAM. At this time, the source signal generated by the panel is high. This high level illuminates the corresponding pixels on the panel based on the image data read by the DDIC, thus refreshing the image on the panel. After refreshing, the panel can pull the source signal low to reduce power consumption when not refreshing the image. It should be noted that during the phase when the source signal is low, the image will not be refreshed; this state can still be called the hold state.

[0058] As can be seen, the TE signal allows control over the process. After each frame refresh, the display driver writes image data to the GRAM based on the rising edge of the TE signal, and the DDIC reads image data from the GRAM based on the falling edge of the TE signal. Furthermore, by controlling the rate at which the display driver writes to the GRAM and the rate at which the DDIC reads from the GRAM, the progress of the DDIC reading from the GRAM is always behind the progress of the display driver writing to the GRAM, thus preventing screen tearing.

[0059] Furthermore, in game scenarios, electronic devices can use the following scheme (also known as the Xsync scheme) for image processing and display: See Figure 4Upper layers, such as game applications and image compositors, can perform drawing / rendering and compositing approximately every 16.6ms to obtain a frame of image, achieving a game frame rate of 60FPS. It should be noted that game applications and image compositors can perform image processing by calling the CPU, GPU, etc. Meanwhile, the TE signal has a frequency of 120Hz, and the display driver (including crtc_commit) can send the image to the display screen at a refresh rate of 120Hz, for example, triggering the display driver to send an image approximately every 8.3ms.

[0060] In other words, the display driver can detect two TE signals during the processing time of each frame of image. If the upper layer can process and obtain a frame of image quickly, the display driver can send the image when it detects the TE signal for the first time, which can reduce the time the display driver waits for the TE signal and send the image to the display screen for display more promptly.

[0061] In this way, electronic devices can respond to user actions more promptly. For example, if a user performs a shooting action in a game, the electronic device's display can more immediately show the shooting effect, such as the effect of firing bullets. This improves the responsiveness of the game.

[0062] In reality, the time interval between processing a few frames in the upper layer (such as game applications and image synthesizers) is uncontrollable. It cannot be guaranteed that the processing of a frame of image will be completed every 16.6ms. This may result in large fluctuations in the game frame rate and a very poor visual experience.

[0063] See also Figure 4 The upper layer completes the processing of Figure A within the first 16.6ms. The display driver can then send Figure A at the end of the first 16.6ms, and Figure A will then be displayed on the screen. Next, the upper layer sends Figure B at the end of the second 16.6ms. At this point, the display duration of Figure A is 16.6ms, and Figure B will then be displayed on the screen. However, the processing of Figure C is not completed within the third 16.6ms. At the end of the third 16.6ms, the image compositer is still performing the composite of Figure C, as shown by the black-filled square in the figure. Therefore, the display driver cannot send Figure C at the end of the third 16.6ms, and Figure B will continue to be displayed on the screen. In the next 8.3ms, the upper layer completes the processing of Figure C. The display driver can then send Figure C at the end of this 8.3ms. At this point, the display duration of Figure B is 25ms, and Figure C will then be displayed on the screen. Next, the upper layer completes the processing of Figure D at the end of the next 8.3ms. The display driver then sends Figure D at the end of the next 8.3ms. Thus, the display time of Figure C is 8.3ms, and then Figure D will be displayed on the screen.

[0064] In the above Figure 4 In the example, because the processing of image C was not completed within the third 16.6ms interval, image C was processed slowly. This resulted in image A displaying correctly in 16.6ms, image B displaying in 25ms, and image C displaying in 8.3ms. Clearly, the game's frame rate fluctuated significantly. Furthermore, the human eye is highly perceptible to frame rate fluctuations, and such large fluctuations lead to a severely poor visual experience.

[0065] To address the aforementioned issues, this application provides an image display method. In this method, the frequency of the TE signal is kept consistent with the frame rate of the upper-layer application. For example, if the frame rate of a game application is 60 FPS, corresponding to a frequency of 60 Hz, then the frequency of the TE signal is also 60 Hz. After obtaining the processed image data, and if the waiting time for the TE exceeds a preset duration, such as 2.7 ms, the display driver can perform image delivery (this process can also be called frame tracking) ahead of time, promptly sending the image to the display screen for display. This avoids response delays caused by excessively long TE waiting times, thereby improving response timeliness. Furthermore, since the frequency of the TE signal is consistent with the application frame rate, the display screen can refresh the image at time intervals consistent with the frame rate, preventing significant frame rate fluctuations and improving the user's visual experience.

[0066] Therefore, it should be noted that the embodiments of this application can be used in game scenarios, or in other image processing scenarios with high requirements for response timeliness. This application does not make any specific limitations on this.

[0067] For example, the electronic devices in this application embodiment may be mobile phones, tablets, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices, etc., that have image processing and display requirements. This application embodiment does not impose any special limitations on the specific form of the electronic device.

[0068] See Figure 5 This is a hardware structure diagram of an electronic device. For example... Figure 5As shown, taking a mobile phone as an example, the electronic device may include a processor 210, an external memory interface 220, an internal memory (RAM) 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0069] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0070] The processor 210 may include one or more processing units, such as an application processor (AP), a modem, a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a modem, and / or a neural network processing unit (NPU).

[0071] Different processing units can be independent devices or integrated into one or more processors. For example, an AP can integrate a CPU, GPU, etc.

[0072] In some embodiments, the processor 210 can execute an image display method by running instructions stored in the internal memory 221.

[0073] In addition, one or more processing units in the aforementioned processor 210, as well as other components of the mobile phone (such as memory, input / output interfaces, etc.), can be integrated into the SoC.

[0074] The charging management module 240 receives charging input from the charger. The power management module 241 connects the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, providing power to the processor 210, internal memory 221, display screen 294, camera 293, and wireless communication module 260, etc. The power management module 241 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance).

[0075] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor.

[0076] Mobile phones can achieve camera functions through the camera 293, ISP, video codec, GPU, display 294, application processor (AP), neural network processor (NPU), etc.

[0077] Mobile phones can perform audio functions, such as music playback and recording, through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, and an application processor.

[0078] Mobile phones can achieve display functions through GPU, display screen 294, and application processor. The GPU is a microprocessor for image processing, connecting the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering.

[0079] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, a mobile phone may include one or more displays screens 294.

[0080] The display screen 294 also includes a driving circuit (such as a DDIC, which will be used as an example below) and a storage unit (such as GRAM, which will be used as an example below). The GRAM can be used to store image data of the image to be displayed. The DDIC can control the refresh rate of the display screen 294, such as controlling the refresh of image data in the GRAM to the display panel and the writing of image data in the GRAM.

[0081] For details on the functions and working principles of the display panel, DDIC, and GRAM, please refer to the previous text. Figure 1 and Figure 2 The details and related information will not be elaborated here.

[0082] The display panel can be a low-temperature polycrystalline oxide (LTPO) display panel, a low-temperature polycrystalline silicon (LTPS) display panel, etc.

[0083] Furthermore, based on the different numbers of transistors in the pixel circuits of the LTPO display panel, the display screen 294 can be divided into 7T-LTPO screens and 8T-LTPO screens. 7T means that the pixel circuit includes 7 transistors, and 8T means that the pixel circuit includes 8 transistors.

[0084] See Figure 6 In (a) of the LTPS / 7T-LTPO screen, the DDIC can generate a TE signal of up to 120Hz, so the refresh rate can reach 120Hz, and the image on the display panel can be refreshed once every 8.3ms.

[0085] Of course, the DDIC of the LTPS / 7T-LTPO screen can also generate TE signals below 120Hz (such as 90Hz, 60Hz, etc.).

[0086] For example, the 90Hz TE signal generated by the DDIC in the LTPS / 7T-LTPO screen, such as Figure 6 As shown in (b), the refresh rate can reach 90Hz, and the image on the display panel can be refreshed once every 11.1ms.

[0087] For example, the 60Hz TE signal generated by the DDIC in an LTPS screen... Figure 6 As shown in (c), the refresh rate can reach 60Hz, and the image on the display panel can be refreshed once every 16.6ms.

[0088] Therefore, it should be noted that the duration of the low-level TE signal is the same in the 120Hz TE signal generated by the DDIC in the LTPS / 7T-LTPO screen, the 90Hz TE signal generated by the DDIC in the LTPS / 7T-LTPO screen, and the 60Hz TE signal generated by the DDIC in the LTPS screen. Correspondingly, the lower the frequency and the longer the period, the longer the duration of the high-level signal. Figure 6 In (c), the duration of the high level of the TE signal is > Figure 6 In (b) of the text, the duration of the high level of the TE signal is > Figure 6 The duration of the high level of the TE signal in (a) is shown.

[0089] However, as mentioned above Figure 6 The difference between the 60Hz TE signal generated by the DDIC in the LTPS screen shown in (c) is: Figure 6 As shown in (d), the 60Hz TE signal generated by the DDIC in the 7T-LTPO screen is equivalent to generating one TE signal every other TE signal from the 120Hz TE signal. Therefore, the duration of the low level of the 60Hz TE signal generated by the DDIC in the 7T-LTPO screen is approximately twice the duration of the low level of the 120Hz TE signal generated by the DDIC in the LTPS / 7T-LTPO screen.

[0090] See Figure 6 In section (e), the DDIC in an 8T-LTPO screen can generate a TE signal of up to 360Hz, thus achieving a refresh rate of 360Hz, which means the image on the display panel can be refreshed approximately once every 2.7ms. Of course, the DDIC in an 8T-LTPO screen can also generate TE signals lower than 360Hz (such as 120Hz or 60Hz), and correspondingly, the refresh rate can be lower than 360Hz.

[0091] It should be noted that in an 8T-LTPO screen, compared to a 360Hz TE signal: a 120Hz TE signal is equivalent to generating one TE signal every two TE signals of the 360Hz TE signal. A 60Hz TE signal is equivalent to generating one TE signal every five TE signals of the 360Hz TE signal.

[0092] The image display method provided in this application embodiment can be applied to the above. Figure 6 The various TE signal scenarios shown are illustrated.

[0093] The software system of the aforementioned electronic device (such as the software system of an AP) can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture based on Android. TMTaking a system as an example, this section illustrates the software structure of an electronic device. A layered architecture divides the software system of an electronic device into several layers, each with a clear role and function, and the layers communicate with each other through software interfaces.

[0094] See Figure 7 Taking mobile phones as an example, the hardware and software architecture of electronic devices can include the application layer, application framework layer, hardware abstraction layer (HAL), kernel layer, and hardware layer, etc.

[0095] The application layer can accommodate various applications such as email, video player, memo, alarm clock, settings, and games. When these applications are running in the foreground, the screen can refresh to display multiple frames of images. For example, a game application can display multiple frames of game footage through screen refresh.

[0096] The application framework layer provides the application layer with an application programming interface (API) and a programming framework. The application framework layer may include a surface-flinger (SF). It should be noted that the term "surface-flinger" can also be translated as "surface thrower," "surface drawing module," or "image compositing service," etc., and this application does not specifically limit the terminology used.

[0097] Image compositers can be used for refresh rate control, image compositing control, and more.

[0098] In one specific implementation, the image synthesizer can simulate the corresponding Vsync signal based on the TE signal. This Vsync signal can trigger the CPU, GPU, etc. to perform image processing, thereby controlling the frequency of the obtained image data and realizing refresh rate control.

[0099] In one specific implementation, the image compositor, after receiving multiple layers with drawn content, can allocate compositing tasks to either the Hardware Composer (HWC) in the Hardware Abstraction Layer or the GPU. For example, if the compositing task includes tasks such as rounded corner cropping, it can be assigned to the GPU for image compositing; if the compositing task only includes simple layer overlay tasks, it can be assigned to the HWC for image compositing. This allows for control over image compositing.

[0100] In some embodiments, the image synthesizer can be used to enable or disable the frame tracking scheme.

[0101] For example, the image compositor can receive settings to enable or disable frame tracking for a specific application, and then write these settings to the display driver, such as via Android Interface Definition Language (AIDL). This allows the display driver to identify applications with frame tracking enabled. Subsequently, based on these applications, the display driver can query whether the foreground application has frame tracking enabled.

[0102] Furthermore, the settings for enabling or disabling frame tracking for a specific application can include an on / off time, thus accurately indicating when frame tracking is enabled or disabled. Correspondingly, when querying whether a foreground application has frame tracking enabled, the display driver can detect whether it is within the valid time frame tracking is enabled or disabled based on the on / off time, thereby accurately determining whether frame tracking is enabled.

[0103] The Hardware Abstraction Layer (HWC) runs in user space, encapsulates kernel-level drivers, and provides calling interfaces to higher layers. The HWC can include image compositing and processing, as well as providing the composited image data to the display driver.

[0104] It should be noted that the image data obtained by the GPU in performing image synthesis can also be sent to the display driver via HWC.

[0105] The kernel layer includes drivers that drive the hardware, such as display drivers. Display drivers can be used to send the synthesized image data to the display screen, such as writing image data to GRAM and then refreshing it to the display screen in response to the arrival of the TE signal.

[0106] In some embodiments, the display driver can also write image data to GRAM and then refresh it to the display screen before the TE signal arrives after waiting for a preset time period, so as to achieve advance image delivery.

[0107] The kernel layer may also include a Display Serial Interface (DSI). DSI can be used for data transmission between the display driver and the display screen, such as transmitting image data.

[0108] The hardware layer may include hardware such as CPU, GPU, and display screen (further including display panel, DDIC, GRAM, etc.).

[0109] Furthermore, image data can be transmitted between the kernel layer and the hardware layer through the Mobile Industry Processor Interface (MIPI).

[0110] At this point, it needs to be clarified that the above... Figure 5 and Figure 7 In the descriptions of electronic devices, the division of hardware and software modules is exemplary and not a limitation in practice. For example, the core display driver function in this article may also be implemented by other modules or merged into other modules.

[0111] The image display method provided in this application can be executed in an electronic device having the above-described hardware structure and software architecture. The image display method of this application embodiment will be further described in detail below with reference to the above-described hardware structure and software architecture.

[0112] See Figure 8 The image display method includes the following steps:

[0113] S801, the display screen generates a TE signal at frequency 1.

[0114] For example, the DDIC in the display screen can generate a TE signal.

[0115] Frequency 1 can be synchronized with the frame rate of the application (such as a game application), unlike the previous... Figure 4 Similarly, the frequency of the TE signal is set to be higher.

[0116] Taking a game application as an example, if the game frame rate is 60 FPS, the corresponding frequency is 60 Hz, then frequency 1 is 60 Hz; if the game frame rate is 90 FPS, the corresponding frequency is 90 Hz, then frequency 1 is 90 Hz.

[0117] It should be noted that S801 is a continuous process; the display can continuously generate a TE signal at frequency 1, and not... Figure 8 The execution timing shown is limited.

[0118] S802, the application program, and the image synthesizer process the data to obtain the k-th frame image.

[0119] The application can perform drawing / rendering processes, such as calling the CPU or GPU to complete image drawing / rendering processes, and obtain a layer with image content drawn on it. For example, the image compositor can simulate a Vsync signal based on the TE signal, and distribute a Vsync signal to the application after each Vsync signal arrives, thereby triggering the application to start processing a new frame of image.

[0120] The image compositor can call upon the GPU, hardware compositor, etc. to perform compositing processing.

[0121] The application can be a game or other application that requires high responsiveness. Once the application is running in the foreground, it will continuously process and obtain each frame of the image according to the application's frame rate.

[0122] The k-th frame image can be any frame image after the application is launched, such as the 1st frame image, the 2nd frame image, and so on. That is, k ≥ 1, and k is an integer.

[0123] S803, The image synthesizer sends the k-th frame image to the display driver.

[0124] Normally, after receiving an image, the image synthesizer can send the image upon detecting the arrival of a TE signal, or upon the arrival of a TE signal pulse, rising edge of the TE signal, etc. However, in this embodiment, the display driver can decide the timing of image sending based on the duration of waiting for the TE signal, as shown in S804 and S808 below.

[0125] S804. If the waiting time does not exceed the preset time, in response to the arrival of the TE signal, the display driver executes the sending of the k-th frame image.

[0126] The display driver can detect the arrival of the TE signal. See also Figure 9 If the display driver receives the k-th frame image before the TE signal arrives, the display driver will receive the k-th frame image at time t1, while the next TE signal will not arrive until time t2. Therefore, it is necessary to wait for a period of time before the arrival of the TE signal can be detected.

[0127] If the display driver detects the arrival of the TE signal before the waiting time exceeds the preset time, it indicates that the image can be sent promptly based on the arrival of the TE signal. Therefore, in response to the arrival of the TE signal, the display driver executes the sending of the k-th frame image, thereby enabling the k-th frame image to be sent to the display screen for display in a timely manner.

[0128] See also Figure 9 If the time interval between time t1 and time t2 is less than the preset duration, the display driver can send the k-th frame image at time t2.

[0129] The preset duration can be flexibly set by those skilled in the art according to their needs, and this application does not impose specific limitations on it. For example, the preset duration can be 2.7ms, 3ms, etc.

[0130] It should be noted that this application does not specifically limit the implementation method for detecting whether the waiting time exceeds the preset time.

[0131] For example, the display driver can start timing after receiving an image (such as the k-th frame). If the arrival of a TE signal is detected before the preset duration is reached, it indicates that the TE signal has been detected without exceeding the preset duration. If the TE signal has not been detected by the preset duration, it indicates that the waiting time for the TE signal has exceeded the preset duration.

[0132] For example, after receiving an image (such as the k-th frame), the display driver can start a timer (referred to as Timer 1) to wait for a preset duration. If a TE signal is detected before the timer 1 expires, it indicates that the TE signal has been detected before the preset duration has been exceeded. If no TE signal is detected after the timer 1 expires, it indicates that the waiting time for the TE signal has exceeded the preset duration.

[0133] S805, The display screen shows the image of the kth frame.

[0134] See also Figure 9 After the display driver sends the k-th frame image at time t2, the display screen can then display the k-th frame image. It should be noted that the actual moment the k-th frame image is refreshed and displayed on the display panel is the end of the TE signal pulse or the end of its falling edge; see the previous text for details. Figures 1-3 The relevant information will not be repeated here.

[0135] S806, the application, and the image synthesizer process to obtain the (k+1)th frame image.

[0136] S807, The image synthesizer sends the (k+1)th frame image to the display driver.

[0137] S808. If the waiting time for the TE signal exceeds the preset time, the display driver will send the image of the (k+1)th frame ahead of schedule.

[0138] If the waiting time for the TE signal exceeds the preset time, the display driver can send the image in advance to avoid delays in image delivery due to excessive waiting time and improve the timeliness of image delivery.

[0139] See also Figure 9 If the display driver receives the (k+1)th frame image at time t3, and the next TE signal arrives at time t4, and the time interval between time t3 and time t4 exceeds the preset duration, such as when it is detected at time t5 that the preset duration has been exceeded, the display driver can execute the image transmission of the (k+1)th frame image before time t4, such as at time t5.

[0140] In some embodiments, the display driver can send the image when it detects that the waiting time exceeds a preset time, thereby enabling early image sending.

[0141] In other embodiments, after detecting that the waiting time exceeds a preset time, the display driver can perform image transmission at a target time. This target time can be a pre-set image transmission time between the arrival times of two adjacent TE signals.

[0142] There can be one or more preset image delivery times, with the target time being one of them. For example, the target time is the first adjacent image delivery time after the waiting time reaches the preset time among multiple image delivery times. Or, if the moment when the waiting time reaches the preset time is exactly an image delivery time, then the target time is that image delivery time.

[0143] When there are multiple map submission times, the interval between two adjacent submission times can be fixed or variable. This article mainly focuses on the case where there are multiple map submission times and the interval between two adjacent submission times is fixed.

[0144] In one specific implementation, a timer (which can be referred to as Timer 2) that determines the timing of image delivery can be used to indicate multiple image delivery times. For example, the frequency of Timer 2's timing can be 360Hz, meaning an image delivery time will occur approximately every 2.7ms. Of course, the frequency of Timer 2's timing can also be 240Hz, 480Hz, etc., and this application does not specifically limit this.

[0145] It's worth noting that the TE signal is generated by the DDIC based on the display's clock, while Timer 2 is generated based on the SoC's clock. It should be understood that the screen clock and the SoC clock differ in frequency, error, and other aspects. Therefore, after each detection of the rising edge of the TE signal, the display driver can initialize / calibrate Timer 2. For example, resetting Timer 2 ensures that it starts timing based on the time of the current detected rising edge of the TE signal, preventing TE signal misalignment due to clock differences and further avoiding screen flickering.

[0146] Taking timer 2 with a timing frequency of 360Hz as an example:

[0147] See Figure 10In (a) of the 7T-LTPS screen, the TE signal at 60Hz has a period of approximately 16.6ms. The timing of Timer 2 can be set to 2.7ms, 5.5ms, 8.3ms, 11.1ms, 13.9ms, and 16.6ms after the rising edge of each TE signal, corresponding to the six image transmission times following the rising edge of each TE signal in the diagram. That is, there are six image transmission times within one TE cycle. It should be noted that for ease of viewing, the last image transmission time within each TE cycle (the transmission time corresponding to 16.6ms) does not completely coincide with the rising edge of the TE signal; in practice, they can coincide. The same applies to the last image transmission time within each TE cycle in the following text.

[0148] See Figure 10 In (b) of the 7T-LTPO screen, the TE signal at 60Hz also includes 6 image transmission times within one TE cycle.

[0149] See Figure 10 In (c) of the diagram, the TE signal at 90Hz has a period of approximately 11.1ms. The timing of Timer 2 can be set to 2.7ms, 5.5ms, 8.3ms, and 11.1ms after the rising edge of each TE signal, corresponding to the four image transmission times following the rising edge of each TE signal in the diagram. That is, one TE cycle includes four image transmission times.

[0150] Furthermore, when the timer 2 reaches its set time, a target signal can be triggered, as described above. Figure 10 At each image delivery moment, a target signal can be generated. This target signal is used to trigger the display driver to execute image delivery. After detecting that the waiting time exceeds a preset time, the display driver can execute image delivery upon detecting the target signal, thereby achieving image delivery at the target time.

[0151] For example, target signal detection is activated after the waiting time exceeds a preset duration, allowing the target signal to be detected. Alternatively, after detecting the target signal and sending the image, target signal detection can be deactivated to avoid sending images based on the target signal in scenarios where pre-sending is not required. Furthermore, with target signal detection enabled, the display driver cannot detect the arrival of the TE signal. Therefore, timely deactivation of target signal detection ensures that the arrival of every TE signal is detected, thereby triggering initialization timer 2.

[0152] Therefore, it should be noted that after the detection that the waiting time exceeds the preset time, there may only be one image transmission time that coincides with the arrival time of the next TE signal. For example, after the second to last timing time within a TE cycle, such as... Figure 10 (a) and Figure 10 After the image delivery time corresponding to the timing of 13.9ms in (b) of the image, Figure 10 After the image transmission time corresponding to the 8.3ms timing in (c) of the image transmission time, if the display driver detects that the waiting time exceeds the preset time, then until the arrival of the next TE signal, there will only be one image transmission time that coincides with the arrival time of the next TE signal, such as... Figure 10 (a) and Figure 10 In (b), the timing interval of 16.6ms corresponds to the image delivery time. Figure 10 The timing of 11.1ms in (c) corresponds to the image sending time.

[0153] In this case, the display driver can send the image when the next TE signal arrives. This way, the display driver can avoid enabling target signal detection, thus ensuring that the arrival of the next TE signal can be detected and used to initialize Timer 2.

[0154] In other words, in the embodiment where image transmission is performed at the target time, if there is an image transmission time between the detection of a waiting time exceeding the preset time and the arrival time of the next TE signal, image transmission can be performed at that time to achieve early image transmission.

[0155] S809, The display screen shows the image of the (k+1)th frame.

[0156] See also Figure 9 After the display driver sends the (k+1)th frame image at time t5, the display screen can then display the (k+1)th frame image.

[0157] Therefore, it should be noted that for the parts not described in detail in S806-S809 above, please refer to the relevant descriptions in S802-S805.

[0158] S811, the application, and the image synthesizer process to obtain the k+2th frame image.

[0159] S812, The image synthesizer sends the k+2th frame image to the display driver.

[0160] S813, In response to the arrival of the TE signal, the display driver executes the sending of the k+2th frame image.

[0161] After the display driver performs early image delivery for the (k+1)th frame, it does not deliver the (k+2)th frame (i.e. the next frame) in advance. This avoids large frame rate fluctuations when returning to the normal frame rate after continuous early image delivery, thus reducing frame rate fluctuations and improving the visual experience.

[0162] In other words, after obtaining the (k+2)th frame image, regardless of whether the waiting time for the TE signal to arrive exceeds the preset time, the display driver will send the (k+2)th frame image when the TE signal arrives.

[0163] See also Figure 9 The display driver receives the (k+2)th frame image at time t6. At time t7, the waiting time exceeds the preset time. However, the display driver executes the image transmission of the (k+2)th frame image when it detects the arrival of the TE signal at time t8.

[0164] Of course, the display driver can also continuously send images ahead of time for the (k+2)th frame if the waiting time exceeds the preset time.

[0165] S814, The display screen shows the (k+2)th frame of the image.

[0166] See also Figure 9 After the display driver sends the (k+2)th frame image at time t8, the display screen can then display the (k+2)th frame image.

[0167] It should be noted that for the parts not explained in detail in S811-S814 above, please refer to the relevant explanations in S802-S805.

[0168] It is understandable that subsequent frames after the (k+2)th frame can also be processed according to the same display procedure as frames k to k+2. Specifically, if the waiting time does not exceed the preset time, the image can be displayed according to the procedure for the kth frame, executing the image display upon detection of the TE signal. If the waiting time exceeds the preset time and the previous frame was not pre-displayed, the image can be pre-displayed according to the procedure for the (k+1)th frame. If the previous frame has already been pre-displayed, the image can be displayed according to the procedure for the (k+2)th frame, executing the image display upon detection of the TE signal.

[0169] In some scenarios, the display driver's premature image delivery may lead to frame drops. See also... Figure 11At time t9, the display driver sends the (k+n-1)th frame image (n>2, where n is an integer, e.g., the (k+n-1)th frame image is the (k+2)th frame image mentioned earlier), writing it into the GRAM. At or before time t10, if the display driver receives the (k+n)th frame image and detects that the waiting time exceeds a preset time, the display driver can send the image earlier at time t10, writing the (k+n)th frame image into the GRAM. It should be noted that between time t9 and time t10, until the falling edge of the TE signal ends, the DDIC does not refresh the (k+n-1)th frame image from the GRAM onto the display panel. At the next time t11, the falling edge of the TE signal ends, and the DDIC refreshes the newly written (k+n)th frame image from the GRAM onto the display panel. Clearly, the (k+n-1)th frame image is lost during this process, resulting in frame loss.

[0170] To address this frame drop issue, the following process (S815-S818) can be used to perform the display sending process. It should be noted that if there is no frame drop issue, S815-S818 can be omitted, and the display driver can perform the advance display sending process similar to that of the (k+1)th frame image described above.

[0171] The S815, application, and image synthesizer process the image to obtain the k+nth frame.

[0172] S816, The image synthesizer sends the k+nth frame image to the display driver.

[0173] S817. If the waiting time for the TE signal exceeds the preset time, but sending the image of the k+n frame in advance would cause frame loss, the display driver will send the image of the k+n frame in response to the arrival of the TE signal.

[0174] For example, the preceding text Figure 11 The example shown illustrates a situation where waiting for the TE signal exceeds the preset time, but sending the k+n frame image prematurely results in frame loss.

[0175] In some embodiments, the display driver can detect the timing of early image delivery for the (k+n)th frame, such as detecting a time when the waiting time exceeds a preset time, a target time, etc., and whether it falls within a high-level range of a TE signal, i.e., after the rising edge and before the falling edge. If the early image delivery timing falls within the high-level range of the TE signal, as described above... Figure 11 If the timing t10 ​​is in the high-level range, the display driver can determine that sending the image prematurely will cause frame drops. If the timing of sending the image prematurely is not in the high-level range of the TE signal, the display driver can determine that sending the image prematurely will cause frame drops.

[0176] Furthermore, in the implementation where the timing of image transmission is indicated by the timing of Timer 2, and the timing is at a fixed frequency, such as 360Hz, the corresponding position of the timing in the TE signal is determined. For example, in the preceding text... Figure 10 In example (a), the timing intervals of 2.7ms, 5.5ms, and 8.3ms are located in the high-level range of the TE signal. For another example, in the preceding text... Figure 10 In (c) of the diagram, the 2.7ms timing interval falls within the high-level range of the TE signal. Based on this, the display driver can determine whether sending the image prematurely will cause frame drops based on the timing interval corresponding to the target time. (Continuing with...) Figure 10 Taking (a) as an example, if the target time is 2.7ms, 5.5ms, or 8.3ms, sending the image in advance will result in frame loss.

[0177] It should be noted that in the implementation of sending images one frame in advance, as shown in the image sending process for the (k+1)th and (k+2)th frames above, if the image for the (k+n-1)th frame is sent when the TE signal arrives, as follows... Figure 11 As shown, the (k+n)th frame may be sent ahead of schedule. Therefore, if the (k+n-1)th frame is not sent during the high-level range of the TE signal, then frame loss will not occur.

[0178] In other embodiments, the display driver can detect whether the timing of sending the (k+n)th frame image in advance and the timing of sending the (k+n-1)th frame image are between the falling edges of two adjacent TE signals. If they are between the falling edges of two adjacent TE signals, it indicates that when the next falling edge arrives, the (k+n)th frame image can be refreshed to the display panel, while the (k+n-1)th frame image will be lost. If they are not between the falling edges of two adjacent TE signals, it indicates that before the timing of sending the (k+n)th frame image in advance, the (k+n-1)th frame image has already been refreshed to the display panel on the falling edge of the preceding TE signal among the two adjacent TE signals. Subsequently, when the (k+n)th frame image is sent in advance, the (k+n)th frame image is sent, and the (k+n)th frame image is refreshed to the display panel on the falling edge of the following TE signal among the two adjacent TE signals, thus preventing frame loss.

[0179] This embodiment is applicable not only to the aforementioned method of sending images in advance at one-frame intervals, but also to the method of sending images in advance continuously.

[0180] If a potential frame drop is detected, the display driver will not perform pre-loading of the image. Instead, it will load the (k+n)th frame only when the TE signal is detected. This avoids frame drop caused by pre-loading the image.

[0181] See Figure 12At time t11, the display driver receives the k+n frame image and detects that the waiting time exceeds the preset time. It determines that the timing for sending the image ahead of time is time t12. Time t12 is exactly in the high level range of the TE signal, which may cause frame loss. Therefore, the display driver only executes the sending of the k+n frame image when the TE signal is detected at time t13.

[0182] S818, The display screen shows the image of the k+nth frame.

[0183] See also Figure 12 After the display driver sends the k+n frame image at time t13, the display screen can then display the k+n frame image.

[0184] Furthermore, to avoid continuous frame drops due to premature image delivery, thus preventing the inability to deliver images ahead of time, the display driver can control the frequency of the TE signal to be increased from frequency 1 to frequency 2, run at frequency 2 for at least one frame, and then return to frequency 1. Increasing the TE signal frequency to frequency 2 reduces the time the display driver waits for the TE signal, allowing for more timely image delivery. Moreover, after returning to frequency 1, the problem of frame drops due to premature image delivery usually does not recur, thus enabling premature image delivery even with long waiting times.

[0185] Frequency 2 can be a frequency one level higher than frequency 1. For example, the TE signal frequencies generated by the display's DDIC can include 60Hz, 90Hz, and 120Hz. Therefore, if frequency 1 is 60Hz, frequency 2 can be 90Hz; if frequency 1 is 90Hz, frequency 2 can be 120Hz. This avoids large fluctuations in frame rate and further enhances the visual experience.

[0186] In one specific implementation, running one frame at frequency 2 then reverts to frequency 1, as shown in S819-S825 below:

[0187] S819, The display driver sends an instruction 1 to the display screen to increase the frequency of the TE signal to frequency 2.

[0188] It should be noted here that the timing of S819's execution is not based on... Figure 8 The above is for illustrative purposes only. In practice, the display driver can send Instruction 1 to the display screen if it detects a possible frame drop. Alternatively, the display driver can also send Instruction 1 to the display screen if it detects the arrival of the TE signal at or before S817.

[0189] S820, the display screen switches to generate a TE signal at frequency 2.

[0190] After the period of the TE signal corresponding to the k+n frame image ends, the display switches to generate a TE signal with frequency 2, that is, DDIC switches to generate a TE signal with frequency 2.

[0191] Taking frequency 1 as 60Hz and frequency 2 as 90Hz as an example, see below. Figure 12 The frequency of the TE signal corresponding to the k+n frame image is 60 Hz. After time t13, DDIC switches the frequency of the TE signal to 90 Hz.

[0192] S821, the application program, and the image synthesizer process the image to obtain the k+n+1th frame.

[0193] S822, The image synthesizer sends the k+n+1th frame of the image to the display driver.

[0194] S823, In response to the arrival of the TE signal, the display driver executes the image sending of the k+n+1th frame.

[0195] During the generation of the TE signal at frequency 2, the display driver sends the image only when the TE signal is detected, instead of sending it in advance. In this way, the display duration of the image only changes from the period corresponding to frequency 1 to the period corresponding to frequency 2, such as from 16.6ms corresponding to 60Hz to 11ms corresponding to 90Hz, without a larger change in frame rate, which can further improve the visual experience.

[0196] It should be understood that because the frequency of the TE signal has been increased, even if the image is not sent in advance, the waiting time for the display driver to wait for the TE signal can be reduced to a certain extent, and the image can be sent faster.

[0197] See also Figure 12 After receiving the k+n+1th frame image, the display driver can detect the arrival of the TE signal at time t14 and execute the image transmission of the k+n+1th frame image.

[0198] S824, The display screen shows the image of the k+n+1th frame.

[0199] See also Figure 12 After the display driver sends the image of the (k+n+1)th frame at time t14, the display screen can then display the (k+n+1)th frame image.

[0200] S825, The display driver sends an instruction 2 to the display screen to restore the frequency of the TE signal to frequency 1.

[0201] It should be noted here that the timing of S825's execution is not based on... Figure 8The above is for illustrative purposes only. In practice, the display driver can send instruction 2 immediately after sending instruction 1 to the display screen, thus indicating a return to frequency 1 after switching to frequency 2. Alternatively, the display driver can send instruction 2 to the display screen before or after detecting the arrival of the TE signal in S823.

[0202] S826, the display screen returns to generating a TE signal at frequency 1.

[0203] After the period of the TE signal corresponding to the k+n+1 frame of the image ends, the display screen resumes to generate a TE signal with frequency 1, that is, DDIC switches to generate a TE signal with frequency 1.

[0204] Taking frequency 1 as 60Hz and frequency 2 as 90Hz as an example, see below. Figure 12 The frequency of the TE signal corresponding to the k+n+1th frame is 90 Hz. After time t14, DDIC restores the frequency of the TE signal to 60 Hz.

[0205] In some embodiments, after the TE signal at frequency 1 is restored, the display of each frame image can be processed according to the process of each frame image at frequency 1 described above, such as the process of each frame image in S802-S818, which will not be repeated here.

[0206] In other embodiments, for the first few frames of the image after the TE signal at frequency 1 is generated, such as the first frame (i.e., the k+n+2th frame), the display driver sends the image only when the TE signal is detected, instead of sending it in advance. In this way, the display duration of the image only changes from the period corresponding to frequency 2 to the period corresponding to frequency 1, such as from 16.6ms corresponding to 60Hz to 11ms corresponding to 90Hz, without a larger frame rate change, which can further improve the visual experience. This article mainly uses this embodiment for illustration, as shown in S827-S830 below:

[0207] S827, the application program, and the image synthesizer process the image to obtain the k+n+2th frame.

[0208] S828, The image synthesizer sends the k+n+2th frame image to the display driver.

[0209] S829. In response to the arrival of the TE signal, the display driver executes the image sending of the k+n+2th frame.

[0210] See also Figure 12 After receiving the k+n+2th frame of the image, the display driver can detect the arrival of the TE signal at time t15 and execute the transmission of the k+n+2th frame of the image.

[0211] S830, the display screen shows the image of the k+n+2th frame.

[0212] See also Figure 12 After the display driver sends the image of the (k+n+2)th frame at time t15, the display screen can then display the (k+n+2)th frame image.

[0213] It is understandable that, during the process of running an application in the foreground, the display of each frame of the image can be referenced as described above. Figure 8 The image submission process is shown for each of the various scenarios.

[0214] If an image frame meets one or more of the following conditions, then early image transmission will not be performed: Condition 1, the waiting time for the TE signal does not exceed the preset time; Condition 2, the previous frame was early image transmission; Condition 3, early image transmission will cause frame loss; Condition 4, the frequency of the TE signal is not frequency 2; Condition 5, the current TE signal frequency is restored to frequency 1 in the first frame.

[0215] If the waiting time for the TE signal exceeds a preset time, then early image transmission can be performed. Furthermore, early image transmission can also be performed under one or more of the following conditions: Condition 1, the previous frame did not undergo early image transmission; Condition 2, early image transmission will not result in frame loss; Condition 3, the TE signal frequency is frequency 1; Condition 4, the current frame is the first frame after the frequency of the TE signal has recovered to frequency 1.

[0216] The preceding embodiments primarily illustrated various scenarios of the image display method provided in this application, focusing on the timing interaction process between modules in the hardware and software architecture. Below, using the display driver as the execution entity, the display process for one frame of image is described:

[0217] For details, see Figure 13 The process of sending and displaying a single frame of an image includes:

[0218] S1301, Obtain the image.

[0219] For example, the image can be any frame from the k-th frame to the (k+n+2)-th frame mentioned above.

[0220] S1302. Check if the foreground application has frame tracking enabled. If yes, proceed to S1303; otherwise, proceed to S1313.

[0221] The frame tracking scheme refers to a scheme that sends images in advance under certain conditions. For example, these certain conditions include waiting time exceeding a preset time or the previous frame not being sent in advance.

[0222] For certain applications, such as some games that require high responsiveness, frame tracking can be used, while for other applications, it is not necessary. Based on this, after acquiring the image, the display driver can check whether the foreground application has frame tracking enabled.

[0223] If the application running in the foreground enables frame tracking, the display driver can execute S1303 and subsequent steps to make the frame tracking decision.

[0224] If the application running in the foreground does not have frame tracking enabled, the display driver can execute S1313 to send the image based on the TE signal.

[0225] Of course, the display driver can also use a frame tracking scheme for all applications, and accordingly, S1302 can be omitted.

[0226] S1303. Check if the previous frame was sent in advance. If not, proceed to S1304; otherwise, proceed to S1313.

[0227] If the previous frame has already performed advance image delivery, there is no need to make a decision on whether to advance the image for the current frame. The display driver can execute S1313 to perform image delivery based on the TE signal. For details, please refer to the previous section on the image delivery process for the (k+2)th frame and related explanations before and after it.

[0228] If the previous frame did not perform advance image delivery, the display driver can execute the following S1304 and subsequent steps to make a decision on whether to perform advance image delivery for the current frame.

[0229] Taking the previous frame as the aforementioned k+2 frame image and the current frame as the k+3 (e.g., n=3) frame image as an example, after obtaining the k+3 frame image, the display driver can detect that the previous frame did not perform advance image sending. Then, for the k+3 frame image, the display driver can use the following S1304 and subsequent steps to decide whether to send the image in advance.

[0230] For example, after performing pre-send image processing, the display driver can record a pre-send flag (1), and then clear flag 1 after processing the next frame. For instance, after pre-sending the (k+1)th frame, the display driver records flag 1, and clears it after processing the (k+2)th frame. In this way, the display driver can detect whether pre-send was performed in the previous frame by reading flag 1. If flag 1 is recorded, it indicates that pre-send was performed in the previous frame; if flag 1 is not recorded, it indicates that pre-send was performed in the previous frame.

[0231] Of course, the display driver can also send images in advance for multiple consecutive frames, that is, there is no need to send images in advance with a one-frame interval. Accordingly, S1303 can also be omitted.

[0232] S1304. Check if the frequency of the TE signal is higher than frequency 1. If yes, proceed to S1305; if no, proceed to S1307.

[0233] If the frequency of the TE signal is higher than frequency 1, it indicates that the frequency of the TE signal has been increased from frequency 1 to frequency 2. The display driver can then execute steps S1305-S1306 to send the image based on the TE signal and return to frequency 1. For details, please refer to the previous section on the image sending process for the (k+n+1)th frame and related explanations.

[0234] If the frequency of the TE signal is not higher than frequency 1, it indicates that the frequency of the TE signal is at frequency 1. The display driver can execute the following S1307 and subsequent steps to further make a decision on whether to send the image in advance for the current frame.

[0235] For example, the display driver can record the frequency changes of the TE signal. After the TE signal frequency switches to frequency 2, such as after sending instruction 1 to the display screen, identifier 2 is recorded. Subsequently, the display driver can query whether the frequency of the TE signal is higher than frequency 1 based on the recorded identifier. If identifier 1 is found, it indicates that the frequency of the TE signal is higher than frequency 1. If identifier 1 is not found, it indicates that the frequency of the TE signal is not higher than frequency 1.

[0236] S1305, In response to the arrival of the TE signal, execute the drawing transmission.

[0237] That is, while the frequency of TE is maintained at frequency 2, the display driver may not perform advance image sending.

[0238] S1306, The frequency of the controlled recovery TE signal is set to frequency 1.

[0239] For example, the display driver can send instruction 2 to the display screen, instructing the display screen to restore the frequency of the TE signal to frequency 1.

[0240] In some embodiments, the frequency of the TE signal can be maintained at frequency 2 for multiple frames. Accordingly, the display driver can record the number of frames the TE signal is maintained at frequency 2. For example, after switching to frequency 2, the frame count is incremented by one each time the arrival of the TE signal is detected. When the frame count reaches 1, the display driver controls the restoration of the TE signal frequency to frequency 1.

[0241] Of course, the display driver may not need to increase the frequency of the TE signal; accordingly, S1304-S1306 can be omitted.

[0242] S1307. Check if the current frame is the first frame after the TE signal at frequency 1 was generated. If yes, execute S1313; otherwise, execute S1308.

[0243] If the current frame is the first frame after the TE signal at frequency 1 was generated, then the image does not need to be sent in advance, and the display driver can execute S1313. For details, please refer to the previous section on the image sending process for the (k+n+2)th frame and related explanations before and after it.

[0244] If the current frame does not return to the first frame after the TE signal at frequency 1 was generated, such as the default frequency 1 or the second, third, etc. frame after returning to frequency 1, the display driver can execute S1308 and its subsequent steps to further decide whether to send the image in advance for the current frame.

[0245] For example, after the TE signal frequency recovers to frequency 1, such as after sending instruction 2 to the display screen, identifier 3 is recorded. Furthermore, the display driver can record the number of frames maintained after the TE signal recovers to frequency 1, such as incrementing the frame count by one each time a TE signal is detected after recovery to frequency 1. Subsequently, the display driver can query the recorded identifier and frame count. If identifier 3 is found and the frame count is 0, it indicates that this is the first frame after the TE signal at frequency 1 was recovered. If identifier 3 is not found, or if identifier 2 is found and the frame count is not 0, it indicates that this is not the first frame after the TE signal at frequency 1 was recovered.

[0246] Of course, the display driver can also choose not to perform pre-sending for multiple consecutive frames after reverting to frequency 1. Accordingly, S1307 above can be updated to: query whether the current frame count is within the preset number of frames since the TE signal at frequency 1 was generated. For example, if identifier 3 is found and the frame count is less than the preset frame count, it indicates that the frame count is within the preset number of frames since the TE signal at frequency 1 was generated. If identifier 3 is not found, or if identifier 2 is found and the frame count is greater than or equal to the preset frame count, it indicates that the frame count is not within the preset number of frames since the TE signal at frequency 1 was generated.

[0247] Starting from the first frame after reverting to frequency 1, the display driver can also perform advance image delivery. Accordingly, the above-mentioned S1307 can be omitted.

[0248] S1308. Check if the waiting time for the TE signal exceeds the preset time. If yes, proceed to S1309; if no, proceed to S1313.

[0249] If the waiting time exceeds the preset time, it indicates that the TE waiting time is too long. The display driver can execute S1309 and its subsequent steps to further determine the timing of image delivery based on whether it will cause frame loss.

[0250] If the waiting time does not exceed the preset time, it indicates that the TE signal can be received in time, and the display driver can execute the following S1313.

[0251] For details on the specific implementation of detecting whether the waiting time exceeds the preset time, please refer to the relevant description above (such as S804), which will not be repeated here.

[0252] S1309. Check if sending the image in advance will cause frame drops. If yes, proceed to S1310; if no, proceed to S1312.

[0253] If sending the image prematurely will cause frame drops, the display driver can execute steps S1310-S1311 to avoid frame drops. For details, please refer to the image sending process for the (k+n)th frame mentioned above and its related explanations.

[0254] If sending the image in advance does not cause frame drops, the display driver can execute S1312 to achieve timely image sending.

[0255] For details on the specific implementation of detecting whether sending images in advance will lead to frame loss, please refer to the relevant explanations in the previous text (such as S817), which will not be repeated here.

[0256] S1310, In response to the arrival of the TE signal, execute the drawing transmission.

[0257] This ensures that no frames are dropped in the current frame.

[0258] S1311, Control the frequency of the TE signal to increase to frequency 2.

[0259] For example, the display driver can send instruction 1 to the display screen, instructing the display screen to increase the frequency of the TE signal to frequency 2.

[0260] Of course, the display driver may not need to increase the frequency of the TE signal, and correspondingly, S1311 can be omitted.

[0261] S1312, Execute advance map delivery.

[0262] For example, the display driver can promptly send the image after detecting that the waiting time exceeds a preset time and no frames are dropped.

[0263] For example, the display driver can send the image at the target time after detecting that the waiting time exceeds the preset time and no frames are dropped.

[0264] For details on the implementation of sending images in advance, please refer to the relevant explanations in the previous text (such as S808), which will not be repeated here.

[0265] S1313, In response to the arrival of the TE signal, execute the image sending.

[0266] Therefore, the following points need to be explained:

[0267] First, the above Figure 13The execution order shown is merely exemplary and is not limited to in actual implementation. In particular... Figure 13 The order of the judgment steps in the process can be flexibly adjusted by those skilled in the art. For example, the judgment in S1308 can be executed first, followed by the judgments in S1304 and S1307. Or, for another example, the judgment in S1307 can be executed first, followed by the judgment in S1304.

[0268] Second, the above Figure 13 Some of the judgment steps can also be omitted from the judgment form. For example, S1304 and S1307 can be omitted, and S1305 and S1306 can be moved after S1311. Then, step A is added before S1305 to obtain the image. And, step B is added after S1306 to obtain the image; step C, in response to the arrival of the TE signal, performs image transmission. In this way, even if it is detected that it would cause frame loss, image transmission will not be performed for three consecutive frames, and the frequency of the TE signal can be switched to frequency 2 and then restored to frequency 1.

[0269] This application also provides an electronic device, which may include a memory and one or more processors (such as a CPU, GPU, NPU, etc.). The memory and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the device in the above method embodiments.

[0270] This application also provides a chip system, such as... Figure 14 As shown, the chip system 1400 includes at least one processor 1401 and at least one interface circuit 1402. The processor 1401 and the interface circuit 1402 are interconnected via lines. For example, the interface circuit 1402 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 1402 can be used to send signals to other devices (e.g., the processor 1401). Exemplarily, the interface circuit 1402 can read instructions stored in memory and send those instructions to the processor 1401. When the instructions are executed by the processor 1401, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0271] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0272] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the image processing method described above.

[0273] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image processing method described above.

[0274] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the image processing methods in the above-described method embodiments.

[0275] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0276] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0278] The unit described as a separate component may or may not be physically separate. The component shown as a unit can be one physical unit or multiple physical units, that is, it can be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0279] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0280] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. An image display method, characterized in that, Applied to an electronic device, wherein a first application runs in the foreground of the electronic device, the method includes: Obtain the first image to be displayed; If the tearing effect TE signal is not received after the waiting time exceeds the preset time, the first image is sent for display at the first moment, which is before the arrival of the next TE signal. The frequency of the TE signal is consistent with the application frame rate of the first application.

2. The method according to claim 1, characterized in that, Before obtaining the first image to be displayed, or after sending the first image to be displayed at a first moment, the method further includes: Obtain the second image to be displayed; If the waiting time for the TE signal does not exceed the preset time, the second image is displayed in response to the arrival of the TE signal.

3. The method according to claim 1 or 2, characterized in that, The first moment includes the first timing period of the first timer, which is the adjacent timing period after the waiting time exceeds the preset time.

4. The method according to any one of claims 1-3, characterized in that, The step of sending the first image for display at the first moment when the tearing effect (TE) signal has not been received after a waiting period exceeding a preset time includes: If the TE signal is not received after a waiting period exceeding the preset time, and sending the image at the first moment will not result in image loss, the first image will be sent for display at the first moment.

5. The method according to claim 4, characterized in that, The method further includes: If the TE signal is not received after a waiting period exceeding the preset time, and sending the image at the first moment would result in image loss, the first image will be sent for display in response to the arrival of the TE signal.

6. The method according to claim 4 or 5, characterized in that, After obtaining the second image to be displayed, and before sending the first image to be displayed, the method further includes: Detect whether the first moment is within the high-level range of the TE signal; If the first moment is within the high-level range of the TE signal, sending a display at the first moment will result in image loss; If the first moment is not within the high-level range of the TE signal, sending the display at the first moment will not cause image loss.

7. The method according to any one of claims 1-4, characterized in that, After displaying the first image at the first moment, the method further includes: Obtain the third image to be displayed, wherein the third image is the frame adjacent to the first image; If the tearing effect TE signal is not received after the waiting time exceeds the preset time, the third image is displayed in response to the arrival of the TE signal.

8. The method according to claim 5, characterized in that, The method further includes: After detecting that sending the display at the first moment would cause image loss, the frequency of the TE signal is switched from the first frequency to the second frequency, where the second frequency is higher than the first frequency; After running at the second frequency for at least one frame, the frequency of the TE signal is restored to the first frequency.

9. The method according to claim 8, characterized in that, In the range operating at the second frequency, the method further includes: Obtain the fourth image to be displayed; In response to the arrival of the TE signal, the fourth image is displayed.

10. The method according to claim 8 or 9, characterized in that, After restoring the frequency of the TE signal to the first frequency, the method further includes: Obtain a fifth image to be displayed, wherein the fifth image is an image within a first number obtained after restoring to the first frequency; In response to the arrival of the TE signal, the fourth image is displayed.

11. An electronic device, characterized in that, include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-10.

12. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-10.

13. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-10.

14. A computer program product comprising computer instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-10.

Images