Display method, electronic device, and storage medium

Abstract

This application provides a display method, an electronic device, and a storage medium. In this method, the electronic device determines a target duration for a second adjustment period based on the synthesis time of at least one display frame within a first adjustment period. Within the second adjustment period, it acquires a first interface image drawn by a target application and determines the expected arrival time of the TE signal for the first display frame corresponding to the first interface image. Based on the expected arrival time of the TE signal and the target duration of the second adjustment period, it determines the frame synthesis trigger time corresponding to the first interface image and synthesizes the first display frame based on the first interface image at the frame synthesis trigger time, thereby displaying the first display frame. In this way, the electronic device can periodically adjust the target duration, thereby reducing the responsiveness during user screen operation, improving responsiveness, increasing the smoothness of screen display, and enhancing the user experience.

Description

Display method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on January 10, 2024, with application number 202410041776.1 and application name “Display method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of smart terminals, and in particular to a display method, an electronic device, and a storage medium. Background Art

[0003] With the advancement of technology, the performance of various terminal devices (such as mobile phones) is getting better and better, and users' performance requirements for various terminal devices are also getting higher and higher. For example, chirality has a significant impact on the user experience of terminal devices.

[0004] Chirality refers to the ability of a terminal device to respond promptly to user actions when the user touches the screen. When using a terminal device, users need to perform various operations on it, such as browsing the web, playing games, and watching videos. When users operate the terminal device screen during use, the terminal device can respond to the user's operation.

[0005] The time it takes for a user to perform an action on the screen and for the corresponding image to appear on the screen is called tracking latency. The lower the tracking latency, the more responsive the terminal device, the better the tracking performance, and the smoother the image. Summary of the Invention

[0006] Embodiments of the present application provide a display method, electronic device, and storage medium. In this method, the electronic device can periodically adjust the time interval between image synthesis and image display, thereby reducing hand tracking delay during user screen operation, improving hand tracking, increasing the smoothness of screen display, and enhancing the user experience.

[0007] In a first aspect, an embodiment of the present application provides a display method. The method includes: in response to the movement of a target operation, a target application in an electronic device draws at least one interface image; the electronic device determines the target duration of a second adjustment period based on the synthesis time of at least one display frame in a first adjustment period; the electronic device obtains the first interface image drawn by the target application in the second adjustment period, and determines the expected tearing effect TE signal arrival time of the first display frame corresponding to the first interface image; the electronic device determines the frame synthesis triggering time corresponding to the first interface image based on the expected TE signal arrival time of the first display frame and the target duration of the second adjustment period; the electronic device synthesizes the first display frame according to the first interface image at the frame synthesis triggering time corresponding to the first interface image; the electronic device displays multiple display frames corresponding to at least one interface image, and the multiple display frames include the first display frame.

[0008] The second adjustment period may be the next adjustment period adjacent to the first adjustment period, and the target duration may be the time interval between the display triggering moment and the synthesis triggering moment of the display frame.

[0009] Specifically, at the synthesis trigger moment, SurfaceFlinger can synthesize the display frame. At the display trigger moment, the display screen can display the synthesized display frame.

[0010] For example, the synthesis trigger moment can be the moment when SurfaceFlinger receives the VsyncSF signal. The display trigger moment can be the moment when the TE signal arrives on the display screen. The target duration can be the time interval between the VsyncSF signal and the TE signal, that is, the SF WorkDuration.

[0011] The target operation may be a user sliding screen operation. The target application may be the application corresponding to the target operation. The synthesis time may be the time taken to synthesize the display frame. Specifically, the synthesis time may be determined by SurfaceFlinger based on the synthesis start time and synthesis end time of the display frame.

[0012] The first interface image may be one of multiple interface images drawn by the target application. The first display frame may be any one of the display frames synthesized within the second adjustment period. The expected TE signal arrival time may be the time when the display screen is expected to display the display frame. It is understandable that each display frame corresponds to an expected TE signal arrival time. The frame synthesis triggering time may be the time when the frame synthesis is triggered.

[0013] In this way, the electronic device can periodically and dynamically adjust the target duration to reduce the tracking delay when the user operates the screen, thereby improving the tracking performance, increasing the smoothness of the screen display, and improving the user experience.

[0014] According to the first aspect, the electronic device determines the target duration of the second adjustment period based on the synthesis duration of at least one display frame within the first adjustment period, which may include: within the first adjustment period, the electronic device determines the target synthesis duration corresponding to the first adjustment period based on the synthesis duration of at least one display frame within the first adjustment period; after the first adjustment period ends, the electronic device determines the preset idle duration, and determines the sum of the target synthesis duration and the preset idle duration as the duration to be adjusted, so as to determine the target duration of the second adjustment period based on the duration to be adjusted.

[0015] The target synthesis time may be a target value of a synthesis time within the adjustment period. For example, the target synthesis time may be the synthesis time of any display frame that exceeds the time threshold, or the average of the synthesis times of multiple display frames that exceed the time threshold, or the maximum synthesis time of each display frame.

[0016] The preset idle time may be a preset idle time after display frame synthesis.

[0017] In this way, the electronic device determines the target duration of the second adjustment period according to the target combined duration corresponding to the first adjustment period and the sum of the preset idle time, thereby achieving dynamic adjustment of the target duration.

[0018] According to the first aspect, or any implementation method of the first aspect above, the electronic device determines the target synthesis time corresponding to the first adjustment period based on the synthesis time of at least one display frame in the first adjustment period, which may include: at the start moment of the first adjustment period, the electronic device sets the target synthesis time to an initial value, and takes each display frame synthesized in the first adjustment period as the current display frame in turn; the electronic device obtains the synthesis time of the current display frame, and when the synthesis time of the current display frame is greater than the target synthesis time, updates the value of the target synthesis time to the synthesis time of the current display frame.

[0019] For example, the initial value may be 0.

[0020] In this way, the electronic device determines the synthesis duration of the current display frame that is greater than the target synthesis duration as the target synthesis duration corresponding to the first adjustment period, thereby being able to more accurately determine the maximum value of the synthesis duration in the first adjustment period.

[0021] According to the first aspect, or any implementation method of the first aspect above, the electronic device determines the target duration of the second adjustment period based on the duration to be adjusted, which may include: the electronic device obtains the first duration; when the duration to be adjusted is greater than the first duration, the electronic device determines the duration to be adjusted as the target duration of the second adjustment period, and when the duration to be adjusted is less than the first duration, the electronic device determines the first duration as the target duration of the second adjustment period.

[0022] The first duration may be a preset minimum SF WorkDuration.

[0023] In this way, the electronic device determines the first duration as the target duration of the second adjustment period when the duration to be adjusted is less than the first duration, thereby avoiding frame loss due to too small SF WorkDuration in the second adjustment period.

[0024] According to the first aspect, or any implementation of the first aspect above, before the electronic device determines the target duration of the second adjustment period based on the synthesis duration of at least one display frame in the first adjustment period, the electronic device may further include: the electronic device determines the target duration of the first adjustment period;

[0025] The electronic device determines the target duration of the second adjustment period based on the synthesis time of at least one display frame within the first adjustment period, which may include: when the synthesis time of all display frames synthesized by the electronic device within the first adjustment period is less than the target duration of the first adjustment period, determining the target duration of the second adjustment period based on the synthesis time of at least one display frame within the first adjustment period.

[0026] In this way, the electronic device adjusts the target duration of the second adjustment period when the synthesis time of all display frames in the first adjustment period is less than the target duration of the first adjustment period, thereby achieving periodic adjustment of the target duration.

[0027] According to the first aspect, or any implementation method of the first aspect above, the electronic device determines the synthesis time of the second display frame within the first adjustment cycle; when the synthesis time of the second display frame is greater than the target time of the first adjustment cycle, the electronic device determines the target time corresponding to the penalty period and starts the penalty period timing; within the penalty period, the electronic device obtains the third interface image drawn by the target application, and determines the expected TE signal arrival time of the third display frame corresponding to the third interface image; the electronic device determines the frame synthesis trigger time corresponding to the third interface image based on the expected TE signal arrival time of the third display frame and the target time corresponding to the penalty period, thereby synthesizing the third display frame according to the third interface image at the frame synthesis trigger time corresponding to the third interface image; after the penalty period timing ends, the next adjustment cycle is determined.

[0028] The target duration of the next adjustment period is the second duration. The second duration may be a pre-set initial value corresponding to SF WorkDuration. The next adjustment period may be the first adjustment period after the penalty period ends.

[0029] The second display frame may be a display frame corresponding to another interface image drawn by the target application. Specifically, the second display frame may be any one of the display frames synthesized within the first adjustment period.

[0030] The third interface image may be another one of the multiple interface images drawn by the target application. Specifically, the third display frame may be any one of the display frames synthesized during the penalty period.

[0031] Specifically, when the synthesis time of the second display frame exceeds the target duration of the first adjustment period, the electronic device starts the penalty period timing. Thus, the start time of the penalty period can be slightly later than the end time of synthesis of the second display frame. In other words, the time interval between the start time of the penalty period and the end time of synthesis of the second display frame can be less than a preset interval value. The preset interval value can be a very small interval value.

[0032] In this way, the electronic device determines the target duration corresponding to the penalty period when the synthesis time of the second display frame is greater than the target duration of the first adjustment period, and can adjust the target duration in time when frame loss occurs, thereby avoiding continuous frame loss and improving user experience.

[0033] According to the first aspect, or any implementation of the first aspect above, the electronic device determines the target duration corresponding to the penalty period, including: the electronic device determines the second duration as the target duration corresponding to the penalty period.

[0034] In this way, the electronic device can adjust the target duration to the second duration when frame loss occurs, so as to avoid continuous frame loss, thereby improving user experience.

[0035] According to the first aspect, or any implementation of the first aspect, the electronic device may determine the start time of the penalty period as the synthesis end time of the second display frame.

[0036] In this way, the electronic device can enter the penalty period in time when frame loss occurs, thereby avoiding continuous frame loss and improving user experience.

[0037] According to the first aspect, or any implementation method of the first aspect above, the electronic device can also determine the synthesis end time of the third display frame and the end time of the penalty period; when the synthesis end time of the third display frame exceeds the end time of the penalty period, the electronic device determines the synthesis end time of the third display frame as the start time of the next adjustment cycle.

[0038] Specifically, the end time of the penalty period can be determined based on the penalty duration and the start time of the penalty period.

[0039] In this way, the electronic device can enter the adjustment cycle again after the penalty period ends, and continue to dynamically adjust the target duration after the penalty period ends, thereby reducing the hand tracking delay during the user's screen operation and improving the user experience.

[0040] According to the first aspect, or any implementation method of the first aspect above, the electronic device determines the target duration of the first adjustment cycle, which may include: when the first adjustment cycle is the first adjustment cycle, the electronic device obtains the second duration, and determines the second duration as the target duration of the first adjustment cycle.

[0041] The first adjustment period may be the first adjustment period after a user operation.

[0042] In this way, the electronic device can determine the second duration as the target duration of the first adjustment cycle.

[0043] According to the first aspect, or any implementation method of the first aspect above, the electronic device determines the synthesis end time of the fourth display frame within the first adjustment period, and the end time of the first adjustment period, and when the synthesis end time of the fourth display frame exceeds the end time of the first adjustment period, the synthesis end time of the fourth display frame is determined as the start time of the second adjustment period.

[0044] The fourth display frame may be a display frame corresponding to another interface image drawn by the target application. Specifically, the synthesis start time of the fourth display frame may be before the end of the first adjustment period or after the end of the first adjustment period. If the synthesis start time of the fourth display frame is after the end of the first adjustment period, the synthesis of the fourth display frame may be the first frame synthesis after the end of the first adjustment period.

[0045] Specifically, the end time of the first adjustment period can be determined according to the period length of the first adjustment period and the start time of the first adjustment period.

[0046] In this way, the electronic device can determine the start time of the second adjustment period, enter the second adjustment period at the start time of the second adjustment period, and then draw, synthesize or display each display frame in the second adjustment period.

[0047] According to the first aspect, or any implementation of the first aspect above, the frequency of the TE signal of the electronic device is greater than the screen refresh frequency of the electronic device.

[0048] For example, the frequency of the TE signal may be 360 ​​Hz, and the screen refresh frequency may be 120 Hz.

[0049] In this way, the electronic device can display the display frames in the high-frequency TE signal scenario to avoid frame loss, thereby increasing the smoothness of the picture display and improving the user experience.

[0050] In a second aspect, an embodiment of the present application provides an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the display method of the first aspect and any one of the first aspects.

[0051] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0052] In a third aspect, an embodiment of the present application provides a computer-readable storage medium comprising a computer program, which, when executed on an electronic device, causes the electronic device to execute the display method of the first aspect and any one of the first aspects.

[0053] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0054] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed, enables a computer to execute a display method as described in the first aspect or any one of the items in the first aspect.

[0055] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0056] In a fifth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the display method according to the first aspect or any one of the first aspects to control the receiving pin to receive a signal and to control the transmitting pin to send a signal.

[0057] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 is a schematic diagram showing a hardware structure of an electronic device;

[0059] FIG2 is a schematic diagram illustrating a software structure of an electronic device;

[0060] FIG3a is a schematic diagram showing an exemplary interface display principle;

[0061] FIG3 b is a schematic diagram illustrating exemplary module interaction during an interface display process;

[0062] FIG4 is a schematic diagram showing exemplary conventional TE signals and high-frequency TE signals;

[0063] Figures 5 and 6 are schematic diagrams illustrating exemplary functional module interactions;

[0064] Figures 7a to 7f are timing diagrams illustrating an exemplary interface display process;

[0065] 8a-8b are schematic timing diagrams illustrating exemplary adjustment periods and penalty periods. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0068] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0069] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0070] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0071] Some terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0072] (1) Frame refers to the smallest unit of a single image in an interface display. A frame can be understood as a still image. Displaying multiple connected frames in rapid succession can create the illusion of motion.

[0073] It should be noted that before the interface displays a frame, it usually needs to go through processes such as frame drawing and frame synthesis.

[0074] Frame drawing refers to the image drawing of the display interface. The display interface can be composed of one or more views. Each view can be drawn by the visual controls of the view system. Each view is composed of subviews. A subview corresponds to a small widget in the view. For example, a subview corresponds to a symbol in the image view.

[0075] Frame synthesis refers to the process of synthesizing multiple or more drawn views into a display interface.

[0076] (2) Vertical scanning (Vsync): A vertical refresh mechanism is a top-to-bottom refresh mechanism. In the vertical refresh mechanism, the electronic device can periodically generate a vertical synchronization signal. The vertical synchronization signal can include a software Vsync signal. The software Vsync signal can trigger the frame drawing and frame synthesis of the electronic device.

[0077] Specifically, the software Vsync signal may include a VsyncAPP signal and a VsyncSF signal. The VsyncAPP signal is used to trigger the frame drawing process. That is, when the application of the electronic device receives the VsyncAPP signal, it can perform frame drawing. The VsyncSF signal is used to trigger the frame synthesis process. That is, when the image synthesis system (SurfaceFlinger) of the electronic device receives the VsyncSF signal, it can perform frame synthesis on the frame after the frame drawing.

[0078] (3) Tearing effect (TE) signal. The TE signal is a signal output by the timing controller of an electronic device to prevent tearing when the screen is refreshed during image display. The TE signal can be understood as a vertical synchronization signal generated by hardware. The TE signal can be set according to the screen refresh frequency. For example, when the screen refresh frequency is 60Hz, the TE signal period can be 16.6ms, that is, the terminal device generates a control signal every 16.6ms to trigger the TE signal period.

[0079] Exemplarily, when the electronic device is ready to refresh the next frame of image, the electronic device generates a TE signal; when the electronic device detects the rising edge (ie, high level) of the TE signal, the display screen of the electronic device starts to display the next frame of image.

[0080] It is understood that the electronic device can pre-set the time intervals between the VsyncAPP signal, the VsyncSF signal, and the TE signal. That is, the electronic device can generate the VsyncSF signal a period of time after generating the VsyncAPP signal, and generate the TE signal a period of time after generating the VsyncSF signal.

[0081] The display method provided in the embodiments of the present application can be applied to electronic devices. Optionally, the electronic devices in the embodiments of the present application can be mobile phones with display functions, sports cameras (GoPro), digital cameras, tablet computers, handheld computers, vehicle-mounted devices, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic devices.

[0082] In order to better understand the embodiments of the present application, the structure of the electronic device according to the embodiments of the present application is introduced below:

[0083] Figure 1 is a schematic structural diagram of an electronic device 100. Optionally, the electronic device 100 may be a terminal, also referred to as a terminal device, which may be a device with a display function such as a cellular phone or a tablet computer, and this application does not limit this.

[0084] It should be understood that the electronic device 100 shown in FIG1 is merely an example of an electronic device, and that the electronic device 100 may have more or fewer components than shown, may combine two or more components, or may have a different component configuration. The various components shown in FIG1 may be implemented in hardware, including one or more signal processing and / or application specific integrated circuits, software, or a combination of hardware and software.

[0085] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an acceleration sensor, a temperature sensor, a motion sensor, an air pressure sensor, a magnetic sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0086] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0087] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0088] The processor 110 may further include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0089] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0090] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0091] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0092] The wireless communication module 160 can provide wireless communication solutions for application on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0093] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0094] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0095] Display screen 194 is used to display images, videos, and receive user operations. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0096] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0097] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye.

[0098] Camera 193 is used to capture still images or videos. The lens generates an optical image of an object and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal.

[0099] The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV and other formats.

[0100] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0101] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121, such as enabling the electronic device 100 to implement the display method in the embodiment of the present application. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0102] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0103] The pressure sensor is used to sense pressure signals and convert the pressure signals into electrical signals. In some embodiments, the pressure sensor can be provided on the display screen 194. The electronic device 100 can also calculate the touch position based on the detection signal of the pressure sensor.

[0104] A touch sensor, also known as a "touch panel," can be mounted on display screen 194. The touch sensor and display screen 194 together form a touch screen, also known as a "touch screen." The touch sensor detects touch operations applied to or near the touch sensor. The touch sensor can communicate the detected touch operations to the application processor to determine the type of touch event.

[0105] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0106] FIG2 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0107] The layered architecture of electronic device 100 divides the software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom, the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0108] The application layer can include a series of application packages.

[0109] As shown in Figure 2, the application package may include a camera, a gallery, and third-party applications with camera functions, etc. For example, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0110] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. It includes various components and services to support developers' Android development. The application framework layer includes some predefined functions.

[0111] As shown in FIG2 , the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, and the like.

[0112] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0113] Content providers are used to store and retrieve data and make it accessible to applications. Data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0114] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0115] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0116] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0117] The application framework layer also includes the input manager and image composition system.

[0118] An input manager is a program that manages input devices. For example, an input manager could be an input system that determines input actions such as mouse clicks, keyboard input, and touch swipes.

[0119] The image synthesis system includes an image synthesis module, a frame loss penalty module, and a cycle adjustment module. The image synthesis module controls frame synthesis and generates vertical synchronization signals. The frame loss penalty module punitively adjusts the tracking delay when it detects frame loss to prevent continuous frame loss. The cycle adjustment module periodically adjusts the tracking delay to reduce the delay during user operation and improve display smoothness.

[0120] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0121] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0122] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0123] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0124] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0125] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0126] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0127] A 2D graphics engine is a drawing engine for 2D drawings.

[0128] The HAL layer is the interface between the operating system kernel and the hardware circuitry. The HAL layer includes, but is not limited to, a camera HAL module and an audio HAL module. The camera HAL module processes image streams, while the audio HAL module processes audio streams (for example, by performing noise reduction, directional enhancement, and other processing).

[0129] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, audio driver, touch screen driver, sensor driver, etc.

[0130] The hardware includes at least a processor, a display, a touch sensor, etc.

[0131] It is understood that the layers in the software structure shown in FIG2 and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0132] It is understandable that, in order to implement the display method in the embodiment of the present application, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0133] The following describes an application scenario of an interface display provided by an embodiment of the present application. In this scenario, an electronic device is taken as an example for explanation.

[0134] As users' demands for mobile phone performance continue to improve, mobile phone interface display technology is also becoming increasingly sophisticated. Users often need to perform various operations on their phones, such as browsing the web, playing games, and watching videos. As users perform operations on the screen, the phone can display images corresponding to the user's operations on the screen.

[0135] For example, when browsing a web page using a mobile phone, the user can refresh the content displayed on the screen by sliding the screen. The mobile phone responds to the user's screen sliding operation and displays the refreshed content on the screen.

[0136] When playing games on a mobile phone, the user can control the actions of the game character by sliding the screen. The mobile phone responds to the user's sliding operation by displaying images corresponding to the game character's actions on the screen.

[0137] When watching a video on a mobile phone, the user can adjust the video progress by sliding the screen. The mobile phone responds to the user's sliding screen operation and displays the corresponding video image on the screen.

[0138] The following describes in detail the processing flow of the electronic device from the user performing an operation on the screen to the screen displaying an image corresponding to the user operation.

[0139] Figure 3a is a schematic diagram illustrating the principles of an exemplary interface display. As shown in Figure 3a, when a user performs a touch operation on a mobile phone, the touchscreen captures the touch operation and generates a touch event. The touchscreen transmits the touch event to the input system, which processes the touch event, converts it into a touch event that the application layer can understand, and then transmits the touch event to the corresponding application.

[0140] When the application receives the VsyncApp signal, it draws the frame based on the received touch event. After the application completes the frame drawing, it sends the completed frame to SurfaceFlinger. The frame drawn by the application can be the application interface image frame.

[0141] It is understandable that during the frame drawing process of the application, the application can store the interface content in a buffer (cache), which is created by the application to facilitate processing and transmission of data. When the application sends the drawn frame to SurfaceFlinger, it can send the buffer to SurfaceFlinger.

[0142] When SurfaceFlinger receives the VsyncSF signal, it synthesizes the frames sent by the application and sends the synthesized frames to the display screen after completing the frame synthesis. The frames synthesized by SurfaceFlinger can be display image frames of the electronic device.

[0143] It is understandable that when SurfaceFlinger performs frame synthesis, it can perform frame synthesis based on one or more completed frames. Exemplarily, when SurfaceFlinger performs frame synthesis, it can synthesize the interface image frame, time view, power view or signal strength view, etc., drawn by the application, into a display image frame.

[0144] When the TE signal arrives, the display will display the synthesized frame on the mobile phone screen.

[0145] The following describes the interface display process in detail in conjunction with specific scenarios. Figure 3b is a schematic diagram of module interaction during the interface display process.

[0146] As shown in FIG3b , the interface display process provided in the embodiment of the present application may specifically include:

[0147] S101: The touch screen driver responds to user operations and sends an initial touch event to the input manager.

[0148] In an embodiment of the present application, the user operation may be an operation of the user sliding the screen.

[0149] The initial touch event may be a touch event generated by the touch screen driver according to a user operation.

[0150] After a user touches the screen, when the touch sensor in the electronic device receives the touch operation, the touchscreen driver processes the touch operation and generates an initial touch event. This initial touch event may include information such as the touch coordinates, touch force, and the touch operation timestamp. After generating the initial touch event, the touchscreen driver can send the initial touch event to the input manager via the kernel layer. The relevant processing flow for the touchscreen driver generating the initial touch event can be found in existing technologies and will not be further described here.

[0151] S102: The input manager processes the initial touch event to obtain a target touch event, and sends the target touch event to the application.

[0152] The target touch event can be a touch event that can be understood by the application layer.

[0153] The input manager processes the received initial touch event, converts it into a target touch event that the application layer can understand, and sends the target touch event to the application. For the relevant processing flow of the input manager for the initial touch event, please refer to the existing technology and will not be repeated here.

[0154] S103: After receiving the target touch event sent by the input manager, the application sends a VsyncAPP signal request to the image synthesis system. After receiving the VsyncAPP signal request, the image synthesis system sends a VsyncAPP signal to the application.

[0155] After receiving the target touch event, the application can send a VsyncAPP signal request to the image synthesis system. After receiving the VsyncAPP signal, the image synthesis system waits for the VsyncAPP signal and sends the VsyncAPP signal to the application when it receives the VsyncAPP signal.

[0156] For example, assuming that the application sends a VsyncAPP signal request to the image synthesis system at time a, the image synthesis system may send the first VsyncAPP signal received after time a to the application.

[0157] S104 , when the application receives the VsyncAPP signal, the application performs frame drawing according to the position information in the target touch event, and after the frame drawing is completed, sends the drawn frame to the image synthesis system.

[0158] When the application receives the VsyncAPP signal, it performs frame drawing based on the location information in the target touch event. Frame drawing can be understood as application interface drawing, frame image drawing, etc. In the embodiments of the present application, the above expressions have the same meaning, that is, the above expressions can all be expressed as drawing the interface image frame of the application.

[0159] Regarding the relevant processing flow of the application performing frame drawing, reference may be made to the existing technology and will not be repeated here.

[0160] S105 , when receiving the VsyncSF signal, the image synthesis system performs frame synthesis according to the drawn frame, and after the frame synthesis is completed, sends the synthesized frame to the display screen.

[0161] When the image synthesis system receives the VsyncSF signal, it performs frame synthesis based on the drawn frames. For the relevant processing flow of the frame synthesis performed by the image synthesis system, please refer to the existing technology and will not be described in detail here.

[0162] S106 : When the display screen receives the TE signal, it updates the screen according to the synthesized frame.

[0163] When the display screen receives the TE signal, it updates the screen according to the synthesized frame to display the interface on the display screen.

[0164] Continuing with Figure 3a, the time interval between the image compositing system receiving the VsyncSF signal and the display receiving the TE signal, from the time the user performs an operation on the screen until the corresponding image is displayed, is the SF WorkDuration. That is, the SF WorkDuration starts when the image compositing system receives the VsyncSF signal, and ends when the display receives the TE signal.

[0165] During the SF WorkDuration, the image synthesis system performs frame synthesis. Understandably, this process takes a certain amount of time. However, the duration of the frame synthesis process fluctuates due to factors such as CPU frequency and the amount of content being synthesized.

[0166] During the display interface process, the electronic device can set the SF WorkDuration according to the synthesis time requirement of SurfaceFlinger.

[0167] If the SF WorkDuration is set too short, the image synthesis system may not complete frame synthesis at the end of the SF WorkDuration. As a result, the synthesized frame cannot be displayed on the screen when the display receives the TE signal, resulting in frame loss. As a result, the phone cannot respond to user operations in a timely manner, and the screen image is stuck and not smooth.

[0168] If the SF WorkDuration is set long enough, the image synthesis system can complete frame synthesis before the SF WorkDuration ends, allowing the screen to display the synthesized frame normally when the display receives the TE signal. However, if the SF WorkDuration is set too long, there will be a long period of idle time after the image synthesis system completes frame synthesis, which may increase hand tracking latency, reduce the phone's hand tracking performance, and affect the user experience.

[0169] In addition, during the interface display process, each frame corresponds to an expected TE signal, and the expected TE signal is temporarily referred to as the expected TE point. That is, the electronic device will display the corresponding frame when the expected TE point arrives. However, if the frame synthesis is not completed when the expected TE point arrives, the frame cannot be displayed when the expected TE point arrives, and can only be displayed when the next TE signal of the expected TE point arrives. At this time, if the refresh frequency of the TE signal is low and the TE signal cycle is long, that is, the time interval between the expected TE point and the next TE signal is long, the screen will freeze.

[0170] In order to solve the above technical problems, an embodiment of the present application provides a display method. In this method, the electronic device can increase the refresh frequency of the TE signal, that is, set the TE signal of the electronic device to a high-frequency TE signal. In this way, in the scenario of high-frequency TE signals, the electronic device can set an adjustment period for SF WorkDuration and dynamically adjust SF WorkDuration at the end of each adjustment period. Thus, during the interface display process, the image synthesis system can perform frame synthesis according to the dynamically adjusted SF WorkDuration. In addition, in the scenario of high-frequency TE signals, the electronic device can also set a penalty period for SF WorkDuration, and the electronic device starts the penalty period timing when a frame is lost, and sets the SF WorkDuration to a default value during the penalty period.

[0171] In this way, during the interface display process, the electronic device can periodically and dynamically adjust the SF WorkDuration in high-frequency TE signal scenarios, and adjust the SF WorkDuration back to the default value in time when frame loss occurs, so as to reduce the tracking delay while avoiding continuous frame loss, thereby increasing the smoothness of the picture display and improving the user experience.

[0172] The following describes in detail the high-frequency TE signal scenario in the display method of the embodiment of the present application.

[0173] The "high frequency" in the high-frequency TE signal refers to the frequency of the TE signal exceeding the screen refresh rate. In other words, the frequency of the high-frequency TE signal exceeds the screen refresh rate. Normally, the frequency of the TE signal is consistent with the screen refresh rate.

[0174] Figure 4 is a schematic diagram illustrating the display of conventional TE signals and high-frequency TE signals. As shown in Figure 4 , the screen refresh rate is 120 Hz (hertz), the frequency of the conventional TE signal is also 120 Hz, and the frequency of the high-frequency TE signal can be 360 ​​Hz. During the display process, the expected TE points corresponding to frame 1 are the conventional TE signal 21 and the high-frequency TE signal 31, and the expected TE points corresponding to frame 2 are the conventional TE signal 22 and the high-frequency TE signal 34.

[0175] 4 , the process of synthesizing and displaying frame 1 by the electronic device specifically includes:

[0176] When SurfaceFlinger receives VsyncSF signal 11, it begins compositing frame 1 and completes the composition at time a. The display receives the composited frame at time a and begins displaying frame 1 when it receives regular TE signal 21 or high-frequency TE signal 31.

[0177] Since SurfaceFlinger completes the synthesis of frame 1 at time a, that is, the synthesis time of frame 1 is less than SF WorkDuration, frame 1 can be displayed when the expected TE point (conventional TE signal 21 or high-frequency TE signal 31) arrives.

[0178] 4 , the process of synthesizing and displaying frame 2 by the electronic device specifically includes:

[0179] When SurfaceFlinger receives the VsyncSF signal 12, it starts to synthesize frame 2 and completes the frame synthesis at time b. The display receives the synthesized frame at time b, but does not receive the synthesized frame when it receives the regular TE signal 22 or the high-frequency TE signal 34. That is, the display cannot display frame 2 when it receives the expected TE point (regular TE signal 22 or high-frequency TE signal 34). At this time, frame 2 can be called a late frame, that is, the arrival time of frame 2 is later than the arrival time of the expected TE point.

[0180] Since SurfaceFlinger completes the synthesis of frame 2 at time b, that is, the synthesis time of frame 2 is greater than SF WorkDuration, frame 2 needs to be displayed when the next TE signal (conventional TE signal 22 or high-frequency TE signal 34) of the expected TE point arrives.

[0181] After receiving the synthesized frame at time b, the display screen waits for the regular TE signal 23 or the high-frequency TE signal 35 and displays frame 2 upon receiving the regular TE signal 23 or the high-frequency TE signal 35. However, the period of the regular TE signal is longer than the period of the high-frequency TE signal, meaning that the regular TE signal 22 arrives later than the high-frequency TE signal 34. Therefore, in the high-frequency TE signal scenario, frame 2 can be displayed more quickly.

[0182] Therefore, in high-frequency TE signal scenarios, the display screen can display late frames more quickly, thereby responding to user operations more quickly, reducing the tracking delay during user operation of the screen, and improving the user experience.

[0183] In an embodiment of the present application, the process of the electronic device setting the TE signal to a high-frequency TE signal may include: the image synthesis system sending a high-frequency TE activation request to the display screen; after receiving the high-frequency TE activation request, the display screen sets the high-frequency TE and notifies the image synthesis system that the high-frequency TE is activated. Regarding the relevant processing flow for setting the high-frequency TE on the display screen, reference may be made to existing technologies and will not be repeated here.

[0184] The following is a detailed description of the specific process of adjusting the SF WorkDuration in the display method of the embodiment of the present application through a specific example. Figures 5 and 6 are schematic diagrams of the interaction of various functional modules.

[0185] In the embodiment of the present application, the SF WorkDuration is adjusted in the scenario of high-frequency TE signals. It should be noted that the adjustment of the SF WorkDuration in the embodiment of the present application is performed when the user slides the screen. If the user leaves the screen, the adjustment of the SF WorkDuration is stopped.

[0186] 5 and 6 , the process of adjusting SF WorkDuration in the embodiment of the present application specifically includes:

[0187] S201 : At the beginning of each adjustment cycle, the cycle adjustment module sets the target synthesis time to 0 and sends an instruction to enter the adjustment cycle to the image synthesis module.

[0188] During an adjustment cycle, the SF WorkDuration remains unchanged and the image synthesis system synthesizes frames based on the SF WorkDuration. After the adjustment cycle ends, the SF WorkDuration is adjusted and the next adjustment cycle begins. During the next adjustment cycle, the image synthesis system synthesizes frames based on the adjusted SF WorkDuration.

[0189] For example, the period length of the adjustment period can be set to 200 ms (milliseconds). It should be noted that the period length of the adjustment period can be configured according to demand, and the embodiment of the present application does not limit this.

[0190] It can be understood that the image synthesis system performs frame synthesis based on the SF WorkDuration. This can be done by the image synthesis module in the image synthesis system determining the triggering moment of the VsyncSF signal based on the SF WorkDuration and the expected TE point corresponding to the frame, and sending a VsyncSF signal request to the Vsync generation module in the image synthesis system. The Vsync generation module in the image synthesis system generates a VsyncSF signal at the triggering moment of the VsyncSF signal based on the VsyncSF signal request, and returns the VsyncSF signal to the image synthesis module in the image synthesis system. The image synthesis module in the image synthesis system performs frame synthesis upon receiving the VsyncSF signal. Accordingly, the image synthesis system performs the same processing flow for frame synthesis based on the adjusted SF WorkDuration.

[0191] The start time of the first adjustment cycle may be the synthesis end time of the first frame synthesis performed by the image synthesis system after the user touches the screen.

[0192] If the user does not leave the screen and does not enter the penalty period, the next adjustment period can be entered after the previous adjustment period ends. In this case, the start time of the next adjustment period can be the synthesis end time of the first frame after the previous adjustment period ends.

[0193] If the user does not leave the screen but enters the penalty period, then after the penalty period ends, the next adjustment cycle is performed. At this time, the start time of the next adjustment cycle can be the synthesis end time of the first frame after the penalty period ends.

[0194] The target synthesis time may be a target value of a synthesis time in the adjustment cycle. The instruction to enter the adjustment cycle may be an instruction to enter the adjustment cycle.

[0195] The cycle adjustment module may set the target synthesis time to 0 at the beginning of each adjustment cycle, and send an instruction to enter the adjustment cycle to the image synthesis module at the beginning of each adjustment cycle.

[0196] In an optional implementation, the target synthesis time may be set to a non-zero fixed value, which may be, for example, a synthesis time threshold.

[0197] S202. After receiving the instruction to enter the adjustment cycle, the image synthesis module determines the synthesis end time of the first current frame and the synthesis time of the first current frame when the synthesis of the first current frame is completed, and sends the synthesis end time of the first current frame and the synthesis time of the first current frame to the frame loss penalty module.

[0198] During the adjustment period, the frame drawing, frame synthesis, and screen update processes are not affected. That is, during the adjustment period, the application draws frames when it receives the VsyncAPP signal, the image synthesis system performs frame synthesis when it receives the VsyncSF signal, and the display screen updates when it receives the TE signal.

[0199] During the adjustment period, the image synthesis system may receive multiple VsyncSF signals, that is, may perform multiple frame synthesis processes. When each frame synthesis ends, the synthesis end time and synthesis time of each frame may be determined.

[0200] The first current frame may be any frame within the adjustment period. The synthesis time is the time taken for the frame synthesis process. It is understood that the synthesis time taken for different frames may be different.

[0201] During the adjustment period, when the synthesis of each first current frame is completed, the image synthesis module can determine the synthesis end time of each first current frame and the synthesis time of each first current frame, and send the synthesis end time of each first current frame and the synthesis time of each first current frame to the frame loss penalty module.

[0202] S203 , the frame loss penalty module determines whether the synthesis time of the first current frame is less than the current SF WorkDuration; if so, execute S204 ; if not, execute S211 .

[0203] The current SF WorkDuration may be the SF WorkDuration within the current adjustment period, that is, within the adjustment period, frame synthesis is performed according to the current SF WorkDuration.

[0204] It should be noted that in the first adjustment cycle, the current SF WorkDuration is the default value. In each adjustment cycle after the first adjustment cycle, the current SF WorkDuration in each adjustment cycle is the SF WorkDuration adjusted at the end of the previous adjustment cycle.

[0205] During the adjustment period, after receiving the synthesis duration of each first current frame, the frame loss penalty module determines whether the synthesis duration of each first current frame is greater than the current SF WorkDuration.

[0206] If the synthesis duration of each first current frame is less than the current SF WorkDuration, the synthesis duration and synthesis end time of each first current frame may be sent to the period adjustment module.

[0207] Referring to Figure 7a, during the adjustment period (i.e., after the start of the adjustment period), the image synthesis system synthesizes frame 1 upon receiving the VsyncSF signal 11, and the display screen displays frame 1 upon receiving the high-frequency TE signal 21. The time interval between the VsyncSF signal 11 and the high-frequency TE signal 21 is SF WorkDuration_1.

[0208] As shown in Figure 7a, the synthesis time of frame 1 is less than SF WorkDuration_1. Therefore, after the synthesis of frame 1 is completed, the adjustment period does not change, and the SF WorkDuration does not change either.

[0209] 7a, the image synthesis system synthesizes frame 2 upon receiving the VsyncSF signal 12, and the display screen displays frame 2 upon receiving the high-frequency TE signal 24. The time interval between the VsyncSF signal 12 and the high-frequency TE signal 24 is SF WorkDuration_1.

[0210] If the synthesis time of a first current frame is greater than the current SF WorkDuration, it means that the first current frame cannot be displayed at the expected TE point, that is, the first current frame will be lost. In this case, the current SF WorkDuration can be updated to the default SF WorkDuration so that the image synthesis system performs frame synthesis according to the default SF WorkDuration, thereby preventing continuous frame loss.

[0211] Referring to Figure 7b , during the adjustment period (i.e., after the start of the adjustment period), the image synthesis system synthesizes frame h upon receiving VsyncSF signal 1a , and the display screen displays frame h upon receiving high-frequency TE signal 2b+4 . The time interval between VsyncSF signal 1a and high-frequency TE signal 2b+4 is greater than SF WorkDuration_1 .

[0212] As shown in FIG7b , the time interval between the VsyncSF signal 1a and the high-frequency TE signal 2b+3 is SF WorkDuration_1, that is, the synthesis time of frame h is greater than SF WorkDuration_1. At this time, SF WorkDuration_1 (current SF WorkDuration) can be updated to SF WorkDuration_2 (default SF WorkDuration).

[0213] Continuing with FIG7b , during the penalty period (i.e., after the penalty period begins), the image synthesis system synthesizes frame h+1 upon receiving VsyncSF signal 1a+2, and the display screen displays frame h+1 upon receiving high-frequency TE signal 2b+10. The time interval between VsyncSF signal 1a+2 and high-frequency TE signal 2b+10 is SF WorkDuration_2.

[0214] S204: The frame loss penalty module sends the synthesis end time of the first current frame and the synthesis time of the first current frame to the period adjustment module.

[0215] When the synthesis time of the first current frame is less than the current SF WorkDuration, it can be confirmed that the first current frame can be displayed at the expected TE point and no frame loss will occur. The frame loss penalty module can send the synthesis end time of the first current frame and the synthesis time of the first current frame to the period adjustment module.

[0216] During the adjustment period, the frame loss penalty module may send the composition end time and composition duration of each first current frame whose composition duration is less than the current SF WorkDuration to the period adjustment module.

[0217] S205: The period adjustment module updates the target synthesis time according to the synthesis time of the first current frame.

[0218] After receiving the synthesis duration of the first current frame, the period adjustment module may update the target synthesis duration according to the synthesis duration of each first current frame within the adjustment period.

[0219] When the synthesis time of the first current frame is greater than the target synthesis time, the target synthesis time is updated to the synthesis time of the first current frame. When the synthesis time of the first current frame is less than the target synthesis time, the target synthesis time is kept unchanged.

[0220] In an optional embodiment, when the target synthesis time is a fixed value other than 0, updating the target synthesis time according to the synthesis time of the first current frame may include: judging the size of the synthesis time of each first current frame and the target synthesis time, and determining any one frame in the first current frame whose synthesis time is greater than the target synthesis time, and determining the synthesis time of the frame as the target synthesis time.

[0221] In another optional embodiment, when the target synthesis time is a fixed value other than 0, updating the target synthesis time according to the synthesis time of the first current frame may also include: judging the size of the synthesis time of each first current frame and the target synthesis time, and determining the average of the synthesis time of each first current frame whose synthesis time is greater than the target synthesis time as the target synthesis time.

[0222] S206 , the period adjustment module determines whether the synthesis end time of the first current frame exceeds the end time of the adjustment period; if so, execute S207 ; if not, return to execute S206 .

[0223] The end time of the adjustment period can be determined according to the period length of the adjustment period and the start time of the adjustment period.

[0224] After updating the target synthesis time, the cycle adjustment module may determine whether the synthesis end time of the first current frame exceeds the end time of the adjustment cycle.

[0225] During the adjustment period, if the synthesis end time of each first current frame does not exceed the end time of the adjustment period, it indicates that the adjustment period has not ended. The image synthesis system can then wait for the next second current frame to be synthesized until the synthesis end time of the first current frame received by the period adjustment module exceeds the end time of the adjustment period. It is understood that during the adjustment period, the current SF WorkDuration remains unchanged.

[0226] Referring to Figure 7c, during the adjustment period (i.e., after the start of the adjustment period), the image synthesis system synthesizes frame i upon receiving the VsyncSF signal 1c, and the display screen displays frame i upon receiving the high-frequency TE signal 2d. The time interval between the VsyncSF signal 1c and the high-frequency TE signal 2d is SF WorkDuration_1.

[0227] As shown in FIG7c, the synthesis end time of frame i does not exceed the end time of the adjustment period, so the adjustment period has not ended. Before the adjustment period ends, SF WorkDuration_1 remains unchanged.

[0228] Continuing with Figure 7c, the image synthesis system synthesizes frame i+1 upon receiving the VsyncSF signal 1c+1, and the display screen displays frame i+1 upon receiving the high-frequency TE signal 2d+3. The time interval between the VsyncSF signal 1c+1 and the high-frequency TE signal 2d+3 is SF WorkDuration_1.

[0229] During the adjustment period, if the synthesis end time of a first current frame exceeds the end time of the adjustment period, it means that the adjustment period ends before the synthesis of the first current frame ends, or ends at the synthesis end time of the first current frame. Then, the period adjustment module can update the current SF WorkDuration, that is, the image synthesis system can perform frame synthesis according to the updated current SF WorkDuration.

[0230] Referring to Figure 7d, during the adjustment period (i.e., after the start of the adjustment period), the image synthesis system synthesizes frame j upon receiving the VsyncSF signal 1e, and the display screen displays frame j upon receiving the high-frequency TE signal 2f. The time interval between the VsyncSF signal 1e and the high-frequency TE signal 2f is SF WorkDuration_1.

[0231] As shown in Figure 7d, the synthesis end time of frame j exceeds the end time of the adjustment period, that is, the adjustment period has ended when the image synthesis system synthesizes frame j+1. At the end of the adjustment period, SF WorkDuration_1 is updated to SF WorkDuration_3.

[0232] Continuing with FIG. 7D , after the adjustment period ends (i.e., after the adjustment period ends), the image synthesis system synthesizes frame j+1 upon receiving the VsyncSF signal 1e+1, and the display screen displays frame j+1 upon receiving the high-frequency TE signal 2f+3. The time interval between the VsyncSF signal 1e+1 and the high-frequency TE signal 2f+3 is SF WorkDuration_3.

[0233] S207 : The cycle adjustment module determines the target SF WorkDuration according to the target synthesis time and the design time margin.

[0234] The time consumption design margin may be a preset spare time consumption value, that is, the time consumption value of waiting for the expected TE point after the frame synthesis is completed.

[0235] When the synthesis end time of the first current frame exceeds the end time of the adjustment period, the period adjustment module may determine the target SF WorkDuration according to the target synthesis time and the design time margin.

[0236] In the embodiment of the present application, the target SF WorkDuration may be equal to the sum of the target synthesis time and the time design margin.

[0237] S208 , the cycle adjustment module determines whether the target SF WorkDuration is less than a preset minimum threshold; if so, execute S209 ; if not, execute S210 .

[0238] The preset minimum threshold may be a preset minimum SF WorkDuration value.

[0239] After determining the target SF WorkDuration, the cycle adjustment module determines whether the target SF WorkDuration is less than a preset minimum threshold.

[0240] If the target SF WorkDuration is less than the preset minimum threshold, the current SF WorkDuration may be updated according to the preset minimum threshold, that is, the current SF WorkDuration may be updated to the preset minimum threshold.

[0241] If the target SF WorkDuration is greater than the preset minimum threshold, the current SF WorkDuration may be updated according to the target SF WorkDuration, that is, the current SF WorkDuration may be updated to the target SF WorkDuration.

[0242] S209 : The period adjustment module updates the current SF WorkDuration according to a preset minimum threshold, and sends the updated current SF WorkDuration to the image synthesis module.

[0243] 7 d , after the adjustment period ends, SF WorkDuration_1 (ie, the current SF WorkDuration) is updated to SF WorkDuration_3 (ie, the preset minimum threshold).

[0244] After the image synthesis module receives the updated current SF WorkDuration, the image synthesis system may perform frame synthesis according to the updated current SF WorkDuration.

[0245] S210: The cycle adjustment module updates the current SF WorkDuration according to the target SF WorkDuration, and sends the updated current SF WorkDuration to the image synthesis module.

[0246] Continuing with FIG. 7 d , after the adjustment period ends, SF WorkDuration_1 (ie, current SF WorkDuration) is updated to SF WorkDuration_3 (ie, target SF WorkDuration).

[0247] S211: The frame loss penalty module updates the current SF WorkDuration according to the default SF WorkDuration and executes S301. After updating the current SF WorkDuration according to the default SF WorkDuration, the frame loss penalty module sends the updated current SF WorkDuration to the image synthesis module.

[0248] The default SF WorkDuration may be a default value of a preset SF WorkDuration.

[0249] When the composition time of the first current frame is greater than the current SF WorkDuration, it means that the first current frame cannot be displayed at the expected TE point, that is, the first current frame will be dropped. In this case, the current SF WorkDuration can be updated to the default SF WorkDuration, and the composition end time of the first current frame is determined as the start time of the penalty period.

[0250] 7b , if the synthesis time of frame h is greater than SF WorkDuration_1, after the synthesis of frame h is completed, SF WorkDuration_1 (i.e., the current SF WorkDuration) is updated to SF WorkDuration_2 (i.e., the default SF WorkDuration), and a penalty period is entered. The synthesis end time of frame h is determined as the start time of the penalty period.

[0251] S301: The frame loss penalty module determines the synthesis end time of the first current frame as the start time of the penalty period, and sends an instruction to enter the penalty period to the image synthesis module.

[0252] The instruction to enter a penalty period may be an instruction to enter a penalty period.

[0253] If frame loss occurs during the adjustment period, the adjustment period ends and the penalty period begins. During the penalty period, the default SF WorkDuration remains unchanged. After the penalty period ends, the next adjustment period can begin.

[0254] If you do not enter the penalty period during an adjustment cycle, you will enter the next adjustment cycle after the adjustment cycle ends. If you enter the penalty period during an adjustment cycle, the adjustment cycle ends and you will not enter the next adjustment cycle until the penalty period ends.

[0255] For example, the duration of the penalty period can be set to 1 second. It should be noted that the duration of the penalty period can be configured according to needs, and the embodiment of the present application does not limit this.

[0256] S302: When the synthesis of the second current frame is completed, the image synthesis module determines the synthesis end time of the second current frame and sends the synthesis end time of the second current frame to the frame loss penalty module.

[0257] During the penalty period, the frame drawing, frame synthesis, and screen update processes are not affected. That is, during the penalty period, the application draws frames when it receives the VsyncAPP signal, the image synthesis system synthesizes frames when it receives the VsyncSF signal, and the display screen updates when it receives the TE signal.

[0258] During the penalty period, the image synthesis system may receive multiple VsyncSF signals, that is, may perform multiple frame synthesis processes. When each frame synthesis ends, the synthesis end time of each frame may be determined.

[0259] The second current frame may be any frame within the penalty period.

[0260] During the penalty period, when synthesis of each second current frame is completed, the image synthesis module may determine the synthesis end time of each second current frame and send the synthesis end time of each second current frame to the frame loss penalty module.

[0261] S303, the frame loss penalty module determines whether the synthesis end time of the second current frame exceeds the end time of the penalty period; if so, execute S304; if not, return to execute S303.

[0262] The end time of the penalty period can be determined based on the length of the penalty period and the start time of the penalty period.

[0263] When receiving the synthesis end time of the second current frame, the frame loss penalty module may determine whether the synthesis end time of the second current frame exceeds the end time of the penalty period.

[0264] During the penalty period, if the synthesis end time of each second current frame does not exceed the end time of the penalty period, indicating that the penalty period has not ended, the image synthesis system can wait for the next second current frame to be synthesized until the synthesis end time of the second current frame received by the frame loss penalty module exceeds the end time of the penalty period. It is understood that during the penalty period, the default SF WorkDuration remains unchanged.

[0265] Referring to Figure 7e, during the penalty period (i.e., after the penalty period begins), the image synthesis system synthesizes frame k upon receiving the VsyncSF signal 1m, and the display screen displays frame k upon receiving the high-frequency TE signal 2n. The time interval between the VsyncSF signal 1m and the high-frequency TE signal 2n is SFWorkDuration_2.

[0266] As shown in Figure 7e, the synthesis end time of frame k does not exceed the end time of the penalty period, so the penalty period has not ended. Before the penalty period ends, SF WorkDuration_2 remains unchanged.

[0267] Continuing with FIG7e , the image synthesis system synthesizes frame k+1 upon receiving the VsyncSF signal 1m+1, and the display screen displays frame k+1 upon receiving the high-frequency TE signal 2n+3. The time interval between the VsyncSF signal 1m+1 and the high-frequency TE signal 2n+3 is SF WorkDuration_2.

[0268] During the penalty period, if the synthesis end time of a second current frame exceeds the end time of the penalty period, it means that the penalty period ends before the synthesis of the second current frame ends, or ends at the synthesis end time of the second current frame, then the frame loss penalty module can send the synthesis end time of the second current frame to the period adjustment module.

[0269] Referring to Figure 7f, during the penalty period (i.e., after the penalty period begins), the image synthesis system synthesizes frame r upon receiving the VsyncSF signal 1p, and the display screen displays frame r upon receiving the high-frequency TE signal 2q. The time interval between the VsyncSF signal 1p and the high-frequency TE signal 2q is SFWorkDuration_2.

[0270] As shown in Figure 7f, the synthesis end time of frame r exceeds the end time of the penalty period. This means that the penalty period has already ended when the image synthesis system synthesizes frame r+1. After the penalty period ends, the synthesis end time of frame r is determined as the start time of the next adjustment cycle, thus entering the next adjustment cycle. During the next adjustment cycle, SF WorkDuration_2 remains unchanged.

[0271] Continuing with FIG7f , the image synthesis system synthesizes frame r+1 upon receiving the VsyncSF signal 1p+1, and the display screen displays frame r+1 upon receiving the high-frequency TE signal 2q+3. The time interval between the VsyncSF signal 1p+1 and the high-frequency TE signal 2q+3 is SF WorkDuration_2.

[0272] S304: The frame loss penalty module sends the synthesis end time of the second current frame to the period adjustment module, and the period adjustment module determines the synthesis end time of the second current frame as the start time of the adjustment period.

[0273] When the frame loss penalty module determines that the synthesis end time of the second current frame exceeds the end time of the penalty period, the synthesis end time of the second current frame is sent to the cycle adjustment module. At this time, the adjustment cycle is entered, and the cycle adjustment module determines the synthesis end time of the second current frame as the start time of the adjustment cycle of the next adjustment cycle.

[0274] When the cycle adjustment module determines the synthesis end time of the second current frame as the start time of the adjustment cycle, the next adjustment cycle is entered, and the cycle adjustment module returns to execute S201.

[0275] It is understandable that after the penalty period ends and the next adjustment period begins, the current SF WorkDuration value is the default SF WorkDuration value during the adjustment period.

[0276] The following is a detailed description of the adjustment period and penalty period in the display method according to an embodiment of the present application through a specific example. Figures 8a and 8b are schematic timing diagrams showing exemplary adjustment periods and penalty periods.

[0277] Referring to Figure 8a, the electronic device may enter the i-th adjustment cycle at the synthesis end time of frame s (i.e., time t1) and end the i-th adjustment cycle at time t2. The image synthesis system may adjust the SF WorkDuration at the synthesis end time of frame s (i.e., time t1). During the i-th adjustment cycle, the image synthesis system may synthesize each frame according to the adjusted SF WorkDuration, and the synthesis time of each frame is less than the SF WorkDuration corresponding to the i-th adjustment cycle.

[0278] Since the synthesis end time of frame s+n (i.e., time t3) exceeds the end time of the i-th adjustment cycle (i.e., time t2), the electronic device can enter the next adjustment cycle (i.e., the i+1-th adjustment cycle) at the synthesis end time of frame s+n (i.e., time t3) and end the i+1-th adjustment cycle at time t4. The image synthesis system can adjust the SF WorkDuration at time t3. Similarly, within the i+1-th adjustment cycle, the image synthesis system can synthesize each frame according to the adjusted SF WorkDuration, and the synthesis time of each frame is less than the SF WorkDuration corresponding to the i+1-th adjustment cycle.

[0279] The electronic device can enter the (i+2)th adjustment cycle at the completion time (i.e., time t5) of the synthesis of the first frame (i.e., frame s+m) after time t4. The image synthesis system can adjust the SF WorkDuration at time t5. As shown in Figure 8a, in this embodiment of the present application, the duration of each adjustment cycle of the electronic device can be 200ms.

[0280] Referring to Figure 8b , during the i-th adjustment cycle, when the image synthesis system synthesizes frame s, the synthesis time of frame s is greater than the SF WorkDuration corresponding to the i-th adjustment cycle. Thus, the electronic device can end the i-th adjustment cycle at the end time of synthesis of frame s (i.e., time t1). In other words, the electronic device can end the i-th adjustment cycle before the end time of the i-th adjustment cycle. Therefore, the duration of the i-th adjustment cycle is less than 200ms.

[0281] The electronic device may enter a penalty period at the end of the composition of frame s+n (i.e., time t2), and the penalty period may end at time t3. The image composition system may adjust the SF WorkDuration at time t2. During the penalty period, the image composition system may combine each frame according to the adjusted SF WorkDuration.

[0282] Since the synthesis end time of frame s+m (i.e., time t4) exceeds the end time of the penalty period (i.e., time t3), the electronic device can enter the jth adjustment cycle at the synthesis end time of frame s+m (i.e., time t4). The SF WorkDuration corresponding to the jth adjustment cycle can be the same as the SF WorkDuration corresponding to the penalty period. In other words, the image synthesis system does not need to adjust the SF WorkDuration at time t4.

[0283] Because the synthesis end time of frame s+p (i.e., time t6) exceeds the end time of the jth adjustment cycle (i.e., time t5), the electronic device can enter the j+1th adjustment cycle at time t6 and end the j+1th adjustment cycle at time t7. The image synthesis system can adjust the SF WorkDuration at time t6.

[0284] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the display method in the above-mentioned embodiment.

[0285] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the display method in the above-mentioned embodiment.

[0286] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the display method in the above-mentioned method embodiments.

[0287] Among them, the electronic devices (such as mobile phones, etc.), computer storage media, computer program products or chips provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0288] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0289] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0290] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display method, characterized in that, The method includes: In response to the movement of a target operation, a target application draws at least one interface image; Determine the target duration of a second adjustment period according to the synthesis time consumption of at least one display frame within a first adjustment period; wherein, the second adjustment period is the next adjustment period adjacent to the first adjustment period, and the target duration is the time interval between the display trigger moment and the synthesis trigger moment of the display frame; Within the second adjustment period, obtain a first interface image drawn by the target application, and determine the arrival moment of the expected tearing effect TE signal of a first display frame corresponding to the first interface image; Determine the frame synthesis trigger moment corresponding to the first interface image according to the expected arrival moment of the TE signal of the first display frame and the target duration of the second adjustment period; At the frame synthesis trigger moment corresponding to the first interface image, synthesize the first display frame according to the first interface image; Display a plurality of display frames corresponding to the at least one interface image, and the plurality of display frames include the first display frame.

2. The method according to claim 1, wherein The determining the target duration of the second adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period includes: Within the first adjustment period, determine the target synthesis time consumption corresponding to the first adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period; After the first adjustment period ends, obtain a preset idle time consumption, and determine the sum value of the target synthesis time consumption and the preset idle time consumption as the duration to be adjusted; Determine the target duration of the second adjustment period according to the duration to be adjusted.

3. The method according to claim 2, wherein The determining the target synthesis time consumption corresponding to the first adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period includes: At the start moment of the first adjustment period, set the target synthesis time consumption to an initial value; Sequentially take each display frame synthesized within the first adjustment period as the current display frame; Obtain the synthesis time consumption of the current display frame, and when the synthesis time consumption of the current display frame is greater than the target synthesis time consumption, update the value of the target synthesis time consumption to the synthesis time consumption of the current display frame.

4. The method according to claim 2, wherein The determining the target duration of the second adjustment period according to the duration to be adjusted includes: Obtain a first duration; When the duration to be adjusted is greater than the first duration, determine the duration to be adjusted as the target duration of the second adjustment period; When the duration to be adjusted is less than the first duration, determine the first duration as the target duration of the second adjustment period.

5. The method according to claim 1, characterized in that, Before determining the target duration of the second adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period, it further includes: Determine the target duration of the first adjustment period; The determining the target duration of the second adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period includes: When the synthesis time consumption of all display frames synthesized within the first adjustment period is less than the target duration of the first adjustment period, determine the target duration of the second adjustment period according to the synthesis time consumption of at least one display frame within the first adjustment period.

6. The method according to claim 5, wherein The method further includes: During the first adjustment period, determining the composition time of the second display frame; When the composition time of the second display frame is greater than the target duration of the first adjustment period, determining the target duration corresponding to the penalty period and starting the penalty period timing; During the penalty period, obtaining the third interface image drawn by the target application and determining the expected arrival time of the TE signal of the third display frame corresponding to the third interface image; According to the expected arrival time of the TE signal of the third display frame and the target duration corresponding to the penalty period, determining the frame composition trigger time corresponding to the third interface image; At the frame composition trigger time corresponding to the third interface image, synthesizing the third display frame according to the third interface image; After the penalty period timing ends, determining the next adjustment period; wherein the target duration of the next adjustment period is the second duration.

7. The method according to claim 6, wherein The determining the target duration corresponding to the penalty period includes: Determining the second duration as the target duration corresponding to the penalty period.

8. The method according to claim 6, wherein The method further includes: The start time of the penalty period is the end time of the composition of the second display frame.

9. The method according to claim 6, wherein The method further includes: Determining the end time of the composition of the third display frame and the end time of the penalty period; When the end time of the composition of the third display frame exceeds the end time of the penalty period, determining the end time of the composition of the third display frame as the start time of the next adjustment period.

10. The method according to claim 5, wherein The determining the target duration of the first adjustment period includes: When the first adjustment period is the first adjustment period, obtaining the second duration; Determining the second duration as the target duration of the first adjustment period.

11. The method according to claim 1, wherein The method further includes: Determining the end time of the composition of the fourth display frame within the first adjustment period and the end time of the first adjustment period; When the end time of the composition of the fourth display frame exceeds the end time of the first adjustment period, determining the end time of the composition of the fourth display frame as the start time of the second adjustment period.

12. The method according to any one of claims 1-11, characterized in that, The method is applied to an electronic device, and the frequency of the TE signal of the electronic device is greater than the screen refresh frequency of the electronic device.

13. An electronic device, characterized in that, It includes: One or more processors; A memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, the electronic device is caused to execute the display method according to any one of claims 1-12.

14. A computer-readable storage medium, comprising a computer program, characterized in that, When the computer program runs on the electronic device, the electronic device is caused to execute the display method according to any one of claims 1-12.

Images