Vsync signal control method, electronic equipment, storage medium and chip

Abstract

The embodiment of the invention relates to a Vsync signal control method, electronic equipment, a storage medium and a chip. The method comprises the steps that a synthesis system receives a Vsync signal request message sent by an application main thread; after receiving the Vsync signal request message, the synthesis system determines the number of rendering times of a rendering thread of the application in unit time; under the condition that the number of rendering times of the rendering thread in unit time is equal to the system refresh rate of the electronic equipment, the Vsync thread sends a Vsync signal to the application main thread at a first frequency, and the first frequency is equal to the system refresh rate; and under the condition that the number of rendering times of the rendering thread in unit time is smaller than the system refresh rate of the electronic equipment, the Vsync thread sends a Vsync signal to the application main thread at a second frequency, and the second frequency is smaller than the system refresh rate. According to the invention, the sending frequency of the Vsync signal can be controlled and adjusted, so that the purpose of adjusting the refresh rate of the application is achieved, unnecessary power consumption waste is reduced, and the performance of the whole machine is improved.

Description

Vsync signal control method, electronic device, storage medium and chip

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 30, 2022, with application number 202211211815.5 and invention name “Vsync signal control method, electronic device, storage medium and chip”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] Embodiments of the present disclosure relate to the field of terminal display technology, and in particular to a Vsync signal control method, electronic device, computer-readable storage medium, and chip. Background Art

[0003] With the rapid development of display technology, the refresh rate supported by electronic devices is also increasing. The refresh rate refers to the number of times the screen is refreshed per second. Currently, to ensure smooth display of electronic devices, the refresh rate of electronic devices is usually set to a fixed value and maintained at a high refresh rate.

[0004] However, in different scenarios, the requirements for refresh rate are also different. If a high refresh rate is used for display, it will lead to unnecessary waste of resources and affect other performance of the entire machine.

[0005] Therefore, we look forward to a solution that can adjust the application refresh rate for different scenarios.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a Vsync signal control method, device, electronic device, and computer-readable storage medium. The main purpose of the present disclosure is to control the application refresh rate by adjusting the transmission frequency of the Vsync signal for different application scenarios, thereby preventing resource waste and improving overall device performance.

[0008] To achieve the above-mentioned objectives, in a first aspect, an embodiment of the present disclosure provides a method for controlling a Vsync signal, which is applied to an electronic device, including: a synthesis system receives a Vsync signal request message sent by an application main thread; after receiving the Vsync signal request message, the synthesis system determines the number of rendering times of the application's rendering thread per unit time; when the number of rendering times of the rendering thread per unit time is equal to the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a first frequency, and the first frequency is equal to the system refresh rate; when the number of rendering times of the rendering thread per unit time is less than the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a second frequency, and the second frequency is less than the system refresh rate.

[0009] In this way, after receiving the request message from the main thread of the application, the synthesis system compares the number of rendering times of the rendering thread in unit time and the system refresh rate. When the number of rendering times of the rendering thread in unit time is less than the system refresh rate, the frequency of sending the Vsync signal by the Vsync thread is reduced to adjust the refresh rate of the main thread of the application, reduce unnecessary application power consumption, prevent resource waste, and improve the performance of the entire machine.

[0010] Optionally, the Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at the second frequency; at least partially, the Vsync signal is sent when the system refresh rate determines that the Vsync signal needs to be sent. In this way, the Vsync thread does not send a Vsync signal when the system refresh rate determines that the Vsync signal needs to be sent, so as to achieve the switching from the first frequency to the second frequency.

[0011] Optionally, if the number of renderings performed by the rendering thread per unit time is less than 1 / n of the system refresh rate, the second frequency can be set to 1 / n of the system refresh rate, where n is a positive integer. Thus, if the number of renderings performed by the rendering thread per unit time is less than the system refresh rate, the rendering speed and the system refresh rate are mismatched, and the main application thread does not need to refresh at a high frequency. Therefore, the second frequency can be set to 1 / n of the system refresh rate, saving high-frequency power consumption of the main thread.

[0012] Optionally, after receiving the Vsync signal sent at the first frequency or the second sending frequency, the application main line measures and layouts the image frame; when the measurement and layout results meet the preset drawing conditions, the application main thread triggers the rendering thread to draw and render the image frame; when the measurement and layout results do not meet the preset drawing conditions, the application main thread does not trigger the rendering thread.

[0013] In a second aspect, an embodiment of the present disclosure provides a method for controlling a Vsync signal, which is applied to an electronic device, including a synthesis system receiving a Vsync signal request message sent by an application main thread; after receiving the Vsync signal request message, the synthesis system determines the number of requests for the application main thread to send the Vsync signal request message within a unit time; when the number of requests for the application main thread to send the Vsync signal request message within a unit time is equal to the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at a third frequency, and the third frequency is equal to the system refresh rate; when the number of requests for the application main thread to send the Vsync signal request message within a unit time is less than the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at a fourth frequency, and the fourth frequency is less than the system refresh rate.

[0014] In this way, after receiving the Vsync signal request message, the synthesis system compares the number of requests for the application main thread to send the Vsync signal request message per unit time with the system refresh rate. If the number of requests for the application main thread to send the Vsync signal request message per unit time is less than the system refresh rate, the frequency of the Vsync thread sending the Vsync signal is adjusted to a fourth frequency that is less than the system refresh rate, thereby adjusting the refresh rate of the application main thread and reducing the application power consumption.

[0015] Optionally, the Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at a fourth frequency; at least partially, the Vsync signal needs to be sent based on the system refresh rate.

[0016] Optionally, when the number of times the application main thread sends a Vsync signal request message within a unit time is less than 1 / m of the system refresh rate, the fourth frequency is 1 / m of the system refresh rate, where m is a positive integer.

[0017] In a third aspect, an embodiment of the present disclosure provides a method for controlling a Vsync signal, which is applied to an electronic device and includes: a synthesis system receiving a Vsync signal request message sent by an application main thread; after receiving the Vsync signal request message, the synthesis system determines the layer parameters of the image frame drawn by the application main thread; when the layer parameters meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a fifth frequency, and the fifth frequency is equal to the system refresh rate; when the layer parameters do not meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency, and the sixth frequency is less than the system refresh rate.

[0018] In this way, when it is determined that the layer parameters of the image frame drawn by the main thread of the application meet the preset conditions, the Vsync thread sends the Vsync signal according to the frequency of the system refresh rate. Otherwise, the frequency of the Vsync signal sent by the Vsync thread is adjusted to the fifth frequency which is less than the system refresh rate. The application refresh rate is adjusted by the sending frequency of the Vsync signal, which saves resources and improves the performance of the entire machine.

[0019] Optionally, the Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency; at least partially, the Vsync signal needs to be sent based on the system refresh rate.

[0020] Optionally, the layer parameters of the image frame include at least one of a layer area, a layer transparency, a zoom factor, and an application layer identifier.

[0021] Optionally, when the layer parameter of the image frame is the layer area, if the layer area of ​​the image frame is less than a first preset value, the layer parameter does not meet the preset conditions; if the layer area of ​​the image frame is greater than or equal to the first preset value, the layer parameter meets the preset conditions.

[0022] Optionally, when the layer parameter of the image frame is layer transparency, when the layer transparency of the image frame is greater than a second preset value, the layer parameter does not meet the preset conditions; when the layer transparency of the image frame is less than or equal to the second preset value, the layer parameter meets the preset conditions.

[0023] Optionally, when the layer parameter of the image frame is a zoom coefficient, when the zoom coefficient of the image frame is greater than a third preset value, the layer parameter does not meet the preset conditions; when the zoom coefficient of the image frame is less than or equal to the third preset value, the layer parameter meets the preset conditions.

[0024] Optionally, when the layer parameter of the image frame is an application layer identifier, if the application layer identifier of the image frame belongs to the preset range of the perceptible application layer list, the layer parameter meets the preset conditions; if the application layer identifier of the image frame does not belong to the preset range of the perceptible application layer list, the layer parameter does not meet the preset conditions.

[0025] In this way, the preset conditions can be judged based on the layer parameters of the image frame drawn by the main thread of the application. For some layers with small area, high transparency, large scaling factor and application layer identifiers that do not belong to the preset perceptible application layer list, users are not sensitive to the application refresh rate of the above layers (also known as, the user's perceptual experience is weak), but the applications of these layers will still continuously send Vsync signal request messages at a higher frequency. For such layers, the frequency of the Vsync thread sending Vsync signals can be adjusted to save resources and ensure stable performance of the main application.

[0026] The first to third aspects of the present disclosure may be used individually or in combination with each other, that is, at least two methods in the first, second and third aspects may be combined to control the Vsync signal.

[0027] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, which includes at least one processor; and a memory and a display screen that are communicatively connected to the at least one processor; wherein the display screen is used to display an application interface; the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned Vsync signal control method.

[0028] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When a computer instruction runs on a terminal device, the computer program is executed by a processor to implement the above-mentioned Vsync signal control method.

[0029] In a sixth aspect, an embodiment of the present disclosure provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the above-mentioned Vsync signal control method.

[0030] It should be understood that the fourth to sixth aspects of the present disclosure correspond to the technical solutions of the first to third aspects of the present disclosure respectively, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a schematic diagram of the hardware system structure of a terminal device provided by an embodiment of the present disclosure;

[0032] FIG2 is a schematic diagram of the structure of a terminal device software system according to an embodiment of the present disclosure;

[0033] FIG3 is a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure;

[0034] FIG4 is a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure;

[0035] FIG5 is a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure;

[0036] FIG6-1 is a detailed flowchart 1 of a method for controlling a Vsync signal according to an embodiment of the present disclosure;

[0037] FIG6-2 is a second detailed flowchart of a method for controlling a Vsync signal according to an embodiment of the present disclosure;

[0038] FIG7 is a flow chart of a method for determining request parameters according to an embodiment of the present disclosure;

[0039] FIG8 is a schematic diagram of a process of switching the frequency of sending a Vsync signal by a Vsync thread according to an embodiment of the present disclosure;

[0040] FIG9 is a schematic diagram of the hardware structure of a data processing device provided in one embodiment of the present disclosure.

[0041] The realization of the objectives, functional features and advantages of the present disclosure will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0043] When the present disclosure refers to ordinal numbers such as "first" or "second", unless the context indicates an order, it should be understood that they are merely used for distinction. The terms "exemplary" or "for example" are used to indicate an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other implementations or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete way.

[0044] In the embodiments of the present disclosure, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0045] The Vsync signal control method provided by the embodiments of the present disclosure can be applied to electronic devices with display functions.

[0046] An electronic device may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. An electronic device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the electronic device.

[0047] In order to better understand the embodiments of the present disclosure, the structure of the electronic device according to the embodiments of the present disclosure is introduced below:

[0048] Taking the electronic device as a terminal device as an example, FIG1 shows a schematic structural diagram of the terminal device 100. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0049] It should be understood that the structures illustrated in the embodiments of the present disclosure do not constitute a specific limitation on the terminal device 100. In other embodiments of the present disclosure, the terminal device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0050] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0051] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0052] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0053] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0054] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing touch functionality in the terminal device 100, such as a mobile phone.

[0055] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0056] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0057] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0058] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the terminal device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0059] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0060] The USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the terminal device 100, or to transfer data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0061] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present disclosure is for illustrative purposes only and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present disclosure, the terminal device 100 may also adopt different interface connection methods from those in the above embodiments, or a combination of multiple interface connection methods.

[0062] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141.

[0063] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.

[0064] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0065] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The antennas in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0066] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0067] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0068] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0069] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that terminal device 100 can communicate with a network and other devices via wireless communication technology. Wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0070] The terminal device 100 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0071] The display screen 194 is used to display images, videos, and receive sliding operations. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0072] The terminal device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0073] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0074] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0075] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0076] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. This allows terminal device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0077] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in the terminal device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0078] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0079] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0080] The terminal device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0081] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0082] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0083] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a call or voice message, the user can hear the voice by placing the receiver 170B close to the ear.

[0084] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In other embodiments, the terminal device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, and can also identify the source of sound, realize directional recording function, etc.

[0085] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0086] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device comprising at least two parallel plates with conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The terminal device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0087] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for shooting anti-shake. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0088] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0089] The magnetic sensor 180D includes a Hall effect sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Furthermore, based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0090] Accelerometer 180E can detect the magnitude of acceleration of the terminal device 100 in all directions (generally three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the electronic device's posture, enabling applications such as landscape and portrait screen switching and pedometers.

[0091] The distance sensor 180F is used to measure distance. The terminal device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0092] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect when the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0093] Ambient light sensor 180L is used to sense ambient light brightness. Terminal device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether terminal device 100 is in a pocket to prevent accidental touches.

[0094] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0095] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to prevent the terminal device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0096] The touch sensor 180K is also called a "touch control device." The touch sensor 180K can be provided on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be provided on the surface of the terminal device 100, at a location different from that of the display screen 194.

[0097] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bones of the human body's vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse the vibration signals of the vibrating bones of the vocal cords acquired by the bone conduction sensor 180M to obtain voice signals and implement voice functions. The application processor can parse heart rate information based on the blood pressure signals acquired by the bone conduction sensor 180M to implement heart rate detection functions.

[0098] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The terminal device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 100.

[0099] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0100] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0101] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the terminal device 100 by inserting or removing it from the SIM card interface 195. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0102] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present disclosure takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0103] FIG2 is a block diagram of the software structure of an electronic device according to an embodiment of the present disclosure.

[0104] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer, and the kernel layer. In the following description, a "process" can contain multiple "threads," while a "thread" can only belong to one process.

[0105] The application layer can include a series of application packages.

[0106] As shown in FIG2 , the application package may include applications such as phone, email, calendar, and camera, and each application may execute a corresponding application process.

[0107] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0108] As shown in FIG2 , in some embodiments, the application framework layer may include a window manager, a frame rate control system, an image synthesis system (referred to as a synthesis system), a view system, a package manager, an input manager, an activity manager, and a resource manager, etc.

[0109] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0110] The frame rate control system is used to adjust the screen refresh rate of electronic devices.

[0111] The synthesis system is used to control image synthesis and generate Vsync (vertical synchronization) signals.

[0112] The compositing system includes the compositing thread, the Vsync thread, and the queue buffer thread. The compositing thread, also known as the SurfaceFlinger thread (SF thread), is awakened by the Vsync signal to perform compositing. The Vsync thread generates the next Vsync signal based on the Vsync signal request. The queue buffer thread stores the buffer, generates Vsync signal requests, and wakes up the compositing thread.

[0113] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0114] The package manager is used for program management within the system, such as application installation, uninstallation, and upgrades.

[0115] The input manager is used to manage input devices. For example, the input system can determine input operations such as mouse clicks, keyboard inputs, and touch swipes.

[0116] The Activity Manager is used to manage the lifecycle of each application and the navigation back function. It is responsible for creating the Android main thread and maintaining the lifecycle of each application.

[0117] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0118] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0119] The core library can contain two parts: one part is the functional functions that need to be called by the Java language, and the other part is the Android core library.

[0120] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0121] The system library can include multiple functional modules, such as image rendering library, image synthesis library, function library, media library, and input processing library.

[0122] The image rendering library is used to render 2D or 3D images. The image synthesis library is used to synthesize 2D or 3D images.

[0123] In a possible implementation, the application renders the image using the image rendering library, and then sends the rendered image to the buffer queue of the compositing system. Whenever a Vsync signal arrives, the compositing system (e.g., surfaceflinger) sequentially retrieves a frame of image to be composited from the buffer queue, and then composites the image using the image compositing library.

[0124] The function library provides macros, type definitions, string operation functions, mathematical calculation functions, and input and output functions used in the C language.

[0125] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library supports a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0126] The input processing library is a library used to process input devices, which can implement mouse, keyboard and touch input processing, etc.

[0127] The hardware abstraction layer (HAL) can contain multiple library modules, such as the hardware composition thread (HWComposer, HWC) and the camera library module. The Android system can load the corresponding library modules for the device hardware, enabling the application framework layer to access the device hardware. Device hardware can include LCD displays and cameras in electronic devices.

[0128] The kernel layer is the layer between hardware and software. The kernel layer includes at least the touch panel (TP) driver, display driver, Bluetooth driver, Wi-Fi driver, keyboard driver, shared memory driver, and camera driver.

[0129] Hardware can be audio devices, Bluetooth devices, camera devices, sensor devices, etc.

[0130] The following describes the workflow of the software and hardware of the terminal device 100 by way of example, in conjunction with the scenario of application startup or interface switching in the application.

[0131] Among them, when the touch sensor 180K in the touch panel receives a touch operation from the outside world, the kernel layer processes the touch operation into an original input event (including touch coordinates, touch force, timestamp of the touch operation and other information). The original input event is stored in the kernel layer. The kernel layer reports the original input event to the input manager of the application framework layer through the input processing library. The input manager of the application framework layer parses the information of the original input event (including: operation type and reporting point position, etc.) and determines the focus application based on the current focus, and sends the parsed information to the focus application. The focus can be the touch point in a touch operation or the click position in a mouse click operation. The focus application is the application running in the foreground of the electronic device or the application corresponding to the touch position in the touch operation. The focus application determines the control corresponding to the original input event based on the parsed information of the original input event (for example, reporting point position).

[0132] The touch operation is a touch sliding operation, and the control corresponding to the touch sliding operation is WeChat Take the list control of the application as an example, WeChat The application uses the view system of the application framework layer to call the image rendering library in the system library to draw and render the image. The application sends the rendered image to the cache queue of the synthesis system. The image synthesis library in the system library synthesizes the rendered image in the synthesis system into WeChat Interface. The synthesis system uses the display driver of the kernel layer to make the screen (display) display WeChat The corresponding interface of the application.

[0133] To facilitate understanding, some of the examples are provided for reference in conjunction with descriptions of concepts related to the embodiments of the present disclosure.

[0134] 1. "User interface (UI)" is a media interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. UID (user identifier) ​​represents user identity proof or user identification number. The user interface is source code written in specific computer languages ​​such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to the user interface related to computer operations displayed in a graphical way. It can be visual interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0135] 2. Frame rate, also known as frame rate, is measured in frames per second, recorded as FPS. Frame rate refers to the number of image frames generated by the graphics card (i.e., GPU) per second, that is, the number of pictures that the graphics card can render and display per second. For example, the frame rate can be set to 30FPS or 60FPS. Since multiple frames of still images are displayed quickly and continuously to form a dynamic picture, the higher the frame rate of the video, the better the continuity of the picture. A high frame rate can produce a smoother and more realistic picture. The more frames per second, the smoother the displayed action will be, and the better the picture quality. It should be noted that the interface display frame of an electronic device usually goes through processes such as drawing, rendering, and synthesis.

[0136] 3. Screen refresh rate is the number of times the screen can refresh the image per second, measured in Hertz (Hz). For example, a 60Hz screen means that the screen can refresh the display 60 times in 1 second. Similarly, a 120Hz screen means that the screen can refresh the display 120 times in 1 second. Therefore, the higher the refresh rate, the more frames the screen can display. Correspondingly, the lower the image delay and the higher the smoothness.

[0137] Among them, when the user is using an electronic device, when the refresh rate and frame rate of the display are not synchronized, for example, when the number of frames per second is large (such as 90FPS or 120FPS) and the refresh rate is low (such as 30Hz), the screen will be stuck and even a tearing effect of screen image misalignment will occur.

[0138] 4. Vsync signal, used to synchronize the refresh rate and frame rate of the display to avoid screen tearing effect.

[0139] 5. The synthesizer, also known as the image synthesizer, can further integrate the image frame data sent by the synthesis system, send the data to the display driver, and then send it to the screen for display through the display driver.

[0140] The following describes the usual image drawing and display process. In the Android system, the image drawing and display process is completed through the collaboration of the software side, the display driver integrated circuit (DDIC) and the display screen. The software side first refreshes the page through the application (App), and then synthesizes the drawn layers through SurfaceFlinger to obtain image data. The image data then passes through the hardware synthesis module (HWC, hwcomposer) and the display driver, and then sends the image data to (write to) the DDIC through the mobile industry processor interface (MIPI). The DDIC stores the image data sent by the software side in a buffer, and by scanning (reading) the image data in the buffer, controls the display screen (such as OLED or LCD) to complete the refresh rate switching, and the display screen performs image refresh display (display).

[0141] In high refresh rate display scenarios, the software generates image data at a high frequency, and accordingly, the display screen refreshes the image at a high frequency based on the image data, thereby improving the smoothness of the picture. In other words, the layer rendering performed by the app and the layer synthesis performed by SurfaceFlinger are both controlled by the Vsync cycle, which determines the speed at which the image displayed on the screen changes after the user clicks or slides on the screen.

[0142] In addition, since the human eye is more sensitive to moving objects, a higher screen refresh rate is required in various dynamic display scenarios. For example, when some applications use the UI thread to perform background tasks in the background, since it does not involve drawing and rendering, its operating frequency is irrelevant to the user experience. If it works at a high refresh rate, it will increase the power consumption of the application and waste resources. In addition, when the foreground process has a high refresh rate working frequency, the background task also has a high refresh rate. When there are multiple APP scenarios in the current background, it will lead to waste of system-level chip resources, causing the background or foreground weak-perception processes to preempt the foreground CPU / GPU / DDR resources, which not only wastes power consumption but also affects the performance of the entire machine.

[0143] Therefore, in the embodiment of the present disclosure, when the electronic device wants to display the scene normally, the current screen refresh rate can be switched to the normal screen refresh rate to meet the high frame rate display requirements. If the current application does not need to be rendered, or belongs to a background or foreground weak perception process, the sending frequency of the Vsync signal can be adjusted to achieve the purpose of adjusting the application refresh rate and reduce unnecessary power consumption. This will not only not affect the user experience, but also improve the overall performance of the device.

[0144] Based on this, the embodiment of the present disclosure provides a method for controlling the Vsync signal. By identifying non-perceived applications, background applications and other applications that do not require high-frequency refresh, the frequency of the Vsync thread sending the Vsync signal is controlled to achieve control of the application refresh frequency, thereby reducing unnecessary application thread power consumption and optimizing the overall machine load, ensuring that foreground tasks can enjoy system resources first.

[0145] For example, FIG3 shows a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure.

[0146] As shown in FIG3 , a method for controlling a Vsync signal according to an embodiment of the present disclosure includes the following steps:

[0147] S11: The synthesis system receives a Vsync signal request message sent by the application main thread;

[0148] S12: After receiving the Vsync signal request message, the synthesis system determines the number of rendering times of the application's rendering thread in a unit time, and then compares the number of rendering times of the rendering thread in a unit time with the system refresh rate;

[0149] S13: When the number of rendering times per unit time by the rendering thread is equal to the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a first frequency, where the first frequency is equal to the system refresh rate;

[0150] S14: When the number of rendering times per unit time of the rendering thread is less than the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a second frequency, where the second frequency is less than the system refresh rate.

[0151] For example, since the second frequency is lower than the first frequency, when adjusting the frequency of the Vsync signal sent by the Vsync thread, the Vsync thread may not send the Vsync signal at at least part of the time, so that the Vsync thread switches from the first frequency to the second frequency to send the Vsync signal to the application main thread. The at least part of the time is the time when the Vsync signal determined based on the system refresh rate needs to be sent. In other words, at the time when the Vsync signal determined based on the system refresh rate needs to be sent, choosing not to send the Vsync signal can lower the sending frequency of the Vsync signal. Specifically, the specific value of the second frequency can be determined by the number of Vsync signals not sent by the Vsync thread, that is, the number of Vsync signals skipped.

[0152] In some embodiments of the present disclosure, after receiving the Vsync signal request message, the synthesis system determines the number of rendering times of the application's rendering thread per unit time (referred to as rendering times). When the number of rendering times of the rendering thread per unit time is less than the system refresh rate of the electronic device, the synthesis system may set a timer to monitor the time point at which the next Vsync signal arrives (the time point at which the Vsync signal needs to be sent based on the system refresh rate) through the timer, and not send the Vsync signal at the next time point, so that the Vsync thread sends the Vsync signal to the application main thread at the second frequency.

[0153] Among them, the value of the second frequency can be adjusted according to the number or times that the Vsync thread does not send the Vsync signal, that is, the Vsync thread skips at least one Vsync signal and does not send it. When the Vsync thread does not send the Vsync signal, the application main thread cannot measure, layout and draw the image frame according to the Vsync signal, thereby avoiding the application main thread waking up the rendering thread to draw unnecessary image frames.

[0154] In addition, when the number of rendering times per unit time of the rendering thread is equal to the system refresh rate of the electronic device, the Vsync thread can send a Vsync signal to the application main thread at the system refresh rate. The application main thread measures and layouts the image frames according to the Vsync signal. When the measurement and layout results meet the preset drawing conditions, the application main thread triggers the rendering thread to draw and render the image frames. When the measurement and layout results do not meet the preset drawing conditions, the application main thread does not trigger the rendering thread.

[0155] In a specific embodiment of the present disclosure, the synthesis system adjusts the frequency of the Vsync signal sent by the Vsync thread by setting a timer. The duration of one cycle of the timer can be set based on the system refresh rate of the electronic device. When the set time of the timer is reached, the synthesis system can wake up the main thread of the application based on the Vsync signal generated by the Vsync thread to draw the corresponding image frame, and synthesize the drawn image frame through the synthesis system and display it on the terminal. Otherwise, the synthesis system can set the timer again and wait for the next Vsync signal to arrive until the frequency of the Vsync signal sent by the Vsync thread is adjusted to the second frequency, thereby adjusting the refresh rate of the application.

[0156] In addition, when the number of rendering times of the rendering thread per unit time is less than the system refresh rate, the proportional relationship between the number of rendering times of the rendering thread per unit time and the system refresh rate can be further determined. When the number of rendering times of the rendering thread per unit time is 0, the second frequency can be directly set within a preset range. When the number of rendering times of the rendering thread per unit time is not 0 and is less than 1 / n of the system refresh rate, the second frequency can be set to 1 / n of the system refresh rate, where n is a positive integer, and 1 / n can be taken as the minimum value of the system refresh rate when the number of rendering times of the rendering thread per unit time is less than the minimum value of the system refresh rate. For example, when the number of rendering times of the rendering thread per unit time is 32 times and the system refresh rate is 120 times, the second frequency can be set to 1 / 2 or 1 / 3 of the system refresh rate, among which 1 / 3 is preferred, that is, n is 3.

[0157] It can be seen that in this embodiment, after receiving the Vsync signal request message, the synthesis system adjusts the frequency of the Vsync signal sent by the Vsync thread based on the number of rendering times of the rendering thread in unit time. It can be seen that other parameters can also be judged to achieve the same frequency adjustment of the Vsync signal sent by the Vsync thread.

[0158] For example, FIG4 shows a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure.

[0159] As shown in FIG4 , a method for controlling a Vsync signal according to an embodiment of the present disclosure includes the following steps:

[0160] S21: The system receives a Vsync signal request message sent by the main thread of the application;

[0161] S22: After receiving the Vsync signal request message, the synthesis system determines the number of requests for the application main thread to send the Vsync signal request message within a unit time;

[0162] S23: When the number of times the application main thread sends the Vsync signal request message within a unit time is equal to the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at a third frequency, where the third frequency is equal to the system refresh rate;

[0163] S24: When the number of times the application main thread sends the Vsync signal request message within a unit time is less than the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at a fourth frequency, and the fourth frequency is less than the system refresh rate.

[0164] Specifically, in this embodiment, after receiving the Vsync signal request message, the synthesis system adjusts the frequency of the Vsync signal sent by the Vsync thread based on the number of requests (request times) by the application main thread to send the Vsync signal request message per unit time. When the number of requests is equal to the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at the system refresh rate, and the application main thread measures, lays out, and draws the image frame according to the Vsync signal. When the number of requests is less than the system refresh rate of the electronic device, it is necessary to adjust the sending frequency of the Vsync signal, that is, the Vsync thread sends the Vsync signal to the application main thread at a fourth frequency that is less than the system refresh rate, thereby reducing the sending frequency of the Vsync signal. The frequency of waking up the application main thread through the Vsync signal will also be reduced, ultimately achieving the purpose of reducing power consumption.

[0165] In some embodiments, when the number of requests for the application main thread to send Vsync signal request messages per unit time is less than 1 / m of the system refresh rate, the fourth frequency can be set to 1 / m of the system refresh rate, where m is a positive integer, and 1 / m can be the minimum value of the number of requests less than the system refresh rate. The value of m can refer to the value of n in the above-mentioned rendering number judgment embodiment.

[0166] Among them, when adjusting the frequency of the Vsync signal sent by the Vsync thread, the Vsync thread may not send the Vsync signal at at least part of the time, so that the Vsync thread switches from the third frequency to the fourth frequency to send the Vsync signal to the application main thread. This at least part of the time is the time when the Vsync signal needs to be sent based on the system refresh rate.

[0167] It should be noted that, for the frequency adjustment method of the Vsync thread sending the Vsync signal and the measurement, layout and drawing process of the application main thread, please refer to the description in the above embodiment of adjusting the Vsync signal sending frequency based on the number of rendering times of the rendering thread in unit time, which will not be repeated here.

[0168] For example, FIG5 shows a flow chart of a method for controlling a Vsync signal according to an embodiment of the present disclosure.

[0169] As shown in FIG5 , a method for controlling a Vsync signal according to an embodiment of the present disclosure includes the following steps:

[0170] S31: The synthesis system receives a Vsync signal request message sent by the application main thread;

[0171] S32: After receiving the Vsync signal request message, the composition system determines the layer parameters of the image frame drawn by the application main thread;

[0172] S33: When the layer parameters meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a fifth frequency, where the fifth frequency is equal to the system refresh rate;

[0173] S34: When the layer parameters do not meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency, where the sixth frequency is less than the system refresh rate.

[0174] In this embodiment, after receiving a Vsync signal request message, the composition system adjusts the frequency of Vsync signals sent by the Vsync thread based on the layer parameters of the image frame drawn by the application main thread. If the layer parameters meet preset conditions, the Vsync thread sends a Vsync signal to the application main thread at the system frequency to wake up the application main thread to measure, layout, and draw the image frame. If the layer parameters do not meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency that is less than the system refresh rate. In this case, the refresh rate of the application main thread is also reduced, thereby reducing unnecessary power consumption of the application thread.

[0175] As a specific example, the Vsync thread does not send the Vsync signal at least partially, so that the Vsync thread sends the Vsync signal to the application main thread at the sixth frequency, and at least partially the time is the time when the Vsync signal needs to be sent based on the system refresh rate.

[0176] Specifically, the layer parameters of an image frame may include at least one of layer area, layer transparency, scaling factor, and application layer identifier. If the layer parameter of an image frame is layer area, if the layer area of ​​the image frame is less than a first preset value, the layer parameter does not meet the preset condition; if the layer area of ​​the image frame is greater than or equal to the first preset value, the layer parameter meets the preset condition. If the layer parameter of an image frame is layer transparency, if the layer transparency of the image frame is greater than a second preset value, the layer parameter does not meet the preset condition; if the layer transparency of the image frame is less than or equal to the second preset value, the layer parameter meets the preset condition. If the layer parameter of an image frame is scaling factor, if the scaling factor of the image frame is greater than a third preset value, the layer parameter does not meet the preset condition; if the scaling factor of the image frame is less than or equal to the third preset value, the layer parameter meets the preset condition. If the layer parameter of an image frame is application layer identifier, if the application layer identifier of the image frame falls within a preset range of a perceptible application layer list, the layer parameter meets the preset condition; if the application layer identifier of the image frame does not fall within the preset range of the perceptible application layer list, the layer parameter does not meet the preset condition.

[0177] It should be noted that the Vsync signal control method disclosed herein can be understood as three different request parameters related to the application main thread, namely, the number of rendering times of the rendering thread per unit time, the number of request times of the application main thread sending Vsync signal request messages per unit time, and whether the layer parameters meet the preset conditions. When determining the Vsync signal sending frequency, the number of rendering times of the rendering thread per unit time, the number of request times of the application main thread sending Vsync signal request messages per unit time, and whether the layer parameters meet the frequency adjustment conditions can be selected for judgment, that is, the judgment of the number of rendering times, the number of request times, and the layer parameters is in an "or" relationship; in addition, the number of rendering times, the number of request times, and the layer parameters can also be judged at the same time, in which case the relationship between the three is "and". When the relationship between the above conditions is "and", the judgment order and combination of the conditions can be flexibly set according to the specific application scenario. For example, the number of rendering times, the number of request times, and the layer parameters can be judged in sequence; or the number of rendering times and the number of request times can be judged in sequence; or the number of rendering times and the layer parameters can be judged in sequence; or the layer parameters and the number of request times can be judged in sequence, and so on.

[0178] Exemplarily, the synthesis system adjusts the frequency of the Vsync signal sent by the Vsync thread by setting a timer. When the setting time of the previous timer arrives, the synthesis system can determine whether the above-mentioned request parameters meet the corresponding adjustment conditions for the Vsync signal sending frequency. If the frequency adjustment conditions are met (including the number of rendering times being less than the system refresh rate, the number of requests being less than the system refresh rate, and the layer parameters not meeting the frequency adjustment conditions), the frequency of the Vsync signal sent by the Vsync thread can be lowered. If the request parameters do not meet the frequency adjustment conditions, the Vsync thread sends the Vsync signal to the application main thread at its inherent frequency, such as the system refresh rate.

[0179] In some embodiments, after receiving the Vsync signal sent by the Vsync thread, the application main thread calls the callback function through the Vsync signal to wake up the application main thread, and the application main thread measures and layouts the image frame. When the measurement and layout results meet the preset drawing conditions, the application main thread triggers the rendering thread, and the rendering thread renders the image frame to form a rendering cache number and sends it to the synthesis system. The synthesis thread in the synthesis system synthesizes the image frame according to the rendering cache data and sends it to the screen for display.

[0180] As a specific example, applying the main thread to draw an image frame according to the Vsync signal and displaying it through the terminal may further include the following steps:

[0181] Step 1: The synthesis system wakes up the Vsync thread in the synthesis system based on the callback function;

[0182] Step 2: The awakened Vsync thread generates a Vsync signal and sends the Vsync signal to the application main thread at the adjusted sending frequency, thereby waking up the application main thread through the Vsync signal;

[0183] Step 3: Based on the application main thread, the rendering thread in the application process is awakened, and the image frame is rendered and drawn by the rendering thread to form rendering buffer data, and sent to the synthesis system;

[0184] Step 4: The synthesis system synthesizes the image frames according to the rendering cache data to form a display image, and sends it to the terminal for final display.

[0185] In some embodiments, when rendering and drawing an image frame through a rendering thread, the rendering thread will form rendering cache data after rendering and send it to the synthesis system; wherein, the rendering cache data can be temporarily stored in a cache area to form a cache queue. In the synthesis system, layer synthesis can be performed in sequence according to the rendering cache data in the cache queue, and the synthesized layer can be sent to the terminal for display. wherein, when performing layer synthesis based on the rendering cache data, the synthesis system can count the layer PID (Process Identification) and the current timestamp of the rendering cache data to obtain the number of renderings of the current application's rendering thread per unit time, and then determine the corresponding rendering rate; then, the synthesis thread performs layer synthesis based on the rendering rate.

[0186] As a specific example, the following will take three different request parameters, namely the number of rendering times, the number of requests and the judgment between the layer parameters and the corresponding frequency adjustment conditions as examples to respectively explain in detail the control method of the Vsync signal disclosed in the present invention.

[0187] Embodiment 1: Controlling the sending of the Vsync signal based on the relationship between the number of rendering times of a rendering thread per unit time and the system refresh rate.

[0188] The basis or request parameter for frequency adjustment is the number of rendering times of the rendering thread per unit time. In embodiment 1, when the number of rendering times of the rendering thread per unit time is less than the system refresh rate, it means that the request parameter meets the frequency adjustment condition, and the sending frequency of the Vsync signal can be reduced at this time.

[0189] The method for controlling the Vsync signal shown in this embodiment of the first embodiment includes:

[0190] First, the synthesis thread obtains the number of renderings of the application's rendering thread per unit time, such as 1 second, based on the first historical map object data. Then, when the number of renderings is less than the system refresh rate, it is determined that the number of renderings meets the frequency adjustment condition, thereby causing the Vsync thread to not send the Vsync signal at least partially, thus skipping several Vsync signals and adjusting the frequency of the Vsync thread sending the Vsync signal to a second frequency that is less than the system refresh rate. The refresh rate of the application's main thread will also be reduced. This process can be further classified and processed as follows based on the specific number of renderings:

[0191] 1. If the composition system determines that the rendering thread has rendered zero times per unit time, it can directly control the refresh rate of the current application in the main application thread to a preset range; this preset range can be set according to the scenario, but is usually lower than the system refresh rate;

[0192] 2. If the synthesis system determines that the rendering thread renders a non-zero number of times per unit time and is less than 1 / n of the system refresh rate, the Vsync thread skips at least one Vsync signal and does not send it. The number of Vsync signal skips is adjusted, that is, the Vsync thread skips n-1 Vsync signals, and the frequency of the Vsync signal is adjusted to 1 / n of the system refresh rate, that is, the second frequency. The application refresh rate is also reduced to the original 1 / n. It can be seen that the more Vsync signals are skipped, the lower the application refresh rate will be. Wherein, n is a positive integer among the preset k positive integers, so that when n is the positive integer, the difference between the value of the system refresh rate divided by the positive integer and the rendering number is the smallest and greater than 0, that is, n=max{n i |(system refresh rate / n i )-rendering times > 0 and n i ∈N, i=1~k, k≠0 and k∈N}. For example, when the rendering thread renders 39 times in 1s and the system refresh rate is 120, the preset multiple positive integers (n1~n k ) is 2, 3, or 6. It can be seen that the rendering times will be less than 1 / 2 of the system refresh rate, and also less than 1 / 3 of the system refresh rate. At this time, 1 / n can be taken as the minimum value between the two, that is, n is 3. At this time, the Vsync thread can skip 2 Vsync signals. When the third Vsync signal arrives, the main thread of the application is woken up to measure, layout, and draw the image frame.

[0193] For example, when the user is browsing a page or playing a game on the front end, and applications such as music playback are used in the background, the music playback application may not render image frames (the rendering times of the application's rendering thread are zero at this time). The user only perceives the sound of the music playback, but not the changes in the interface of the music playback application. Therefore, the refresh rate of such applications can be lowered and adjusted, and the user will not experience any difference. In addition, when the user experiences low-frame rate games such as Candy Crush Saga, when the system refresh rate is 120Hz, the main thread of the application will still render image frames at a frequency of 60Hz. At this time, the frequency of the Vsync thread sending the Vsync signal can be reduced to 60Hz, and the refresh rate of the application can be adjusted to save half the power consumption of the application UI thread.

[0194] Among them, the above-mentioned first historical map object data includes the rendering times of the application's rendering thread within 1 second. The data can be obtained based on the relevant data of the rendering thread in the historical rendering process. Among them, the first historical map object data includes a key value and a value value corresponding to the key value, wherein the key value is used to record the PID of the application, and the value value is used to record the rendering times corresponding to the PID. When the rendering times are less than the system refresh rate, it is determined that the frequency of sending the Vsync signal to the Vsync thread needs to be adjusted.

[0195] Example 2

[0196] The sending of the Vsync signal is controlled based on the relationship between the number of times the application main thread sends the Vsync signal request message per unit time and the system refresh rate.

[0197] In the second embodiment, when the number of times the application main thread sends Vsync signal request messages per unit time is less than the system refresh rate, it indicates that the request parameters meet the frequency adjustment conditions, and the sending frequency of the Vsync signal can be reduced.

[0198] The Vsync signal control method shown in this embodiment includes:

[0199] The synthesis system obtains the number of requests for sending Vsync signal request messages by the application main thread within a unit time, for example, 1 second, based on the second historical map object data;

[0200] The synthesis system determines whether the number of requests is less than the system refresh rate;

[0201] When the synthesis system determines that the number of requests is less than the system refresh rate, the Vsync thread sends a Vsync signal to the application main thread at a fourth frequency, and the fourth frequency is less than the system refresh rate.

[0202] For example, if the synthesis system determines that the number of times the application main thread sends Vsync signal request messages per unit time is less than 1 / m of the current system refresh rate, the Vsync thread can skip m-1 Vsync signals and reduce the sending frequency of the Vsync signal to the original 1 / m; wherein m is a positive integer among the preset p positive integers, so that when m is the positive integer, the difference between the value of the system refresh rate divided by the positive integer and the number of requests is minimum and greater than zero, wherein p is a positive integer, that is, m=max{m i |(system refresh rate / m i )-Number of requests > 0 and m i ∈N, i=1~p, p≠0 and p∈N}, the value of m is determined in the same way as the value of n in the above-mentioned rendering times determination process, and the specific examples can refer to each other.

[0203] Among them, the above-mentioned second historical map object data includes the number of requests for each application to send Vsync signal request messages within a unit time, for example, within 1 second. This data can be obtained based on the relevant data of the number of requests for Vsync signal request messages sent by the application main thread. The second historical map data also includes key values ​​and value values, wherein the key value is used to record the PID of the application, and the value value is used to record the number of requests for the application main thread corresponding to the PID to send Vsync signal request messages.

[0204] It should be noted that the number of rendering times of the above-mentioned rendering thread per unit time and the number of requests of the application main thread to send Vsync signal request messages per unit time can be judged separately or in combination with each other for comprehensive judgment. When the number of rendering times and the number of requests are judged at the same time, if the number of rendering times satisfies the condition of not adjusting the frequency of sending the Vsync signal, it can be further judged whether the number of requests meets the frequency adjustment condition. If one of the two meets the frequency adjustment condition, the frequency of sending the Vsync signal of the Vsync thread can be directly reduced.

[0205] As a specific embodiment, when the number of rendering times and the number of requests are combined for judgment, the control method of the Vsync signal may include: when the application main thread sends rendering cache data to the synthesis system, the synthesis system counts the number of rendering times of the application's rendering thread rendering image frames within a preset time, and records the number of requests for the Vsync signal request message sent by the application main thread. When the number of requests of the application main thread is approximately equal to the current system refresh rate, the synthesis system further judges the situation of the rendering cache. If the application does not send rendering cache data to the synthesis system, the refresh rate of the application main thread is directly reduced to below the critical value or within a preset range. If the number of times the application sends rendering cache data to the synthesis system is not 0 and is less than 1 / 2, 1 / 3, 1 / 4, etc. of the current system refresh rate, the synthesis system determines that the current application's rendering speed cannot meet the current frame rate requirement, and the synthesis system can reduce the frequency of the Vsync thread sending the Vsync signal, and the application refresh frequency is reduced to 1 / 2, 1 / 3, 1 / 4, etc. of the original.

[0206] In this second embodiment, if the number of times the application main thread sends Vsync signal request messages is relatively low, the synthesis system can directly reduce the application refresh rate by adjusting the Vsync sending frequency. For example, when playing the Star Ball game in the Alipay application, at a refresh rate of 120Hz, the application may request Vsync signals less than 1 / 2 of the system refresh rate due to lag. For the user's performance experience, the application refresh rate can be fixed at less than 1 / 2 of the system refresh rate.

[0207] In this second implementation example, the second historical map object data includes the layer PID and the requested frame rate of the application main thread sending the Vsync signal request message per unit time (the requested frame rate can be determined based on the number of requests). In the process of judging the number of requests, first, the synthesis system will determine the timestamp of the last Vsync signal consumption of the historical application with the same layer PID based on the layer PID of the application main thread that sends the Vsync signal request message; then, obtain the difference between the current time and the time point corresponding to the last consumption timestamp; determine whether the difference meets the preset frame rate threshold. If not, adjust the sending frequency of the Vsync signal through the Vsync thread, and adjust the frequency of the application main thread sending the Vsync signal request message. The preset frame rate threshold can be set according to the specific application scenario.

[0208] It can be seen that this embodiment can adjust the frequency of the Vsync signal sent by the Vsync thread according to the number of requests for the Vsync signal request message sent by the application main thread, thereby reducing the sending frequency of the Vsync signal for unnecessary requests from the application main thread and applications whose request speed cannot meet the current frame rate, thereby achieving the adjustment of the application refresh rate and preventing resource waste.

[0209] Example 3:

[0210] The sending of the Vsync signal is controlled based on the relationship between the layer parameters of the image frame drawn by the main thread of the application and the preset conditions.

[0211] In this third embodiment, the layer parameters include the layer area, layer transparency, scaling factor, and application layer identifier of the image frame requested to be drawn; when the layer parameters do not meet the preset conditions, it means that the request parameters meet the frequency adjustment conditions. At this time, the sending frequency of the Vsync signal can be reduced, and the Vsync thread sends the Vsync signal at a sixth frequency that is less than the system refresh rate.

[0212] In the third embodiment, the method for controlling the Vsync signal includes:

[0213] The synthesis system obtains the layer area in the layer parameters of the current image frame requested to be drawn based on the third historical map object data; the synthesis system determines the relationship between the current layer area and the first preset value. If the layer area is smaller than the first preset value, the first preset value can be set according to the size of the display screen of the specific electronic device. It can be considered that the area of ​​the image frame requested to be drawn is small and has little impact on the user's perceptual experience. Rendering and drawing can be omitted for such image frames. Therefore, it can be considered that the layer parameters of the current image frame meet the frequency adjustment conditions, and the Vsync thread does not send the Vsync signal at least part of the time, so that the Vsync thread sends the Vsync signal to the application main thread at the sixth frequency.

[0214] In the third embodiment, the Vsync signal control method may further include:

[0215] Based on the third historical map object data, the synthesis system obtains the layer transparency in the layer parameters of the current image frame requested to be drawn, and determines the relationship between the current layer transparency and the second preset value through the synthesis system. If the layer transparency of the current image frame is greater than the second preset value, it can be considered that the transparency of the image frame requested to be drawn is higher, which may be a blurred display covering other layers displayed on the display screen, or a display that has little impact on the user experience. For such cases, it can be considered that the layer parameters of the current image frame meet the frequency adjustment conditions, and the Vsync thread does not send the Vsync signal at least part of the time, so that the Vsync thread sends the Vsync signal to the application main thread at the sixth frequency.

[0216] As an example, layer transparency is expressed as a percentage. Usually, the stage from a layer to colorless and transparent can be divided into 100 parts, and then the percentage is used to express the effect of the current image frame after overlapping another layer. For example, the alpha channel (Alpha Channel) or the alpha value of a layer refers to the degree of transparency or translucency of an image, which affects the effect of its overlapping with another image (or background).

[0217] In the third embodiment, the Vsync signal control method may further include:

[0218] Based on the third historical map object data, the synthesis system obtains the scaling factor in the layer parameters of the current image frame requested to be drawn, and determines the relationship between the current scaling factor and the third preset value through the synthesis system. If the scaling factor of the current image frame is greater than the third preset value, it can be considered that the scaling factor of the image frame requested to be drawn is large, resulting in a reduction in resolution. For such cases, it can be considered that the layer parameters of the current image frame meet the frequency adjustment conditions, and the Vsync thread does not send the Vsync signal at at least part of the time, so that the Vsync thread sends the Vsync signal to the application main thread at the sixth frequency.

[0219] During the display process, the image frame will be scaled. The absolute pixels of the layer, that is, the final actual value of the pixels, are usually controlled by two parameters: resolution and scale. The larger the image scale, the lower the resolution. Therefore, when determining whether the layer parameters meet the third preset condition, the scale threshold can be used as a reference. Then, some layers with resolutions that meet the requirements can be displayed, while other layers are ignored.

[0220] In the third embodiment, the Vsync signal control method may further include:

[0221] Based on the third historical map object data, the composition system obtains an application layer identifier from the layer parameters of the current image frame requested for drawing. The composition system then obtains a preset list of perceptible application layers and determines whether the application layer identifier belongs to the preset list of perceptible application layers. The application layer identifier may be a unique identifier for the layer or an identification symbol that can distinguish different layers, such as a layer process ID. If the application layer identifier does not belong to the preset list of perceptible application layers, the composition system determines that the layer parameters of the current image frame meet the frequency adjustment conditions and, through the Vsync thread, does not send a Vsync signal at least partially, or directly sets a corresponding frame rate range for the case where the application layer identifier does not belong to the perceptible application layer list and adjusts the frame rate according to the frame rate range.

[0222] In this third embodiment, the composition system can use the application layer identifier to find the layer PID of the corresponding layer. Based on the layer PID, it determines whether the current application is the focused application. If not, the current application can be considered a background process and its application UID can be found. At this time, if the allowed frame rate range for background processes is configured, then when the application main thread sends a Vsync signal request message, the composition system can control the message interval of the application main thread sending the request message, controlling the Vsync signal request rhythm within a preset range, for example, the preset range can be set to 20Hz-40Hz. In addition, when there is no preset frame rate range, the application refresh rate can be adjusted based on the relationship between the system refresh rate and the application refresh rate.

[0223] It should be noted that, in the above-mentioned process of judging the layer area, layer transparency, zoom factor and application layer identifier of the current image frame, the above-mentioned situations are in an "or" relationship, that is, when judging whether the current image frame parameters meet the preset conditions, the layer area, layer transparency, zoom factor and application layer identifier of the current image frame can be obtained at the same time. If one of them cannot be met, it can be considered that the current image frame belongs to the situation where the user perception is not strong. At this time, the Vsync thread does not send the Vsync signal at least part of the time, and adjusts the refresh rate of the application.

[0224] In this third embodiment, the perceptible application list is a list of layer ranges corresponding to functions that are highly perceived by the user, of particular interest to the user, or that are frequently used. In specific applications, the range of the perceptible application list can be set based on the usage scenario. Furthermore, when users are listening to music, watching videos, playing games, browsing the web, and other operations, they can flexibly set corresponding preset conditions based on different application scenarios, optionally setting the layer area, transparency, zoom factor, and application layer identifier.

[0225] Among them, the above-mentioned third historical map object data includes data such as the area, transparency, zoom factor, application layer identifier, etc. of the current image frame corresponding to the layer parameters. The third historical map object data includes a key value and a value value corresponding to the key value, wherein the key value is used to record the PID of the application, and the value value is used to record the area value, transparency value, zoom factor value, application layer identifier, etc. corresponding to the PID.

[0226] In this third embodiment, when determining the layer transparency and scaling factor of the above-mentioned current image frame, in order to simplify the processing process, the synthesis system may also perform block processing on the current image frame, that is, divide the current image frame into at least two areas, such as upper, middle and lower or left, middle and right area divisions, and then sample the transparency and scaling factor of each area, and then perform hash operations on the sampled data of each part, and finally determine the layer transparency and scaling factor based on the results of the hash operations.

[0227] On the other hand, when the layer parameters do not meet the preset conditions, the synthesis system can further record the layer PID and application UID of the current image frame, and search for the configured frame rate range of the unaware application according to the preset configuration file; based on the frame rate range of the unaware application, the frame rate of the Vsync signal requested by the application UID is controlled.

[0228] It can be seen that the above-mentioned embodiment three can determine whether to adjust the sending frequency of the Vsync signal through the Vsync thread through the layer parameters of the current image frame, and can ignore the foreground applications and background threads that are not highly perceived by the user, thereby preventing invalid high-frequency refresh in the background when the foreground is working at a high refresh rate, thereby avoiding waste of SOC resources and improving the performance of the entire machine.

[0229] Since the number of rendering times of the rendering thread per unit time can be determined based on the first historical map object data, the number of requests for the application main thread to send the Vsync signal request message per unit time can be determined based on the second historical map object data, and the layer parameters of the image frame can be determined based on the third historical map object data, when the sending frequency of the Vsync signal is controlled by the above three request parameters, at least one request parameter can be selected, or at least two request parameters can be selected for combined control. For example, when the sending frequency of the Vsync signal is controlled by a combination of three request parameters, it is necessary to obtain the first historical map object data, the second historical map object data and the third historical map object data, and then determine the corresponding request parameters based on the value values ​​recorded in the three object data. The three object data can be obtained at different stages of the image frame drawing process.

[0230] Specifically, FIG. 6-1 and FIG. 6-2 together illustrate a schematic flow of a process for acquiring first historical map object data, second historical map object data, and third historical map object data in a method for controlling a Vsync signal according to an embodiment of the present disclosure.

[0231] As shown in Figures 6-1 and 6-2, the processes involved in this process may include application processes (including the application main thread and rendering thread), synthesis system processes (including SF threads and Vsync threads), and HWC hardware synthesizer processes. The processes involved here are the main modules for implementing this process. In actual implementation, other module processes will also be included. The specific process can be determined according to actual usage requirements and is not limited by the embodiments of this disclosure. As shown in Figures 6-1 and 6-2, the process may include steps A1 to A10.

[0232] HWC has the function or ability to use hardware to combine and display image data. Specific image display can be achieved through the collaboration of multiple classes such as SurfaceFlinger (SF), HWC, and the display. HWC is the hardware abstraction layer for window / layer synthesis and display in the Android system. Its implementation is device-specific and is usually completed by the display device manufacturer (OEM) to provide hardware support for the SurfaceFlinger service.

[0233] It should be noted that the Vsync signal can be divided into a software Vsync signal and a hardware Vsync signal. The software Vsync signal includes Vsync-APP and Vsync-SF. Vsync-APP is used to trigger the drawing and rendering thread in the application process. Vsync-SF is used to trigger the synthesis thread in the synthesis system. The hardware Vsync signal (Vsync-HW) is used to trigger the screen display refresh process. Since the Vsync signal includes Vsync-APP and Vsync-SF, when adjusting the sending frequency of the Vsync signal, the sending frequency of Vsync-APP or the sending frequency of Vsync-SF can be adjusted separately, or the sending frequencies of Vsync-APP and Vsync-SF can be adjusted at the same time. All of the above situations can reduce power consumption waste. The following will be described in detail using the adjustment of the sending frequency of Vsync-APP as an example.

[0234] Step A1: After the service is started, the SF class is initialized and the configuration file is loaded;

[0235] Step A11: The synthesis system loads the configuration file and reads the frame rate range that can be automatically adjusted in the embodiment of the present disclosure, that is, the preset range, such as 20 Hz-40 Hz.

[0236] Step A2: the application main thread draws the Nth frame of the image online;

[0237] Step A21: the application main thread calls the requestNextVsync() function to request the next Vsync-app message from the SF thread, which is used to wake up the application main thread to draw the N+1th frame;

[0238] A3: The SF thread calculates the time point of the next Vsync-app message, records the layer PID and socket fd of the current Vsync-app message request, and sets a timer. The timer is set to wake up the application main thread to draw the N+1 frame after the timer expires. Before the timer expires, the Vsync thread does not send a Vsync signal to the application main thread.

[0239] A31: The synthesis system updates map object A (i.e., the second historical map object data, the same below) and records the number of times the application corresponding to the layer PID sends Vsync signal request messages within 1 second;

[0240] A4: When the timer expires, the synthesis system traverses to determine which applications have requested the Vsync-app, and sends the Vsync signal to the socket fd that requested the Vsync signal;

[0241] A41: The composition system updates the map object M (this data is used to determine the application refresh rate) and records the time when the layer PID last received the Vsync-app message. Since the map object M records the time when the application corresponding to the layer PID last received the Vsync-app message, the application refresh rate can be determined based on this time point and the current time so that it can be adjusted. For example, when the screen is off, the application refresh rate can be directly adjusted to a preset range.

[0242] A5: The Vsync thread sends the Vsync signal to the application main thread, and wakes up the application main thread accordingly;

[0243] A51: The application main thread performs measurement and layout based on the Vsync signal, and calls the rendering thread to draw the N+1 frame;

[0244] A511: The application main thread calls the requestNextVsync() function to request the next Vsync-app message from the SF thread, which is used to wake up the main thread to draw the N+2 frame;

[0245] A512: The SF thread calculates the time point of the next Vsync-app message, records the PID and socket fd of the current Vsync-app message request, and sets a timer. When the timer expires, the application main thread is woken up to draw the N+2 frame.

[0246] A513: The synthesis system updates map object A and records the number of times the layer PID sends Vsync signal request messages within 1 second;

[0247] A52: The rendering thread calls the dequeueBuffer() function to expose an empty buffer from the cache queue (queueBuffer) for rendering; the cache queue is used to store the buffer, generate Vsync signal requests, and wake up the SF thread, etc.

[0248] A53: The rendering thread sends the rendered Buffer to the SF thread through the cache queue function;

[0249] A6: The compositing system calls the setTransactionState() function in the SF thread across threads and sets the Vsync-sf timer in the function to wake up the SF main thread for compositing.

[0250] A61: The synthesis system updates map object B (i.e., the first historical map object data, the same below), and records the number of queueBufers of the layer PID within 1 second, also known as the rendering count;

[0251] A7: The Vsync-sf timer expires, and the Vsync thread wakes up the SF thread for synthesis.

[0252] A71: SF thread synthesizes the buffer of the N+1th frame;

[0253] A72: The compositing system iterates over all visible layers in the Output:collectVisibleLayers function.

[0254] A73: The synthesis system calculates the layer area size of each visible layer, searches for the layer PID to which the layer belongs, records the layer area of ​​the visible layer corresponding to each layer PID, updates the map object C (i.e., the area in the third historical map object data), records the transparency of the visible layer corresponding to each layer PID (i.e., the transparency in the third historical map object data), records the screen status update status value E, and records whether the screen is currently on.

[0255] A8: The synthesis system completes synthesis of the image frame and calls the HwcPresentDisplay() function to send the data to the HWC hardware synthesizer process;

[0256] A81: The HWC hardware compositor process further integrates the layer data and sends the data to the display driver. The display driver waits for the TE signal to arrive and sends it to the screen for display.

[0257] After the above steps are completed, the first historical map object data, the second historical map object data and the third historical map object data can be obtained, and then when the application main thread sends the Vsync signal request message again, a judgment can be made based on this and the application refresh rate can be adjusted.

[0258] A9: When the next timer expires, the synthesis system traverses the Vsync-apps requested by the applications.

[0259] A91: The composition system determines the current application refresh rate based on object M and the current time t, and then decides whether to send a Vsync-app message to the application based on map objects A, B, C, and D;

[0260] A10: When the decision result of step A91 is to send the Vsync-app message, the Vsync thread sends a Vsync signal to the application main thread to wake up the main thread.

[0261] It can be seen that if the decision result of step A91 is not to send, a timer is set to continue waiting for the next judgment.

[0262] After obtaining the first historical map object data, the second historical map object data and the third historical map object data through the above process, in A91, based on the current time t and the map objects A, B, C, D, M, a decision can be made whether to send a Vsync-app message to the application main thread through the Vsync thread.

[0263] Specifically, FIG7 shows a process for deciding whether to send a Vsync signal based on historical map data according to an embodiment of the present disclosure, that is, a step of judging the request parameters. Step A91 may further include the following steps:

[0264] 1. The synthesis system determines the time interval between the object M and the current time t, and determines the application refresh rate based on the time interval so that it can be adjusted later;

[0265] 2. The synthesis system determines whether the current terminal display is off based on the status value E. If so, the application refresh rate is directly controlled within a preset range and step A10 is executed;

[0266] Among them, if there are APP applications such as QQ Music, they are 120Hz when used in the foreground. After the screen is turned off, the UI and rendering threads of the application still work at a frequency of 120Hz, which will cause the whole machine to consume more power. At this time, if the screen is turned off, the application refresh rate can be directly reduced to within the preset range.

[0267] 3. If the current terminal display is not off, then based on object B, calculate the number of times the application's rendering thread renders the buffer within 1 second (i.e., the number of renderings), and make a judgment. If no buffer is generated, execute the above step 2; if the number of times the buffer is rendered is less than 1 / n of the current system refresh rate, control the application refresh rate to 1 / n of the system refresh rate, and execute step A10;

[0268] 4. If the number of rendering buffers is not less than 1 / n of the current system refresh rate, determine based on object A whether the number of Vsync signal request messages sent by the application is less than 1 / m of the system refresh rate; if so, adjust the application refresh rate to 1 / m of the system refresh rate based on object A and execute step A10;

[0269] 5. If the number of times the application sends a Vsync signal request message is not less than 1 / m of the system refresh rate, determine whether the area and transparency of the current image frame meet the requirements based on object C and object D; if so, directly execute step A10; otherwise, control the Vsync frequency within a preset range by setting a timer, and then execute step A10.

[0270] As a specific example, if the visible area of ​​the current image frame is less than 1 / 3 of the terminal screen area, or its transparency alpha value is less than 0.5, it may indicate that the current layer parameters do not meet the requirements, and the Vsync thread adjusts the rate of sending the Vsync signal to be lower than the system refresh rate.

[0271] It should be noted that the specific numerical values ​​shown in Figure 7 of the present disclosure are all examples and are not limited to the only embodiment of the present disclosure. For example, in the process of determining the area and transparency of the current image frame, the zoom factor and whether it belongs to the preset perceptible application layer list can be further determined.

[0272] As a specific example, FIG8 shows a schematic process of switching the frequency of sending a Vsync signal by a Vsync thread according to an embodiment of the present application.

[0273] As shown in FIG8 , the Vsync signal control method according to an embodiment of the present application may include:

[0274] S1: The main application thread draws the current image frame, i.e., frame 10, and requests the next Vsync signal request message from the composition system;

[0275] S2: The Vsync thread in the synthesis system receives the Vsync signal request message sent by the application main thread, sets the next timer to trigger the next Vsync-APP event, and records the number of Vsync signal request messages sent by the application main thread;

[0276] S3: Render frame 10 and put the resulting rendering buffer data into the cache queue of the compositing system. The compositing system records the number of application rendering buffers to determine the number of rendering times per unit time.

[0277] S4: The Vsync thread of the synthesis system sends a Vsync signal to the main thread of the application;

[0278] S5: When the time of the next timer set in S2 arrives (Vsync-APP, ID3), the synthesis system synthesizes frame 10, displays historical frame 9, and sends a next Vsync signal request message to request the next Vsync signal for drawing and rendering frame 11;

[0279] S6: After receiving the Vsync signal sent by the Vsync thread at (Vsync-APP, ID5), the application main thread draws and renders frame 11 and requests the next Vsync signal;

[0280] S7: After looping through the above steps for a period of time, the synthesis system calculates that the number of times the application's rendering is rendered per unit time is less than 1 / 2 of the system refresh rate, adjusts the frequency of the Vsync thread sending Vsync signals, reduces the frequency of the Vsync thread sending signals to the application's main thread to 1 / 2 of the system refresh rate, and records the time when the Vsync signal is sent;

[0281] S8: After receiving the Vsync signal sent by the Vsync thread at (Vsync-APP, ID34), the application main thread draws the rendering frame 24 and requests the next Vsync signal;

[0282] S9: At the time point when the next Vsync signal should arrive, which is determined based on the system refresh rate, the synthesis system calculates the difference between this time point and the time point when the Vsync signal was last sent. If the difference is less than the length of the frame period corresponding to 1 / 2 of the system refresh rate, the Vsync signal is not sent at the current moment, and a timer is set to trigger the next Vsync-APP signal.

[0283] S10: When the next timer time is reached, the synthesis system calculates the time difference between the current time and the time of sending the Vsync signal in S7, which is not less than the length of the frame period corresponding to 1 / 2 of the system refresh rate. Then, the Vsync signal is sent through the Vsync thread at the current moment, thereby adjusting the frequency of sending the Vsync signal.

[0284] FIG9 is a schematic diagram showing the hardware structure of a data processing device provided by an embodiment of the present disclosure.

[0285] As shown in FIG9 , the device includes: a memory 1901, a processor 1902, and an interface circuit 1903. The device may also include a display screen 1904, wherein the memory 1901, the processor 1902, the interface circuit 1903, and the display screen 1904 can communicate; illustratively, the memory 1901, the processor 1902, the interface circuit 1903, and the display screen 1904 can communicate via a communication bus, the memory 1901 is used to store computer-executable instructions, the execution of which is controlled by the processor 1902, and the communication is performed by the interface circuit 1903, thereby implementing the data processing method provided in the embodiments of the present disclosure.

[0286] Optionally, the interface circuit 1903 may further include a transmitter and / or a receiver. Optionally, the processor 1902 may include one or more CPUs, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present disclosure may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0287] The data processing device provided in the embodiment of the present disclosure as shown in FIG9 is used to execute the data processing method of the above embodiment. The technical principles and technical effects are similar and will not be described in detail here.

[0288] The present disclosure provides an electronic device, the structure of which is shown in Figure 1 . The electronic device's memory can be used to store at least one program instruction, and its processor can be used to execute at least one program instruction to implement the technical solution of the aforementioned method embodiment. The implementation principles and technical effects are similar to those of the aforementioned method embodiment and are not further described here.

[0289] The present disclosure provides a chip. The chip includes a processor configured to invoke a computer program stored in a memory to execute the technical solutions described in the above embodiments. The implementation principles and technical effects are similar to those of the above-described related embodiments and will not be further elaborated here.

[0290] The present disclosure provides a computer program product that, when executed on an electronic device, enables the terminal device to execute the technical solution in the above embodiment. The implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be repeated here.

[0291] The present disclosure provides a computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by a terminal device, the terminal device executes the technical solution of the above-mentioned embodiment. The implementation principle and technical effects thereof are similar to those of the above-mentioned related embodiments and will not be further described here.

[0292] The present disclosure embodiment is described with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each process and / or box in the flow chart and / or block diagram and the combination of the process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes and / or one box or multiple boxes of the flow chart. For those skilled in the art, it is obvious that the present disclosure is not limited to the details of the above-mentioned exemplary embodiments, and without departing from the spirit or basic characteristics of the present disclosure, the present disclosure can be realized in other specific forms.

[0293] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present disclosure is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0294] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not limiting. Although the present disclosure has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present disclosure may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present disclosure.

Claims

1. A method for controlling a Vsync signal, characterized in that: The method is applied to an electronic device, and includes: The synthesis system receives the Vsync signal request message sent by the application main thread; After receiving the Vsync signal request message, the synthesis system determines the number of rendering times of the rendering thread of the application within a unit time; When the number of rendering times per unit time by the rendering thread is equal to the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a first frequency, where the first frequency is equal to the system refresh rate; When the number of rendering times of the rendering thread per unit time is less than the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a second frequency, and the second frequency is less than the system refresh rate.

2. The method according to claim 1, characterized in that The Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at a second frequency; The at least part of the time is the time when the Vsync signal needs to be sent, which is determined based on the system refresh rate.

3. The method according to claim 1, characterized in that When the number of rendering times of the rendering thread per unit time is less than 1 / n of the system refresh rate, the second frequency is 1 / n of the system refresh rate, where n is a positive integer.

4. The method according to claim 1, wherein The application main line performs measurement and layout of image frames; When the measurement and layout results meet the preset drawing conditions, the application main thread triggers the rendering thread to perform image frame drawing and rendering; When the measurement and layout results do not meet the preset drawing conditions, the application main thread does not trigger the rendering thread.

5. A method for controlling a Vsync signal, characterized in that: The method is applied to an electronic device, and includes: The synthesis system receives the Vsync signal request message sent by the application main thread; After receiving the Vsync signal request message, the synthesis system determines the number of requests for the application main thread to send the Vsync signal request message within a unit time; When the number of times the application main thread sends the Vsync signal request message per unit time is equal to the system refresh rate of the electronic device, the Vsync thread sends the Vsync signal to the application main thread at a third frequency, where the third frequency is equal to the system refresh rate; When the number of times the application main thread sends a Vsync signal request message per unit time is less than the system refresh rate of the electronic device, the Vsync thread sends a Vsync signal to the application main thread at a fourth frequency, and the fourth frequency is less than the system refresh rate.

6. The method according to claim 5, characterized in that The Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at a fourth frequency; The at least part of the time is the time when the Vsync signal needs to be sent, which is determined based on the system refresh rate.

7. The method according to claim 5, characterized in that When the number of times the application main thread sends the Vsync signal request message within a unit time is less than 1 / m of the system refresh rate, the fourth frequency is 1 / m of the system refresh rate, where m is a positive integer.

8. A method for controlling a Vsync signal, characterized in that: The method is applied to an electronic device, and includes: The synthesis system receives the Vsync signal request message sent by the application main thread; After receiving the Vsync signal request message, the composition system determines the layer parameters of the image frame drawn by the main thread of the application; When the layer parameters meet the preset conditions, the Vsync thread sends a Vsync signal to the application main thread at a fifth frequency, where the fifth frequency is equal to the system refresh rate; When the layer parameter does not meet the preset condition, the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency, and the sixth frequency is less than the system refresh rate.

9. The method according to claim 8, characterized in that The Vsync thread does not send a Vsync signal at least partially, so that the Vsync thread sends a Vsync signal to the application main thread at a sixth frequency; The at least part of the time is the time when the Vsync signal needs to be sent, which is determined based on the system refresh rate.

10. The method according to claim 8, characterized in that The layer parameters of the image frame include at least one of a layer area, a layer transparency, a zoom factor, and an application layer identifier.

11. The method according to claim 10, characterized in that When the layer parameter of the image frame is a layer area, if the layer area of ​​the image frame is smaller than a first preset value, the layer parameter does not meet the preset condition; When the layer area of ​​the image frame is greater than or equal to the first preset value, the layer parameter meets the preset condition.

12. The method according to claim 10, characterized in that When the layer parameter of the image frame is layer transparency, if the layer transparency of the image frame is greater than a second preset value, the layer parameter does not meet the preset condition; When the layer transparency of the image frame is less than or equal to the second preset value, the layer parameter meets the preset condition.

13. The method according to claim 10, characterized in that When the layer parameter of the image frame is a scaling factor, if the scaling factor of the image frame is greater than a third preset value, the layer parameter does not meet the preset condition; When the scaling factor of the image frame is less than or equal to the third preset value, the layer parameter meets the preset condition.

14. The method according to claim 10, characterized in that When the layer parameter of the image frame is an application layer identifier, and the application layer identifier of the image frame belongs to a preset range of a perceptible application layer list, the layer parameter meets the preset condition; When the application layer identifier of the image frame does not belong to the preset perceptible application layer list range, the layer parameter does not meet the preset condition.

15. An electronic device, characterized in that: The electronic device includes at least one processor; and A memory and a display screen in communication with the at least one processor; wherein, The display screen is used to display the application interface; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the Vsync signal control method according to any one of claims 1 to 14.

16. A computer-readable storage medium storing a computer program, characterized in that: When the computer instructions are run on the terminal device, the computer program is executed by the processor to implement the steps in the Vsync signal control method according to any one of claims 1 to 14.

17. A chip, characterized in that: The chip includes a processor, and the processor is used to call a computer program in a memory to execute the Vsync signal control method according to any one of claims 1 to 14.

Images