Dynamic region refreshing method and device, terminal and storage medium

Abstract

The invention provides a dynamic region refreshing method and device, a terminal and a storage medium, and relates to the technical field of communication, and the method comprises the steps: comparing a frame with a current frame to generate dirty region information; determining whether there is an update region for the current frame and the target frame based on the dirty region information; if the updating areas for the current frame and the target frame exist, performing local refreshing according to the updating areas for the current frame and the target frame; wherein the target frame is the next frame of the current frame. According to the method and the device, local refreshing can be performed when the DPU executes refreshing, so that the use power consumption of the terminal is reduced.

Description

[Technical field]

[0001] Embodiments of the present invention relate to the technical field of communications, and in particular, to a dynamic area refresh method, device, terminal and storage medium. [Background technology]

[0002] With the improvement of intelligence, users have put forward higher and higher requirements on the performance of handheld terminal products (such as mobile phones and tablet computers), especially with respect to terminal energy consumption and heat generation.

[0003] Energy consumption analysis of terminals shows that when users use handheld terminal products, the screen consumes a lot of energy, especially when refreshing the handheld screen. Although the relevant technology mentions the strategy of adopting local refresh, it mainly reduces the load of rendering data to reduce power consumption. This power reduction method is not applicable in some situations. Therefore, it is urgent to provide a local area refresh method for use in all scenarios to reduce receipt power consumption. [Summary of the invention]

[0004] The embodiments of the present invention provide a dynamic area refresh method, device, terminal and storage medium, so as to perform local refresh when a DPU executes refresh, thereby reducing the power consumption of the terminal.

[0005] In a first aspect, the present application provides a dynamic area refresh method, which is applied to a terminal and specifically includes the following steps:

[0006] In response to a refresh request for a target frame, rendering the target frame generates a rendering result of the target frame, and comparing the target frame with the current frame to generate dirty area information;

[0007] Determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0008] If there are update areas for the current frame and the target frame, local refresh is performed according to the update areas for the current frame and the target frame;

[0009] The target frame is the next frame of the current frame.

[0010] Through the embodiments of the present application, the configuration of the DPU is modified according to the dirty area information when refreshing, thereby reducing the load of the DPU when refreshing and reducing the power consumption of the mobile phone.

[0011] One possible approach is that the terminal includes a display processor DPU,

[0012] The steps of performing local refresh according to the update area of ​​the current frame and the target frame include:

[0013] The dirty area information and the target layer are synchronized to the display processor DPU, the display processor DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering results of the update area for the current frame and the target frame, a local refresh is performed according to the update area for the current frame and the target frame;

[0014] The target layer is generated by the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0015] One possible way is that the terminal uses SurfaceFlinger and hardware synthesizer HWC to send the display;

[0016] The step of determining whether there is an update area for the current frame and the target frame based on the dirty area information includes:

[0017] In response to the synthesis display request, the dirty area information and the target layer are synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the dirty area information and the target layer to the hardware synthesizer HWC;

[0018] The hardware synthesizer HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information;

[0019] The synthetic display request is generated after the refresh request for the target frame is generated.

[0020] One possible approach is that if there is no update region for the current frame and the target frame, the method further includes:

[0021] A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

[0022] One possible way is that the rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, the dirty area information and the target layer are synchronized to the display processor DPU, the display processor DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering result of the update area of ​​the current frame and the target frame, a process of performing a partial refresh according to the update area of ​​the current frame and the target frame includes:

[0023] The display processor DPU reads rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

[0024] In a second aspect, the present application provides a dynamic area refresh method, which is applied to a terminal and executed in a rendering thread, wherein the rendering thread includes: a first sub-thread and a second sub-thread;

[0025] In response to a refresh request for a target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer;

[0026] The first sub-thread generates a rendering completion notification, which is completed after rendering the target frame, and sends the rendering completion notification to the second sub-thread;

[0027] The second sub-thread obtains dirty area information in response to the rendering completion notification, and determines whether there is an update area for the current frame and the target frame based on the dirty area information;

[0028] If there are update regions for the current frame and the target frame, the update regions for the current frame and the target frame are partially refreshed based on the dirty area information, the target layer, and the rendering results of the update regions for the current frame and the target frame;

[0029] The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0030] One possible way is that the terminal uses SurfaceFlinger, hardware synthesizer HWC and display processor DPU to send display;

[0031] In the step of synchronizing the target layer by the second sub-thread, the second sub-thread performs the following operations:

[0032] In response to the synthesis display request, the target layer is synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the target layer to the hardware synthesizer HWC;

[0033] The hardware synthesizer HWC obtains the full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes it to the display processor DPU;

[0034] The composite display request is generated after the refresh request for the target frame is generated.

[0035] One possible approach is that the process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering results of the update area for the current frame and the target frame includes:

[0036] The display processor DPU modifies the full-screen configuration information into dirty area information.

[0037] One possible approach is that if there is no update region for the current frame and the target frame, the method further includes:

[0038] A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

[0039] One possible approach is that the rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, and the process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering result for the update area for the current frame and the target frame includes:

[0040] The display processor DPU reads rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

[0041] One possible approach is that the dirty region information includes an identifier corresponding to the target frame.

[0042] In a third aspect, the present application provides a dynamic area refresh device, applied to a terminal, comprising:

[0043] A first response module: used for responding to a refresh request for a target frame, rendering the target frame to generate a rendering result of the target frame, and comparing the target frame with the current frame to generate dirty area information;

[0044] The first determination module is used to determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0045] The first execution module: if there are update areas for the current frame and the target frame, then locally refresh according to the update areas for the current frame and the target frame;

[0046] The target frame is the next frame of the current frame.

[0047] In a fourth aspect, an embodiment of the present application provides a dynamic area refresh device, which is applied to a terminal and executed in a rendering thread, and the rendering thread includes: a first sub-thread and a second sub-thread;

[0048] The second response module is used to respond to the refresh request for the target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer;

[0049] Sending module: used for the first sub-thread to generate a rendering completion notification, which is completed after the target frame is rendered, and sends the rendering completion notification to the second sub-thread;

[0050] A second determination module: used for the second sub-thread to obtain dirty area information in response to the rendering completion notification, and determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0051] The second execution module is used for partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer and the rendering results of the update area for the current frame and the target frame if there is an update area for the current frame and the target frame;

[0052] The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0053] In a fifth aspect, an embodiment of the present application provides a terminal, including:

[0054] at least one processor; and at least one memory in communication with the processor, wherein:

[0055] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method of the first aspect or the second aspect.

[0056] In a sixth aspect, the present application provides a computer-readable storage medium storing computer instructions, wherein the computer instructions enable a computer to execute a method as claimed in the first aspect or the second aspect.

[0057] It should be understood that the second to sixth aspects of the embodiments of the present invention are consistent with the technical solutions of the first aspect of the embodiments of the present invention, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be described in detail.

Brief Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 It is a schematic diagram of local refresh in the prior art;

[0060] Figure 2(a) to Figure 2(b) It is a schematic diagram of display refresh for different applications;

[0061] Figure 3 A schematic diagram of a dynamic area refresh method provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of the terminal structure provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of the software structure provided for the embodiment of the present application;

[0064] Figure 6 A schematic diagram of a scenario provided for an embodiment of the present application;

[0065] FIG. 7( a) is a schematic diagram of an exemplary current frame provided in an embodiment of the present application;

[0066] FIG7( b) is a schematic diagram of an exemplary target frame provided in an embodiment of the present application;

[0067] FIG7(c) is a schematic diagram of dirty area information based on FIG7(a) and FIG7(b) provided in an embodiment of the present application;

[0068] Figure 8 A schematic diagram of another dynamic area refresh method provided in an embodiment of the present application;

[0069] Fig. 9 A schematic diagram of data reading under another dynamic area refresh method provided in an embodiment of the present application;

[0070] Fig.10 A schematic diagram of another dynamic area refresh method provided in an embodiment of the present application;

[0071] Fig.11 A flow chart of a dynamic area refresh method provided in an embodiment of the present application;

[0072] Fig.12 A flow chart of another dynamic area refresh method provided in an embodiment of the present application;

[0073] Fig.13 A structural diagram of a dynamic area refresh device provided in an embodiment of the present application;

[0074] Fig.14 A structural diagram of another dynamic area refresh device provided in an embodiment of the present application. [Specific implementation method]

[0075] In order to better understand the technical solution of the embodiment of the present invention, the embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0076] In order to reduce power consumption of handheld terminal screens, the relevant technology specifically adopts the method of partial refresh, that is, to study how to reduce the load of GPU when writing DDR data, please refer to Figure 1In the stage of writing DDR data, GPU first performs local rendering, and reuses the previous frame for the unrendered part, that is, keeps it the same as the current display interface frame, and writes the locally rendered data into DDR. In the process of synthesizing and displaying, DPU reads the whole frame data, and uses MIPI interface to send the whole frame data to the screen. Combined with the above, it can be seen that in the process of DPU reading DDR data and DPU sending data to the screen, it still reads the whole frame data. For this technical solution, it can only be used for the layers of ordinary applications. The so-called layers of ordinary applications are mainly characterized by the layers of ordinary applications having multiple view controls forming a large window. The view controls are used to achieve local rendering. Please refer to Figure 2(a). The areas numbered 1, 2, and 3 in the figure represent the windows composed of view controls. When the two frames before and after the area numbered 3 change, only the area numbered 3 needs to be refreshed.

[0077] For some applications, such as game applications, in order to improve the efficiency of rendering and drawing and reduce display delay, their layers do not have independent view controls, so that each frame is a whole. Therefore, even if there is only a small difference in content between the previous and next frames, the display system still sends full-screen data. Even when the game screen is completely still or there is no content update, the terminal still repeatedly sends the same data to the DPU for re-update according to the preset fixed frame. That is to say, referring to Figure 2(b), although only the diagonal area between the previous and next frames has changed, the entire frame still needs to be refreshed.

[0078] Due to the particularity of the game layer, the current method of local rendering for local refresh is not suitable for game scenes. To this end, it is urgent to provide a power reduction strategy that can be applicable to game scenes.

[0079] For this, please refer to Figure 3 The embodiments of the present application provide a dynamic area refresh method, device, terminal and readable medium. For the game layer without VIEW control, the GPU is used to count the dirty area information in real time, and the DPU is set using the dirty area information, so that the DPU performs a local refresh. Here, the dirty area information can be synchronized to the DPU in steps with the layer (see the paths of 1.1 and 1.2, and the DPU configuration is modified using the dirty area configuration of the system), or it can be synchronized to the DPU with the layer (refer to the path of 2). The DPU is directly configured according to the dirty area information, thereby realizing that the DPU only needs to read the local load from the DDR when refreshing, thereby reducing energy consumption.

[0080] First, the terminal provided in the embodiment of the present application is introduced: Figure 4 A schematic structural diagram of the terminal 100 is shown.

[0081] Figure 4A schematic diagram of the structure of the terminal 100 is shown.

[0082] The terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0083] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0084] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0085] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0086] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0087] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0088] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal 100.

[0089] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0090] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0091] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0092] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal 100.

[0093] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0094] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal 100, and can also be used to transmit data between the terminal 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0095] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0096] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0097] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0098] The wireless communication function of the terminal 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0099] The terminal 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0100] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0101] The terminal 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0102] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0103] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0104] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0105] Video codecs are used to compress or decompress digital videos. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record videos in multiple coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0106] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0107] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0108] The terminal 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0109] Figure 5 It is a software structure block diagram of the terminal 100 according to an embodiment of the present invention.

[0110] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0111] The application layer can include a series of application packages.

[0112] like Figure 5 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0113] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0114] like Figure 5 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0115] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0116] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0117] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0118] A 2D graphics engine is a drawing engine for 2D drawings.

[0119] The following is a brief introduction to the terms involved in this application:

[0120] GPU: Graphics Processing Unit, a microprocessor for image processing. As mentioned above, GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0121] DPU: Display Processing Unit, display processor, its main purpose is scan conversion. Specifically, scan conversion is the process of displaying different graphic objects as a set of pixels. In the embodiments described later in this application, the main function of DPU is to refresh the data transmitted from the upper layer according to the configuration information.

[0122] DDR: Double Data Rate, double data rate synchronous dynamic random access memory. During the process of DPU executing refresh and display, DPU needs to read data load from DDR. It is understandable that during the process of DPU executing refresh and display, the power consumption of the terminal is positively correlated with the size of the data load read from DDR. In other words, during the process of DPU executing refresh and display, the larger the data load read by DPU from DDR, the greater the power consumption of the terminal.

[0123] Layers, composed of many pixels, are the basic units that make up an image. A frame of an image can be a layer or multiple layers stacked up and down. Layers are like films containing elements such as text or graphics, which are stacked together in order to form the final effect of the image.

[0124] The following describes the dynamic area refresh method provided by the present application.

[0125] Please refer to Figure 6 , the current player is using the mobile phone to play shooting competitive games.

[0126] For shooting competitive games, users use the phone buttons or the phone screen to control the object to shoot the target to win. During this game process, the DPU refreshes at a preset frequency. Assuming that the user uses the phone buttons or the phone screen to control the object to move, at this time, there is a difference between the previous and next frames of the phone. Assuming that the object controlled by the player remains still and waits for the game results of other players, the previous and next frames of the phone remain consistent.

[0127] When the time interval between the current moment and the last execution of the refresh process reaches a threshold, a refresh request for the target frame is generated at this time. The terminal responds to the refresh request for the target frame by rendering the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information;

[0128] It should be noted that the current frame in the embodiment of the present application corresponds to the frame to be refreshed, and the current frame is the frame displayed on the current mobile phone interface. Please refer to Figure 7(a). Assuming that the time interval between the current moment and the last execution of the refresh process reaches the threshold, the mobile phone currently displays frame A, where frame A corresponds to the current frame, and the next frame of the current frame corresponds to frame B, which corresponds to the target frame, as shown in Figure 7(b). For ease of distinction, the target frame is represented by a dotted line and the current frame is represented by a solid line.

[0129] It is worth noting that the dirty area information here includes: the position information where the target frame differs from the current frame. Specifically, the position information here includes: the coordinate information and pixel information where the target frame differs from the current frame. The dirty area information can be used to determine whether there is a difference between the target frame and the current frame.

[0130] Please refer to FIG. 7( c ), for example, when the object currently controlled by the player moves, the area that changes between the current frame and the target frame (ie, the diagonal area) corresponds to the dirty area information.

[0131] It should also be noted that the rendering for the target frame is a whole-frame rendering. Combined with the foregoing, a GPU is provided in the mobile phone, and the GPU is used to perform rendering for the target frame. At the same time, a comparison module is provided in the GPU. The main function of the comparison module is to perform comparison in real time to output dirty area information in real time. At the same time, it can be understood that the rendered target frame is stored in the DDR.

[0132] After the GPU outputs the dirty region information in real time, it can determine whether there is an update region for the current frame and the target frame based on the dirty region information.

[0133] For example, refer to Figure 7(a) to Figure 7(c) , the current player controls the object to move, the current frame is frame A, and the target frame is frame B. The coordinate information and pixel information of the oblique line area in Figure 7(c) are dirty area information. Based on this, the terminal performs a judgment and determines that there is an update area for the current frame and the target frame. At this time, the terminal performs a partial refresh on the update area of ​​the current frame and the target frame, and the rest of the frame remains the original frame, that is, the current frame.

[0134] In this step, the DPU specifically executes the refresh process. First, the DPU will modify the configuration properties of the DPU based on the dirty area information and the target layer. The modified configuration properties can enable the DPU to perform a partial refresh on the target layer only for the update area of ​​the current frame and the target frame.

[0135] Furthermore, in this step, the DPU will read the rendering results of the updated areas of the current frame and the target frame from the rendering results of the target frame based on the dirty area information. Combined with Figure 7(c), the DPU will read the rendering results of the diagonal area from the DDR, which reduces energy consumption compared to reading the rendering results of the entire frame from the DDR in the related art. At the same time, in order to realize data communication between the DPU and the screen, a MIPI interface is set between the DPU and the screen. Compared with the traditional display mode of transmitting the entire frame to the screen through the MIPI interface, in the present application, only the rendering results of the diagonal area need to be transmitted to the screen for display, which further achieves energy saving.

[0136] That is, the steps of performing local refresh according to the update area of ​​the current frame and the target frame include:

[0137] The dirty area information and the target layer are synchronized to the DPU. The DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering results of the update area for the current frame and the target frame, a local refresh is performed according to the update area for the current frame and the target frame.

[0138] It should be noted that the target layer is generated by the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0139] It should be noted that, in the present application, there is no specific limitation on how to calculate the dirty area information and how to synchronize the dirty area information with the target layer. That is to say, the dirty area information and the target layer can be synchronized by using the Client's synthesis display process or the Device's SurfaceFlinger synthesis display process in combination with the relevant technology.

[0140] The main difference between the Client's SurfaceFlinger synthesis and display delivery process and the Device's SurfaceFlinger synthesis and display delivery process is: in the Client's SurfaceFlinger synthesis and display delivery process, the GPU is responsible for synthesizing the layers synchronized by SurfaceFlinger, while for the Device's SurfaceFlinger synthesis and display delivery process, the GPU is not responsible for layer synthesis. By comparison, when the Client's SurfaceFlinger synthesis and display delivery process is adopted, it will undoubtedly increase the GPU's operating load. Therefore, in order to further reduce the load, the Device's SurfaceFlinger synthesis and display delivery process is adopted in the display delivery process.

[0141] For the SurfaceFlinger synthesis and display process using Device on the terminal side, HWC is responsible for determining whether there is an update area for the current frame and the target frame based on the dirty area information. Furthermore, in this step, after the dirty area information is generated, the terminal generates a synthesis and display request. The terminal synchronizes the dirty area information as an attribute value together with the target layer to SurfaceFlinger. SurfaceFlinger synchronizes the target layer and the dirty area information to HWC. HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information. If so, the aforementioned DPU display process is executed.

[0142] That is, the step of determining whether there is an update region for the current frame and the target frame based on the dirty region information includes:

[0143] In response to the composite display request, the dirty area information and the target layer are synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the dirty area information and the target layer to HWC.

[0144] HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information, where the target layer is generated by the target frame.

[0145] For the specific DPU display sending process, based on the above, when the DPU is refreshed, the DPU configuration needs to be modified according to the dirty area information. For the Device's SurfaceFlinger synthesis display sending process, before executing the DPU display sending process, the HWC first needs to synchronize the target layer and dirty area information to the DPU. At this time, the DPU is configured according to the dirty area information and performs a local refresh on the update area of ​​the current frame and the target frame.

[0146] It is worth noting that the above synchronization of dirty area information through HWC is only one possible implementation method in the present application. Those skilled in the art may refer to other methods for synchronizing dirty area information and target layers to DPU, which is not limited here.

[0147] It should be noted that, for the unrefreshed part, the mobile phone reuses the current frame, and after refreshing, the mobile phone screen displays the target frame.

[0148] The above are the specific steps of the SurfaceFlinger synthesis display sending process of the Device. Through the above display sending method, energy consumption can be further reduced.

[0149] On the basis of the foregoing embodiment, in order to further reduce invalid load, for the situation where the object controlled by the player remains still and waits for the game results of other players, since the previous and next frames of the mobile phone are consistent, based on the foregoing, the dirty area information can determine the situation where there is a difference between the target frame and the current frame. In this scenario, the corresponding dirty area information is empty. At this time, the DPU display sending process can be not executed. That is to say, the mobile phone uses HWC to determine whether there is an update area for the current frame and the target frame based on the dirty area information. If the dirty area information is empty, the DPU display sending process is skipped directly, that is, the HWC directly generates a display sending flag, and feeds the display sending flag back to SurfaceFlinger, and the mobile phone screen reuses the current frame.

[0150] That is, if there is no update region for the current frame and the target frame, the method further includes:

[0151] A display sending completion mark is generated. In response to the display sending completion mark, the terminal screen reuses the current frame. At this time, the DPU does not perform a local refresh on the update area of ​​the current frame and the target frame.

[0152] Based on the above embodiments, the dynamic area refresh method of the embodiment of the present application is specifically introduced as follows:

[0153] Please refer to Figure 8 ,Assuming that a refresh request for the target frame is generated, first the main thread of the mobile phone obtains the refresh request for the target frame, and the main thread assigns the refresh request for the target frame to the rendering thread. At this time, the rendering thread responds to the refresh request for the target frame, commands the GPU to render the target frame to generate the rendering result of the target frame, and stores the rendered target frame in DDR. The GPU is equipped with a comparison module, which outputs the comparison result between the current frame and the target frame, that is, the dirty area information, in real time.

[0154] After the dirty area information is generated, the synthesis and display process begins. In this process, the mobile phone synchronizes the dirty area information as an attribute value together with the target layer to SurfaceFlinger. It should be noted that the rendering thread is responsible for executing the above synchronization process, and the target layer is determined based on the target frame. At this time, SurfaceFlinger obtains the dirty area information and the target layer, and synchronizes the dirty area information and the target layer to HWC. HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information. If so, a local refresh is performed on the update area of ​​the current frame and the target frame, that is, the dirty area information and the target layer are synchronized to the DPU. The DPU is configured according to the dirty area information. At this time, the DPU reads the rendering results of the update area of ​​the current frame and the target frame from the DDR, and transmits the rendering results of the update area of ​​the current frame and the target frame to the mobile phone screen using the MIPI interface.

[0155] When the display transmission is completed, a display transmission completion prompt message is generated. The DPU responds to the display transmission completion prompt message and marks the display transmission completion. At this time, a display transmission completion flag is generated and the display transmission completion flag is fed back to SurfaceFlinger.

[0156] In some situations, the current frame and the target frame remain stationary. In this case, the DPU directly generates a display completion mark. At this time, the DPU does not perform a refresh on the target frame, and the mobile phone screen displays the current frame.

[0157] Please refer to Fig. 9 For data transmission, GPU writes the rendering result of the target frame (i.e. the whole frame data) into DDR. When DPU refreshes, DPU reads the rendering result of the updated area of ​​the current frame and the target frame (i.e. local data). For DPU transmission to the screen, only the rendering result of the updated area of ​​the current frame and the target frame (i.e. local data) is transmitted. That is, the whole frame data does not need to be transmitted during refresh and MIPI transmission, thereby achieving power saving.

[0158] It should be noted that the dynamic area refresh method provided in the embodiments of the present application is not only applicable to game applications, but can also be used for other applications to achieve power consumption reduction for DDR reading and MIPI interface transmission.

[0159] Based on the above embodiments, it can be seen that for SurfaceFlinger, the GPU must perform rendering before it can be displayed, and the GPU rendering requires waiting time, which affects the refresh speed. In order to further improve the refresh speed, an asynchronous processing method is adopted here to process GPU rendering and synthetic display at the same time.

[0160] Another dynamic area refresh method provided by the present application is described below.

[0161] Consistent with the above embodiment, referring to Figure 4 , assuming that the current player is using a mobile phone to play a shooting competitive game.

[0162] Reference Fig.10 For asynchronous processing, the rendering thread of the mobile phone is provided with two sub-threads, namely the first sub-thread and the second sub-thread ( Fig.10The first sub-thread corresponds to the GPU thread, which is mainly used to generate rendering results and perform real-time comparison. For the description of dirty area information, please refer to the above, which will not be repeated here. The second sub-thread corresponds to the synthesis and display thread. Regarding the synthesis and display thread, its synthesis and display process can be the SurfaceFlinger synthesis and display process of the Client and the SurfaceFlinger synthesis and display process of the Device. Please refer to the above for the specific difference. To avoid repetition, it will not be repeated here.

[0163] Assume that when the player controls the object to move, when the rendering thread obtains the refresh request of the target frame, in response to the refresh request of the target frame, the first sub-thread renders the target frame to generate the rendering result of the target frame, and the second sub-thread synthesizes the target layer, that is, the GPU renders the target frame and generates the rendering result of the target frame, and at the same time, the synthesis and display are performed synchronously. When the first sub-thread completes the rendering, the dirty area information is output. For the description of the dirty area information, please refer to the above, and it will not be repeated to avoid repetition.

[0164] When the rendering is completed, the first sub-thread generates a rendering completion notification and sends it to the sub-thread. At this time, the terminal determines the dirty area information to determine whether there is an update area for the current frame and the target frame. Since the player controls the object to move, there is an update area for the current frame and the target frame. At this time, based on the dirty area information and the target layer, the rendering results of the update area for the current frame and the target frame are partially refreshed.

[0165] It is worth noting that the target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0166] In combination with the above-mentioned embodiments, it can be known that the rendering result of the target frame is stored in the DDR, and the DPU only needs to read the rendering result for the current frame and the update area of ​​the target frame from the DDR, and does not need to read the rendering result of the entire frame of the target frame from the DDR.

[0167] It can be seen that the process of partially refreshing the update area of ​​the current frame and the target frame based on the dirty area information and the target layer and the rendering results of the update area of ​​the current frame and the target frame includes: the DPU reads the rendering results of the update area of ​​the current frame and the target frame from the DDR. This reduces the load of DDR reading, and in the process of MIPI transmission, only the rendering results of the update area of ​​the target frame need to be transmitted, thereby achieving energy saving.

[0168] At the same time, asynchronous processing is adopted to synchronize the synthesis and display process with the rendering process, which further shortens the response time and improves the smoothness of screen refresh.

[0169] Based on the foregoing, it can be known that the SurfaceFlinger synthesis and display delivery process is different from the Client's SurfaceFlinger synthesis and display delivery process and the Device's SurfaceFlinger synthesis and display delivery process according to the synthesis hardware. In this embodiment, the time required for the synthesis and display delivery process is less than the time required for the rendering thread. On the other hand, if the Client's SurfaceFlinger synthesis and display delivery process is to be adopted, a second sub-thread will need the GPU to perform synthesis. At this time, the GPU is performing rendering, and the second sub-thread still needs to wait for the GPU rendering to complete before performing layer synthesis. Therefore, in this embodiment, the Device's SurfaceFlinger synthesis and display delivery process is adopted to ensure that synthesis and display delivery and rendering are performed simultaneously.

[0170] The following further describes the specific steps of the Device's SurfaceFlinger synthesis and display process.

[0171] For the SurfaceFlinger synthesis and display process of the Device, the target layer is synthesized, that is, the corresponding full-screen display synthesis. Furthermore, the second sub-thread in the synthesis and display process is no different from the display delivery method in the related technology. SurfaceFlinger first obtains the target layer and synchronizes the target layer to the HWC. The HWC obtains the full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes the full-screen configuration information to the DPU.

[0172] In other words, at this time, the terminal displays according to the full-screen configuration. Those skilled in the art will know that when the full-screen display is performed, the HWC can obtain the full-screen configuration information from the upper layer of the system, which will not be repeated here.

[0173] That is, in the step of synchronizing the target layer with the second sub-thread, the second sub-thread performs the following operations:

[0174] In response to the synthesis display request, the target layer is synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the target layer to HWC;

[0175] HWC obtains full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes it to DPU;

[0176] The above is the specific process of the SurfaceFlinger synthesis and display sending process of the Device in the embodiment of the present application. Through this method, the DPU can obtain the target layer and full-screen configuration information, and execute the DPU display sending described later.

[0177] Regarding the DPU described later, in the embodiment of the present application, after the synthesis and display synchronization is completed, the DPU obtains the target layer and full-screen configuration information, and waits for the GPU rendering to end.

[0178] In summary, for synthesis and display, in the process of transmitting the target layer as a whole, the configuration information of the next layer is informed at the same time. For full-screen configuration, when HWC synchronizes the target layer, it also transmits the full-screen configuration information. After the first child thread generates a rendering notification, it moves based on the user control object. There is a difference between the current frame and the target frame, that is, there is dirty area information. At this time, the DPU performs a partial refresh. In order to achieve a partial refresh, the current configuration information (that is, the full-screen configuration information) transmitted by the upper layer needs to be modified to the dirty area information. In other words, the process of partially refreshing the update area of ​​the current frame and the target frame based on the dirty area information and the target layer, and the rendering results of the update area of ​​the current frame and the target frame includes:

[0179] The DPU modifies the full-screen configuration information into dirty area information.

[0180] It can be understood that at this time, the DPU performs a partial refresh of the update area of ​​the current frame and the target frame according to the dirty area information, based on the rendering results of the target layer, the update area of ​​the current frame and the target frame.

[0181] When the refresh is completed, the DPU can send it to the display using the MIPI interface.

[0182] Continuing with the aforementioned shooting competitive game as an example, at this time the current player is waiting for other players to finish the game, that is, the target frame is consistent with the current frame. For this scenario, in order to reduce invalid load and further reduce power consumption, the mobile phone can directly generate a display completion mark when the dirty area information is empty. The terminal responds to the display completion mark and the screen displays the current frame. In other words, the DPU does not execute the aforementioned refresh process and the MIPI interface display process.

[0183] Different from the previous embodiment, in this embodiment, after the display is completed, the dirty area information is directly transmitted to the DPU, and the HWC does not obtain the dirty area information. Therefore, the DPU executes the step of determining whether there is an update area for the current frame and the target frame based on the dirty area information.

[0184] That is, after the CPU completes rendering and outputs the dirty area information, the dirty area information is synchronized to the DPU, and the DPU then executes the step of determining whether there is an update area for the current frame and the target frame based on the dirty area information.

[0185] Based on the above embodiments, it can be known that the process of transmitting the target layer to the DPU and the process of generating the dirty area information are inconsistent. In order to make the correspondence between the layer and the dirty area information exist, the dirty area information also includes the identifier corresponding to the target frame.

[0186] Please refer to Fig.10 .Assuming that a refresh request for the target frame is generated, the main thread of the mobile phone first obtains the refresh request for the target frame, and the main thread assigns the refresh request for the target frame to the rendering thread. At this time, two sub-threads are set under the rendering thread, corresponding to the first sub-thread and the second sub-thread. Under the first sub-thread, the GPU performs rendering, and under the second sub-thread, the Device's SurfaceFlinger full-frame synthesis and display process is executed, that is, the terminal synchronizes the target layer to SurfaceFlinger first, SurfaceFlinger synchronizes the target layer to HWC, and HWC synchronizes the target layer and full-screen configuration information. At this time, HWC synchronizes the layer synchronization and full-screen configuration information to DPU.

[0187] When rendering is completed, it is determined whether there is an update area for the current frame and the target frame based on the dirty area information.

[0188] If so, the DPU configuration information is modified to dirty area information, and the full-screen configuration information is modified to dirty area information. The rendering result corresponding to the dirty area information is read from DDR and refreshed. Otherwise, the display completion mark is directly generated. Please refer to the above for the specific steps, which will not be repeated here.

[0189] Please refer to Fig.11 , Fig.11 A flow chart of a dynamic area refresh method provided in an embodiment of the present application, applied to a terminal, specifically includes the following steps:

[0190] S101: In response to a refresh request for a target frame, render the target frame to generate a rendering result of the target frame, and compare the target frame with the current frame to generate dirty area information;

[0191] S102: Determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0192] S103: If there are update areas for the current frame and the target frame, perform partial refresh according to the update areas for the current frame and the target frame;

[0193] The target frame is the next frame of the current frame.

[0194] One possible approach is that the terminal includes a display processor DPU,

[0195] The steps of performing local refresh according to the update area of ​​the current frame and the target frame include:

[0196] The dirty area information and the target layer are synchronized to the display processor DPU. The display processor DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering results of the update area for the current frame and the target frame, a local refresh is performed according to the update area for the current frame and the target frame.

[0197] The target layer is generated by the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0198] One possible way is that the terminal uses SurfaceFlinger and hardware synthesizer HWC to send the display;

[0199] The step of determining whether there is an update area for the current frame and the target frame based on the dirty area information includes:

[0200] In response to the synthesis display request, the dirty area information and the target layer are synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the dirty area information and the target layer to the hardware synthesizer HWC;

[0201] The hardware synthesizer HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information;

[0202] The synthetic display request is generated after the refresh request for the target frame is generated.

[0203] One possible approach is that if there is no update region for the current frame and the target frame, the method further includes:

[0204] A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

[0205] One possible way is that the rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, the dirty area information and the target layer are synchronized to the display processor DPU, the display processor DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering result of the update area of ​​the current frame and the target frame, a process of performing a partial refresh according to the update area of ​​the current frame and the target frame includes:

[0206] The display processor DPU reads rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

[0207] Please refer to Fig.12 , the present application provides an embodiment of another dynamic area refresh method, which is applied to a terminal and executed in a rendering thread, and the rendering thread includes: a first sub-thread and a second sub-thread;

[0208] S201: In response to a refresh request for a target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer;

[0209] S202: In response to a refresh request for a target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer;

[0210] S203: the second sub-thread obtains dirty area information in response to the rendering completion notification, and determines whether there is an update area for the current frame and the target frame based on the dirty area information;

[0211] S204: If there are update regions for the current frame and the target frame, locally refresh the update regions for the current frame and the target frame based on the dirty region information, the target layer, and the rendering results of the update regions for the current frame and the target frame;

[0212] The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0213] One possible way is that the terminal uses SurfaceFlinger, hardware synthesizer HWC and display processor DPU to send display;

[0214] In the step of synchronizing the target layer by the second sub-thread, the second sub-thread performs the following operations:

[0215] In response to the synthesis display request, the target layer is synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the target layer to the hardware synthesizer HWC;

[0216] The hardware synthesizer HWC obtains the full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes it to the display processor DPU;

[0217] The composite display request is generated after the refresh request for the target frame is generated.

[0218] One possible approach is that the process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the synthesized target layer, and the rendering results of the update area for the current frame and the target frame includes:

[0219] The display processor DPU modifies the full-screen configuration information into dirty area information.

[0220] One possible approach is that if there is no update region for the current frame and the target frame, the method further includes:

[0221] A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

[0222] One possible approach is that the rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, and the process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering result for the update area for the current frame and the target frame includes:

[0223] The display processor DPU reads rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

[0224] One possible approach is that the dirty region information includes an identifier corresponding to the target frame.

[0225] Please refer to Fig.13 , Fig.13 A structural diagram of a dynamic area refresh device provided in an embodiment of the present application, applied to a terminal, specifically includes the following steps:

[0226] A first response module: used for responding to a refresh request for a target frame, rendering the target frame to generate a rendering result of the target frame, and comparing the target frame with the current frame to generate dirty area information;

[0227] The first determination module is used to determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0228] The first execution module: if there are update areas for the current frame and the target frame, then locally refresh according to the update areas for the current frame and the target frame;

[0229] The target frame is the next frame of the current frame.

[0230] One possible method is: the terminal uses SurfaceFlinger and hardware synthesizer HWC for display;

[0231] The first determination module is specifically used to: in response to the synthesis display request, synchronize the dirty area information and the target layer to SurfaceFlinger, and SurfaceFlinger synchronizes the dirty area information and the target layer to the hardware synthesizer HWC;

[0232] The hardware synthesizer HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information;

[0233] Among them, the target layer is generated through the target frame, and the synthetic display request is generated after the refresh request for the target frame is generated.

[0234] In one possible manner, the first execution module is specifically used for:

[0235] The dirty area information and the target layer are synchronized to the display processor DPU. The display processor DPU is configured according to the dirty area information, and based on the dirty area information, the target layer, and the rendering results of the update area for the current frame and the target frame, a local refresh is performed according to the update area for the current frame and the target frame.

[0236] The rendering result of the target frame includes: rendering results for the current frame and the update area of ​​the target frame.

[0237] Please refer to Fig.14 , Fig.14 Another structural diagram of a dynamic area refresh device is provided for the present application, which is applied to a terminal and executed in a rendering thread, and the rendering thread includes: a first sub-thread and a second sub-thread;

[0238] The second response module is used to respond to the refresh request for the target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer;

[0239] Sending module: used for the first sub-thread to generate a rendering completion notification, which is completed after the target frame is rendered, and sends the rendering completion notification to the second sub-thread;

[0240] A second determination module: used for the second sub-thread to obtain dirty area information in response to the rendering completion notification, and determine whether there is an update area for the current frame and the target frame based on the dirty area information;

[0241] The second execution module is used for partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer and the rendering results of the update area for the current frame and the target frame if there is an update area for the current frame and the target frame;

[0242] The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering result for the current frame and the update area of ​​the target frame.

[0243] One possible approach is that the terminal includes SurfaceFlinger, a hardware compositor HWC, and a display processor DPU;

[0244] The second response module is specifically used for:

[0245] In response to the synthesis display request, the target layer is synchronized to SurfaceFlinger, and SurfaceFlinger synchronizes the target layer to the hardware synthesizer HWC;

[0246] The hardware synthesizer HWC obtains the full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes it to the display processor DPU;

[0247] The composite display request is generated after the refresh request for the target frame is generated.

[0248] The above describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0249] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0250] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0251] It should be noted that the terminals involved in the embodiments of the present invention may include but are not limited to personal computers (Personal Computer; hereinafter referred to as: PC), personal digital assistants (Personal Digital Assistant; hereinafter referred to as: PDA), wireless handheld devices, tablet computers (Tablet Computer), mobile phones, MP3 players, MP4 players, etc.

[0252] The application embodiment provides an electronic device, which may be a terminal device or a circuit device built into the terminal device. The electronic device may be used to execute the functions / steps in the above method embodiment.

[0253] An embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the functions / steps in the above method embodiment.

[0254] The embodiment of the present application also provides a computer program product comprising instructions, and when the computer program product is run on a computer or any at least one processor, the computer executes the functions / steps in the above method embodiment.

[0255] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0256] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0257] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0258] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0259] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A dynamic area refresh method, characterized in that: Applied to the terminal, specifically including the following steps: In response to a refresh request for a target frame, rendering the target frame to generate a rendering result of the target frame, and comparing the target frame with a current frame to generate dirty region information; Determine whether there is an update area for the current frame and the target frame based on the dirty area information; If the update area for the current frame and the target frame exists, locally refreshing is performed according to the update area for the current frame and the target frame; The target frame is the next frame of the current frame.

2. The method according to claim 1, characterized in that: The terminal comprises a display processor DPU, The step of performing local refreshing according to the update area for the current frame and the target frame comprises: Synchronize the dirty area information and the target layer to the display processor DPU, the display processor DPU is configured according to the dirty area information, and performs the local refresh according to the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering results for the update area for the current frame and the target frame; The target layer is generated by the target frame, and the rendering result of the target frame includes: the rendering results for the current frame and the update area of ​​the target frame.

3. The method according to claim 2, characterized in that The terminal uses SurfaceFlinger and hardware synthesizer HWC for display; The step of determining whether there is an update area for the current frame and the target frame based on the dirty area information comprises: In response to the synthesis display request, the dirty area information and the target layer are synchronized to the SurfaceFlinger, and the SurfaceFlinger synchronizes the dirty area information and the target layer to the hardware synthesizer HWC; The hardware synthesizer HWC determines whether there is an update area for the current frame and the target frame based on the dirty area information; The synthetic display request is generated after the refresh request for the target frame is generated.

4. The method according to claim 3, characterized in that: If the update area for the current frame and the target frame does not exist, the method further includes: A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

5. The method according to claim 3, characterized in that: The rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, the dirty area information and the target layer are synchronized to the display processor DPU, the display processor DPU is configured according to the dirty area information, and the process of performing local refresh according to the update area of ​​the current frame and the target frame based on the dirty area information, the target layer, and the rendering result of the update area of ​​the current frame and the target frame includes: The display processor DPU reads the rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

6. A dynamic area refresh method, characterized in that: Applied to a terminal and executed in a rendering thread, the rendering thread includes: a first sub-thread and a second sub-thread; In response to a refresh request for a target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer; The first sub-thread generates a rendering completion notification, wherein the rendering completion is completed after rendering the target frame, and sends the rendering completion notification to the second sub-thread; The second sub-thread obtains the dirty area information in response to the rendering completion notification, and determines whether there is an update area for the current frame and the target frame based on the dirty area information; If the update area for the current frame and the target frame exists, locally refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering result of the update area for the current frame and the target frame; The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering results for the update area of ​​the current frame and the target frame.

7. The method according to claim 6, characterized in that The terminal uses SurfaceFlinger, hardware synthesizer HWC and display processor DPU for display; In the step of synchronizing the target layer by the second sub-thread, the second sub-thread performs the following operations: In response to the synthesis display request, the target layer is synchronized to the SurfaceFlinger, and the SurfaceFlinger synchronizes the target layer to the hardware synthesizer HWC; The hardware synthesizer HWC obtains full-screen configuration information, synchronizes the target layer to the full-screen configuration information and synchronizes it to the display processor DPU; The synthetic display request is generated after the refresh request for the target frame is generated.

8. The method according to claim 7, characterized in that The process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer, and the rendering results for the update area for the current frame and the target frame includes: The display processor DPU modifies the full-screen configuration information into the dirty area information.

9. The method according to claim 6, characterized in that If the update area for the current frame and the target frame does not exist, the method further includes: A display transmission completion mark is generated, and in response to the display transmission completion mark, the terminal screen reuses the current frame.

10. The method according to claim 6, characterized in that The rendering result of the target frame is stored in a double data rate synchronous dynamic random access memory DDR, and the process of partially refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer and the rendering result for the update area for the current frame and the target frame includes: The display processor DPU reads the rendering results of the update area for the current frame and the target frame from the double data rate synchronous dynamic random access memory DDR.

11. The method according to any one of claims 6 to 10, characterized in that: The dirty region information includes an identifier corresponding to the target frame.

12. A dynamic area refresh device, characterized in that: Applied to terminals, including: A first response module: configured to respond to a refresh request for a target frame, render the target frame to generate a rendering result of the target frame, and compare the target frame with the current frame to generate dirty area information; A first determination module: used for determining whether there is an update area for the current frame and the target frame based on the dirty area information; The first execution module: if the update area for the current frame and the target frame exists, perform partial refresh according to the update area for the current frame and the target frame; The target frame is the next frame of the current frame.

13. A dynamic area refresh device, characterized in that: Applied to a terminal and executed in a rendering thread, the rendering thread includes: a first sub-thread and a second sub-thread; The second response module is used to respond to the refresh request for the target frame, the first sub-thread renders the target frame to generate a rendering result of the target frame, and compares the target frame with the current frame to generate dirty area information, and the second sub-thread synchronizes the target layer; Sending module: used for the first sub-thread to generate a rendering completion notification, the rendering completion is completed after the target frame is rendered, and send the rendering completion notification to the second sub-thread; A second determination module: configured for the second sub-thread to obtain the dirty area information in response to the rendering completion notification, and determine whether there is an update area for the current frame and the target frame based on the dirty area information; A second execution module: if the update area for the current frame and the target frame exists, locally refreshing the update area for the current frame and the target frame based on the dirty area information, the target layer and the rendering result of the update area for the current frame and the target frame; The target frame is the next frame of the current frame, the target layer is determined based on the target frame, and the rendering result of the target frame includes: the rendering results for the update area of ​​the current frame and the target frame.

14. A terminal, characterized in that: include: at least one processor; and at least one memory in communication with the processor, wherein: The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 5 or 6 to 11 by calling the program instructions.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method according to any one of claims 1 to 5 or 6 to 11.

Images