Memory usage method and electronic device

Abstract

The application provides a memory usage method and an electronic device. The memory usage method comprises: rendering a first image frame in a first framebuffer, the first image frame comprising a non-rectangular region corresponding to the shape of a display screen, the non-rectangular region comprising a first part and a second part, the second part being a region of the non-rectangular region other than the first part; after the first image frame is rendered, copying the first part to a first memory so that the first memory obtains a first copied part, and copying the second part to a second framebuffer so that the second framebuffer obtains a second copied part; copying the first copied part in the first memory to the second framebuffer so that the second framebuffer obtains a third copied part, wherein the second copied part and the third copied part form a target image frame corresponding to the shape of the display screen; and displaying the target image frame on the display screen based on the second framebuffer. In this way, based on the scheme provided by the application, the electronic device can save the resources of the on-chip memory while ensuring the frame rate performance during the process of displaying the target image frame on the non-rectangular display screen.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and particularly relates to a memory usage method and an electronic device. BACKGROUND

[0002] With the rapid development of display screen technology, a non-rectangular display screen is derived. The non-rectangular display screen can be applied to electronic devices such as wearable devices.

[0003] In the case that the electronic device has a non-rectangular display screen, the electronic device stores an image frame in a frame buffer of an on-chip memory, and displays the image frame on the non-rectangular display screen. The frame buffer is usually designed to be rectangular, and each frame buffer is usually used to accommodate one image frame. If the pixels in the image frame are completely displayed, the image frame is displayed as a rectangle. However, the non-rectangular display screen cannot completely display the rectangular image frame.

[0004] Therefore, the image frame includes part of the pixels that cannot be displayed on the non-rectangular display screen. These pixels occupy the resources of the on-chip memory, resulting in the waste of the resources of the on-chip memory. SUMMARY

[0005] The embodiments of the present application provide a memory usage method and an electronic device, which can solve the problem of the waste of the resources of the on-chip memory in the electronic device with a non-rectangular display screen.

[0006] In a first aspect, an embodiment of the present application provides a memory usage method applied to an electronic device, the electronic device comprising a display screen, a memory and a processor, the display screen, the memory and the processor being coupled, wherein the shape of the display screen comprises a shape other than a rectangle, the memory comprising a first framebuffer, a second framebuffer and a first memory, the first framebuffer and the second framebuffer being arranged on a chip of the processor, and the first memory being arranged outside the chip of the processor and connected to the processor, the method comprising: rendering a first image frame in the first framebuffer, the first image frame comprising a non-rectangular region corresponding to the shape of the display screen, the non-rectangular region comprising a first part and a second part, and the second part being a region of the non-rectangular region other than the first part; after the rendering of the first image frame is completed, copying the first part to the first memory so that the first memory obtains a first copied part, and copying the second part to the second framebuffer so that the second framebuffer obtains a second copied part; copying the first copied part in the first memory to the second framebuffer so that the second framebuffer obtains a third copied part, wherein the second copied part and the third copied part form a target image frame corresponding to the shape of the display screen; and based on the second framebuffer, sending the target image frame for display so as to display the target image frame on the display screen.

[0007] The method provided by the present application can be applied to an electronic device with a non-rectangular display screen, and the method can copy the non-rectangular region corresponding to the shape of the display screen in the image frame in a region-by-region manner. Thus, compared with the conventional copying manner of the rectangular image frame, the present application can save the resources of the on-chip memory while ensuring the frame rate performance.

[0008] In an implementation manner, before rendering the first image frame in the first framebuffer, the method further comprises: modifying the memory capacity of the second framebuffer so as to match the memory capacity with the memory occupied by the non-rectangular region. By using the implementation manner, the electronic device can dynamically allocate the memory capacity of the second framebuffer so as to adapt the display requirement of the non-rectangular display screen. Moreover, modifying the memory capacity of the second framebuffer can also reduce the system cost and power consumption.

[0009] In an implementation, the memory capacity of the second framebuffer is modified, including: modifying the memory capacity based on a graphics memory management unit (GFX MMU). With this implementation, a specific way of modifying the memory capacity of the second framebuffer is shown, so that the electronic device can dynamically allocate the memory capacity of the second framebuffer to adapt the display requirement of the non-rectangular display screen. Moreover, modifying the memory capacity of the second framebuffer can also reduce the system cost and power consumption.

[0010] In an implementation, after the first image frame is rendered, the first part is copied to the first memory so that the first memory obtains the first copied part, and the second part is copied to the second framebuffer so that the second framebuffer obtains the second copied part, including: copying the first part to the first memory based on a first channel so that the first memory obtains the first copied part, and copying the second part to the second framebuffer based on a second channel so that the second framebuffer obtains the second copied part, wherein the first channel and the second channel have different copying devices. With this implementation, the electronic device can copy the image frame based on different channels, so that the time-consuming length of the copying process is reduced, and the frame rate performance of the electronic device is not affected.

[0011] In an implementation, the first copied part in the first memory is copied to the second framebuffer so that the second framebuffer obtains a third copied part, including: copying the first copied part to the second framebuffer based on a third channel so that the second framebuffer obtains the third copied part, wherein the third channel and the first channel have different copying devices. With this implementation, the electronic device can copy the image frame based on different channels, so that the time-consuming length of the copying process is reduced, and the frame rate performance of the electronic device is not affected.

[0012] In an implementation, the copy device corresponding to the first channel includes a graphics processing unit (GPU), and the copy device corresponding to the second channel includes a direct memory access two-dimensional graphics accelerator (DMA2D), the copying the first part based on the first channel to the first memory to cause the first memory to obtain the first copy part, and the copying the second part based on the second channel to the second framebuffer to cause the second framebuffer to obtain the second copy part, includes: copying the first part based on the GPU corresponding to the first channel to the first memory to cause the first memory to obtain the first copy part, and copying the second part based on the DMA2D corresponding to the second channel to the second framebuffer to cause the second framebuffer to obtain the second copy part. With the implementation, the specific copy device is shown, so that the electronic device can copy the image frame based on different channels to reduce the time length of the copying process and will not affect the frame rate performance of the electronic device.

[0013] In an implementation, the copy device corresponding to the third channel includes a DMA2D, the copying the first copy part based on the third channel to the second framebuffer to cause the second framebuffer to obtain the third copy part, includes: copying the first copy part based on the DMA2D corresponding to the third channel to the second framebuffer to cause the second framebuffer to obtain the third copy part. With the implementation, the specific copy device is shown, so that the electronic device can copy the image frame based on different channels to reduce the time length of the copying process and will not affect the frame rate performance of the electronic device.

[0014] In an implementation, after the copying the first part to the first memory to cause the first memory to obtain the first copy part, and the copying the second part to the second framebuffer to cause the second framebuffer to obtain the second copy part, further includes: rendering a second image frame in the first framebuffer, the second image frame being a next frame image frame of the first image frame. With the implementation, after the first image frame in the first framebuffer has been completely copied, the first framebuffer can render the next frame image frame, so as not to affect the copying time length and to ensure the frame rate performance of the electronic device.

[0015] In an implementation, the rendering the first image frame in the first framebuffer includes: rendering, by a kernel layer, the first image frame in the first framebuffer. With the implementation, the specific software architecture for rendering the first image frame is shown.

[0016] In an implementation, copying the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copied part, and copying the second part to the second framebuffer based on the DMA2D corresponding to the second channel to make the second framebuffer obtain the second copied part, comprises: copying, by the kernel layer, the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copied part, and copying, by the kernel layer, the second part to the second framebuffer based on the DMA2D corresponding to the second channel to make the second framebuffer obtain the second copied part. With this implementation, the kernel layer can copy the image frame based on different channels, thus reducing the time length of the copying process and not affecting the frame rate performance of the electronic device.

[0017] In an implementation, copying the first copied part to the second framebuffer based on the DMA2D corresponding to the third channel to make the second framebuffer obtain the third copied part, comprises: copying, by the kernel layer, the first copied part to the second framebuffer based on the DMA2D corresponding to the third channel to make the second framebuffer obtain the third copied part. With this implementation, the kernel layer can copy the image frame based on different channels, thus reducing the time length of the copying process and not affecting the frame rate performance of the electronic device.

[0018] In an implementation, sending the target image frame for display based on the second framebuffer to display the target image frame on the display screen, comprises: sending, by the kernel layer, the target image frame for display based on the second framebuffer to display the target image frame on the display screen. With this implementation, the specific software architecture for sending the target image frame for display is shown.

[0019] In an implementation, rendering the second image frame in the first framebuffer, comprises: rendering, by the kernel layer, the second image frame in the first framebuffer. With this implementation, the specific software architecture for rendering the second image frame is shown, so that after the first image frame in the first framebuffer has been completely copied, the first framebuffer can perform rendering of the next frame of image frame, without affecting the copying time length, and the frame rate performance of the electronic device can be ensured.

[0020] In an implementation, the kernel layer comprises a first thread, and the kernel layer renders the first image frame in the first framebuffer, comprising: calling, by the first thread, the GPU to render the first image frame in the first framebuffer. With this implementation, the specific software architecture for rendering the first image frame is shown.

[0021] In an implementation, the kernel layer further comprises a second thread, the kernel layer copies the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copied part, and the kernel layer copies the second part to the second framebuffer based on the DMA2D corresponding to the second channel to make the second framebuffer obtain the second copied part, and the copying the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copied part and the copying the second part to the second framebuffer based on the DMA2D corresponding to the second channel are performed by the second thread. With this implementation, the second thread can copy the image frame based on different channels, thus reducing the time length of the copying process and not affecting the frame rate performance of the electronic device.

[0022] In an implementation, the kernel layer copies the first copied part to the second framebuffer based on the DMA2D corresponding to the third channel to make the second framebuffer obtain the third copied part, and the copying the first copied part to the second framebuffer based on the DMA2D corresponding to the third channel to make the second framebuffer obtain the third copied part is performed by the second thread. With this implementation, the second thread can copy the image frame based on different channels, thus reducing the time length of the copying process and not affecting the frame rate performance of the electronic device.

[0023] In an implementation, the kernel layer composes the target image frame based on the second framebuffer to display the target image frame on the display screen, and the composing the target image frame based on the second framebuffer to display the target image frame on the display screen is performed by the second thread. With this implementation, the specific software architecture for composing the target image frame is shown.

[0024] In an implementation, the kernel layer renders the second image frame in the first framebuffer, and the rendering the second image frame in the first framebuffer is performed by the second thread sending a rendering instruction to the first thread to make the first thread render the second image frame in the first framebuffer in response to the rendering instruction. With this implementation, the specific software architecture for rendering the second image frame is shown, so that the first framebuffer can perform the rendering of the next image frame after the first image frame in the first framebuffer has been completely copied, thus ensuring the frame rate performance of the electronic device without affecting the copying time length.

[0025] In a second aspect, an embodiment of the present application provides a memory usage apparatus applied to an electronic device, the electronic device comprising a display screen, a memory and a processor, the display screen, the memory and the processor being coupled, wherein the shape of the display screen comprises a shape other than a rectangle, the memory comprising a first framebuffer, a second framebuffer and a first memory, the first framebuffer and the second framebuffer being arranged on a chip of the processor, and the first memory being arranged outside the chip of the processor and connected to the processor, the apparatus comprising: a rendering module, configured to render a first image frame in the first framebuffer, the first image frame comprising a non-rectangular region corresponding to the shape of the display screen, the non-rectangular region comprising a first part and a second part, and the second part being a region of the non-rectangular region other than the first part; a first copying module, configured to copy the first part to the first memory after the first image frame is rendered, so that the first memory obtains a first copied part, and copy the second part to the second framebuffer, so that the second framebuffer obtains a second copied part; a second copying module, configured to copy the first copied part in the first memory to the second framebuffer, so that the second framebuffer obtains a third copied part, wherein the second copied part and the third copied part form a target image frame corresponding to the shape of the display screen; and a display sending module, configured to send the target image frame based on the second framebuffer, so as to display the target image frame on the display screen.

[0026] The apparatus provided by the present application can be applied to an electronic device with a non-rectangular display screen, and the apparatus can copy a non-rectangular region corresponding to the shape of the display screen in an image frame in a region-by-region manner. Thus, compared with a conventional copying manner of a rectangular image frame, the present application can save the resources of on-chip memory while ensuring the frame rate performance.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a display screen, a memory and a processor, the display screen, the memory and the processor being coupled, wherein the shape of the display screen comprises a shape other than a rectangle, and the memory stores computer program code, the computer program code comprising computer instructions, when the computer instructions are executed by the processor, causing the electronic device to perform the memory usage method in the first aspect and any implementation manner.

[0028] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium comprising computer instructions, when the computer instructions are run on an electronic device, causing the electronic device to perform the memory usage method in the first aspect and any implementation manner.

[0029] In a fifth aspect, the embodiments of the present application provide a computer program product, which, when running on an electronic device, causes the electronic device to perform the memory usage method in the first aspect above and any implementation manner thereof.

[0030] It can be understood that the technical solutions provided by the third aspect to the fifth aspect above can achieve the beneficial effects as described in the first aspect and any implementation manner thereof, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0032] Figure 1 is a schematic diagram of the memory usage manner in the process of displaying an image frame by a wearable device;

[0033] Figure 2 is a schematic diagram of the hardware structure of an electronic device provided by the embodiments of the present application;

[0034] Figure 3 is a schematic diagram of the software structure of an electronic device provided by the embodiments of the present application;

[0035] Figure 4 is a first scenario diagram of displaying an image frame by an electronic device provided by the embodiments of the present application;

[0036] Figure 5 is a first flowchart of a memory usage method provided by the embodiments of the present application;

[0037] Figure 6 is a first interaction diagram of a memory usage method provided by the embodiments of the present application;

[0038] Figure 7 is a schematic diagram of a non-rectangular region provided by the embodiments of the present application;

[0039] Figure 8 is a schematic diagram of the memory capacity of a modified second framebuffer provided by the embodiments of the present application;

[0040] Figure 9 is a schematic diagram of the copying process of a first image frame provided by the embodiments of the present application;

[0041] Figure 10 is a second scenario diagram of displaying an image frame by an electronic device provided by the embodiments of the present application;

[0042] Figure 11is a second flow chart of the memory using method provided by the embodiment of the present application;

[0043] Figure 12 is a second interaction schematic diagram of the memory using method provided by the embodiment of the present application;

[0044] Figure 13 is a third flow chart of the memory using method provided by the embodiment of the present application;

[0045] Figure 14 is a structural schematic diagram of the memory using device provided by the embodiment of the present application;

[0046] Figure 15 is a structural schematic diagram of the memory using device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0048] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more. "First", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0049] It should be noted that in the present application, "exemplary" or "for example" is used to mean example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0050] The application scenario of the embodiments of the present application will be described first below with reference to the drawings.

[0051] Memory, also known as Random Access Memory (RAM), is a type of hardware used in electronic devices to temporarily store data and programs. Memory can store various types of data, including text, numbers, images, audio, and video.

[0052] Memory includes on-chip RAM and off-chip RAM.

[0053] On-chip RAM is a memory integrated inside the Central Processing Unit (CPU) chip, located on the same silicon die as the processor core. This means that on-chip RAM is closer to the processor core, allowing the processor core to access it quickly with low latency.

[0054] Off-chip RAM is a memory located outside the processor chip, such as Dynamic Random Access Memory (DRAM) or Static Random Access Memory (SRAM). Off-chip RAM needs to be connected to the processor chip through a slot or interface on the motherboard. This means that off-chip RAM is farther away from the processor core, resulting in slower access speed and higher latency.

[0055] Electronic devices involve the use of memory during the display of image frames.

[0056] Electronic devices include but are not limited to mobile phones, tablets, personal computers, Personal Digital Assistants (PDAs), workstation devices, large-screen devices (such as smart screens, smart TVs, etc.), wearable devices (such as smart bracelets, smart watches), handheld game consoles, home game consoles, Virtual Reality (VR) devices, Augmented Reality (AR) devices, mixed reality devices, and in-vehicle intelligent terminals. This application does not limit the specific technology and specific device form of the electronic device. The electronic device involved in this application can be equipped with Harmony or other operating systems, which are not limited in this application.

[0057] This application takes the electronic device as an example of a wearable device.

[0058] Figure 1is a schematic diagram of a memory usage manner in a wearable device during displaying an image frame.

[0059] As shown in Figure 1 In the wearable device, an application can cooperate with multiple threads such as an interface refresh thread (also referred to as a User Interface (UI) thread), a Display thread, and the like to realize drawing and displaying of an image frame. The interface refresh thread is used to process events related to the UI interface, for example, updating and redrawing of the UI interface. The Display thread is used to display the image frame drawn by the interface refresh thread.

[0060] Specifically, the interface refresh thread can first send the image frame to be drawn to a Graphics Processing Unit (GPU), and further, the GPU draws the image frame based on a double framebuffer architecture in an on-chip memory.

[0061] The double framebuffer architecture can provide two framebuffers. The two framebuffers include, for example, a first framebuffer and a second framebuffer, the first framebuffer can be used to render the image frame, and the second framebuffer can be used to display the image frame.

[0062] It should be noted that each framebuffer can accommodate one image frame. For example, the image frame corresponds to a width pixel value of 466 and a height pixel value of 466, that is, the image frame corresponds to a square (i.e., a special rectangle) display area, the display area has 466*466 (i.e., 217156) pixels, and can use an RGB color model. In the RGB color model, each pixel is composed of three color channels of red (R), green (G), and blue (B), and each color channel needs to occupy one byte. In this way, the image frame accommodated in each framebuffer occupies 466*466*3 (i.e., 651468) bytes, about 650 kilobytes (KB).

[0063] Data copying between the two framebuffers can be based on two independent General Purpose Direct Memory Access (GPDMA) devices. For example, the 650 KB image frame in one framebuffer can be copied to another framebuffer based on two GPDMA.

[0064] Direct Memory Access (DMA) is a hardware feature that allows a part of hardware subsystem to access memory directly without the intervention of a Central Processing Unit (CPU), thus, the data transfer process based on DMA has a high transfer speed and does not occupy CPU resources, which can improve the overall system efficiency.

[0065] It should be noted that, in the above process, the intervention of the CPU means that the CPU does not directly intervene in the access and copying process of the memory, actually, the CPU needs to manage the DMA.

[0066] The GPDMA device is a hardware implementation form of DMA, which supports memory-to-memory transfer. That is, the GPDMA device can copy data between the first Framebuffer and the second Framebuffer. In the case of setting two GPDMA devices between the first Framebuffer and the second Framebuffer, the two GPDMA devices can transfer data in the same direction or in the opposite direction, for example, one of the GPDMA devices can transfer a part of the image frame from the first Framebuffer to the second Framebuffer, and the other GPDMA device can transfer another part of the image frame from the first Framebuffer to the second Framebuffer, in this way, the setting mode of the two-way GPDMA device can improve the data copying efficiency.

[0067] The GPU can first render the image frame in the first Framebuffer, after the rendering process in the first Framebuffer is completed, the two-way GPDMA device can copy the image frame in the first Framebuffer in full to copy the image frame to the second Framebuffer. At this time, the first Framebuffer can render the next frame of image frame, and the second Framebuffer can display the copied image frame to the display screen. Wherein, the process of the two-way GPDMA copying the image frame in full between the first Framebuffer and the second Framebuffer takes about 1.8 milliseconds (ms).

[0068] It should be noted that, in order to meet the display requirements of most electronic devices, the Framebuffer is usually designed as a rectangle, and accordingly, the image frame contained in the Framebuffer will be displayed as a rectangle.

[0069] Since one framebuffer occupies 650KB of on-chip memory, two framebuffers occupy 1300KB of on-chip memory, which is equivalent to occupying 1.27 megabytes (M) of on-chip memory. In this way, the wearable device uses 1.27M of on-chip memory in the process of displaying an image frame, and the on-chip memory copy time reaches 1.8ms for each image frame copy process.

[0070] Some wearable devices use non-rectangular display screens, for example, circular display screens. In the case of a wearable device with a circular display screen, the circular display screen cannot completely display a rectangular image frame.

[0071] That is, the 1.27M of on-chip memory corresponding to the image frame is not completely displayed in the circular display screen, and the area of the image frame that cannot be displayed occupies the on-chip memory, forming data redundancy and causing waste of on-chip memory.

[0072] In a wearable device, the resource of on-chip memory is very valuable, so it is necessary to save memory in a wearable device, and the problem of waste of on-chip memory needs to be solved.

[0073] To solve this problem, an embodiment of the present application provides a memory usage method, which can be applied to an electronic device, for example, a wearable device, and the specific type of the electronic device is not limited in the present application.

[0074] Figure 2 is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present application.

[0075] As shown in Figure 2As shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 01, an antenna 02, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a Subscriber Identification Module (SIM) card interface 195, and a DMA device 196, etc. Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] The processor 110 can include one or more processing units, for example: the processor 110 can include an Application Processor (AP), a modem processor, a GPU, an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and / or a Neural-network Processing Unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0077] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc. These interfaces are used to connect with other components in the electronic device 100.

[0078] The controller can generate operation control signals according to the instruction operation code and the timing signal, and then control the process of obtaining and executing instructions.

[0079] The memory can be used to store computer executable program codes, and the executable program codes include instructions. The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function (for example, an application required by a sound playing function, an image playing function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.).

[0080] The memory can include an internal memory 121.

[0081] The internal memory 121 can include one or more Random Access Memories (RAMs) and one or more Non-Volatile Memories (NVMs).

[0082] The random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, such as the fifth generation of DDR SDRAM commonly referred to as DDR5 SDRAM), and the like.

[0083] It should be noted here that the internal memory 121 refers to the memory inside the electronic device 100, which includes not only the memory on the processor chip, but also the memory outside the processor chip, i.e., the on-chip memory and off-chip memory involved in the foregoing, and the internal memory 121 in the present application is not based on the division of the inside and outside of the processor chip.

[0084] The processor 110 can be connected to the off-chip memory through an interface therein.

[0085] In the embodiments of the present application, the internal memory 121 includes a first framebuffer, a second framebuffer, and a first memory. Among them, the first framebuffer and the second framebuffer are on-chip memories, and the first memory is an off-chip memory. That is, the first framebuffer and the second framebuffer are both arranged on the chip of the processor, and the first memory is arranged outside the chip of the processor and connected to the processor.

[0086] The non-volatile memory can include a disk storage device, a flash memory. The flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operation principle, and can include single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential order, and can include universal flash storage (UFS), embedded multimedia storage (eMMC), etc. according to the storage specification.

[0087] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (for example, machine instructions) of an operating system or other programs running, and can also be used to store data of users and application programs, and the like. The non-volatile memory can also store executable programs and store data of users and application programs, and the like, and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.

[0088] In the embodiment of the application, the code for implementing the memory usage method in the embodiment of the application can be stored on the non-volatile memory. In the process of running the application program by the electronic device 100, the electronic device 100 can load the executable code stored in the non-volatile memory to the random access memory.

[0089] The external memory interface 120 can be used to connect an external non-volatile memory, to realize the expansion of the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120, to realize the data storage function. The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, and the like.

[0090] It can be understood that the interface connection relationship between the modules shown in the embodiments of the application is only illustrative, and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the application, the electronic device 100 can also use different interface connection modes in the above embodiments, or a combination of multiple interface connection modes.

[0091] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive the charging input of the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142, and at the same time, can also supply power to the electronic device through the power management module 141.

[0092] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the memory 120, the display screen 194, the camera 193, and the wireless communication module 160.

[0093] The wireless communication function of the electronic device 100 can be implemented by the antenna 01, the antenna 02, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0094] The antenna 01 and the antenna 02 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 01 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0095] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 01, filter, amplify, etc. the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor and radiate the signals as electromagnetic waves through the antenna 01. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be disposed in the same device.

[0096] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A and the microphone 170B, etc.) or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In some other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as the mobile communication module 150 or other functional modules.

[0097] The wireless communication module 160 can provide solutions for wireless communication including Wireless Local Area Networks (WLAN) (e.g., Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR) technology, etc. The wireless communication module 160 receives electromagnetic waves via the antenna 02, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 02.

[0098] In some embodiments, the antenna 01 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 02 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology.

[0099] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0100] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be manufactured by using an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a Microled, a Micro-oled, a quantum dot light emitting diode (QLED), and the like. In some embodiments, the electronic device can include one or N display screens 194, where N is a positive integer greater than 1.

[0101] In some embodiments, the display screen 194 can be a non-rectangular display screen, such as a circular, elliptical, or rounded rectangular display screen. The shape of the display screen 194 is not limited in the embodiments of the present application.

[0102] In some embodiments, the electronic device 100 can display a UI based on the display functions provided by the GPU, the display screen 194, and the application processor.

[0103] In some embodiments, the electronic device 100 can call the GPU to render an image frame through an interface refresh thread.

[0104] The electronic device 100 can implement a photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0105] The ISP is configured to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert the electrical signal into an image visible to the naked eye. The ISP can also optimize algorithms for image noise, brightness, and skin color. The ISP can also optimize parameters such as exposure and color temperature of a shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0106] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard image signal in a format such as RGB, YUV, or the like. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than one.

[0107] The digital signal processor is configured to process digital signals, including digital image signals. For example, when the electronic device 100 is selecting a frequency, the digital signal processor is configured to perform a Fourier transform on the frequency energy, and the like.

[0108] The NPU is a neural-network (NN) computing processor that is configured to process input information quickly by emulating a biological neural network structure, such as by emulating the transmission mode between neurons in the human brain, and is further configured to learn continuously. The NPU can be used to implement intelligent cognitive applications of the electronic device 100, such as image recognition, facial recognition, speech recognition, text understanding, and the like.

[0109] The electronic device 100 can be configured to implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, the application processor, and the like. For example, the electronic device 100 can be configured to play music, record audio, and the like.

[0110] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194.

[0111] The gyroscope sensor 180B can be configured to determine the motion posture of the electronic device 100. In some embodiments, the gyroscope sensor 180B can be configured to determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for anti-shake photography. For example, when a shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and causes the lens to compensate for the shaking of the electronic device 100 by moving in the opposite direction, thereby achieving anti-shake.

[0112] The barometric pressure sensor 180C is used to measure barometric pressure. In some embodiments, the electronic device 100 calculates altitude, assists in positioning and navigation, using barometric pressure values measured by the barometric pressure sensor 180C.

[0113] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect opening and closing of a flip cover using the magnetic sensor 180D.

[0114] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (typically, three axes). The acceleration sensor 180E can detect the magnitude and direction of gravity when the electronic device 100 is stationary.

[0115] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focus when taking a picture.

[0116] The proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. The electronic device 100 emits infrared light outwardly using the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.

[0117] The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display 194 according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust white balance when taking a picture. The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locking, fingerprint picture taking, fingerprint call answering, and the like.

[0118] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 performs temperature handling strategies using temperature detected by the temperature sensor 180J.

[0119] Touch sensor 180K, also referred to as a "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also referred to as a "touch panel". The touch sensor 180K is configured to detect a touch operation acting on or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0120] The bone conduction sensor 180M can obtain a vibration signal.

[0121] The keys 190 include a power key, a volume key, and the like. The electronic device 100 can receive a key input and generate a key signal input related to user settings and function control of the electronic device 100.

[0122] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback.

[0123] The indicator 192 can be an indicator light, which can be used to indicate a charging state, a power change, and also can be used to indicate a message, a missed call, a notification, and the like.

[0124] The SIM card interface 195 is configured to connect a SIM card. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1.

[0125] The DMA device 196 is configured to copy data from one storage space to another storage space without CPU intervention, for example, copying data from one memory unit to another memory unit, or copying data from an external memory to an internal memory. The number of DMA devices 196 can be one or more, and one DMA device 196 can correspond to one or more DMA channels, and the DMA device 196 can control the data transmission process on each DMA channel based on a DMA controller.

[0126] In an embodiment of the present application, the DMA device 196 can be configured to copy at least part of an image frame from a first framebuffer to a second framebuffer. The present application does not limit the copy object corresponding to the DMA device 196.

[0127] In an embodiment of the present application, the DMA device 196 can be a direct memory access two-dimensional graphics accelerator (Direct Memory Access 2D, DMA2D). The present application does not limit the specific type of the DMA device 196.

[0128] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0129] Figure 3 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0130] like Figure 3 As shown, the layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0131] The application layer can include a series of application packages.

[0132] like Figure 3 As shown, the application package may include applications such as battery management, camera, gallery, calendar, call, map, navigation, music, video, and SMS.

[0133] In this embodiment, when the electronic device 100 is a wearable device, the application package may include, for example, communication applications (e.g., messaging applications, call applications), system-level applications (e.g., alarm clock applications, weather applications), interconnectivity applications (e.g., remote camera applications, find my phone applications), health applications (e.g., heart rate applications, blood oxygen applications), sports applications, device management applications, and third-party applications (e.g., music applications). This embodiment does not limit the specific type of application. The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0134] like Figure 3 As shown, the application framework layer may include a window manager, an input manager, a sensor manager, a phone manager, a resource manager, a notification manager, etc.

[0135] The input manager can be used to listen to input events of a user, such as a click event, a sliding event, and the like performed by a finger of the user on the display screen 194 of the electronic device 100. By listening to the input events, the electronic device 100 can determine whether the electronic device is being used.

[0136] The sensor manager is used to listen to data returned by various sensors in the electronic device, such as motion sensor data, proximity light sensor data, temperature sensor data, and the like. Using the data returned by the various sensors, the electronic device can determine whether it is shaking, or whether the display screen 194 is blocked, and the like.

[0137] In an embodiment of the present application, in a case where the electronic device 100 is a wearable device, the application framework layer can include, for example, an Ability Cross Platform Environment User Interface (ACE UI) framework, the ACE UI framework including an ACE application development module, and the ACE application development module including a Java Script (JS) engine. The JS engine can create a series of threads to form an independent thread model, and thus implement a high-performance rendering process.

[0138] The application framework layer can further include a system basic capability framework, an underlying software service capability framework, a motion health service capability framework, and a hardware service capability framework. The system basic capability framework can include, for example, a communication service module, an audio service module, and the like. The underlying software service capability framework can include, for example, a basic service module, which can include, for example, a Power Manage Service (PMS) module, a log service submodule, a dual machine communication submodule, a file system submodule, a memory management submodule, and the like. The motion health service capability framework can include, for example, a motion service module, which can include, for example, a data storage synchronization submodule, a heart health submodule, and the like. The hardware service capability framework can include, for example, a positioning service module, a traditional Bluetooth module. The positioning service module can include, for example, a fusion positioning submodule. The traditional Bluetooth module can include, for example, a connection management submodule. Different modules and submodules can perform corresponding service functions.

[0139] The application framework layer of an embodiment of the present application includes, but is not limited to, the above-described frameworks, and the above-described frameworks include, but are not limited to, the above-described submodules. The type of framework and the type of module in the application framework layer are not limited in the present application.

[0140] The Android Runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0141] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java file of the application layer and the application framework layer into a binary file. The virtual machine is used to execute the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection and the like.

[0142] The system library can include a plurality of function modules. For example: a surface manager (Surface Manager), media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL) and the like.

[0143] The surface manager is used for managing the display subsystem, and provides a plurality of applications with the fusion of 2D and 3D layers.

[0144] The media library supports a plurality of commonly used audio, video format playback and recording, and static image files and the like. The media library can support a plurality of audio and video coding formats, for example: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG and the like.

[0145] The three-dimensional graphics processing library is used for realizing three-dimensional graphics drawing, image rendering, synthesis, and layer processing and the like.

[0146] The 2D graphics engine is a drawing engine for 2D drawing.

[0147] The kernel layer is a layer between hardware and software.

[0148] The kernel layer can include a basic kernel, an extended kernel, a hardware abstraction layer (Hardware Abstraction Layer, HAL) and a driver layer.

[0149] The basic kernel can realize corresponding core functions such as memory management, task management, process communication, and interrupt management based on a memory (Memory, Mem) interface, a task interface, an inter-process communication (Inter Process Communication, IPC) interface, and an interrupt interface and the like.

[0150] The HAL layer can abstract the hardware operation interface to encapsulate the bottom driver interface into a unified application programming interface (Application Programming Interface, API). In this way, the hardware abstraction layer can simplify the complexity of the hardware operation of the application program.

[0151] It should be noted that the embodiments of the present application only take the HAL layer as a part of the kernel layer for example, and in fact, the HAL layer can also be independent of the kernel layer, and the embodiments of the present application do not limit this.

[0152] The driver layer at least includes display driver, camera driver, audio driver, and sensor driver.

[0153] In the embodiments of the present application, the driver layer includes DMA driver. The DMA driver is used to load the DMA device 196 in the hardware layer.

[0154] The specific hierarchical structure of the kernel layer is not further divided in the drawings of the present application. In the embodiments of the present application, the kernel layer further includes other types of interfaces, for example, channel allocation interface, initialization interface, and data copy interface.

[0155] The embodiments of the present application relate to the calling of the interrupt interface, the channel allocation interface, the initialization interface, and the data copy interface. This will be specifically described in subsequent embodiments.

[0156] In the embodiments of the present application, the display driver can include Liquid Crystal Display (LCD) driver. The LCD driver can be used to drive a non-rectangular display screen to implement screen initialization, data transmission, display control, and the like.

[0157] In the embodiments of the present application, the kernel layer further includes thread resources, for example, including interface refresh thread, display thread, and the like. The tasks performed by the threads in the thread resources can be managed by the upper layer, for example, by the virtual machine execution thread management of the upper layer.

[0158] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0159] The memory usage method provided by the embodiments of the present application can be applied in the scenario of displaying an image frame by the electronic device 100.

[0160] Figure 4 FIG. 1 is a first scenario diagram of displaying an image frame by the electronic device provided by the embodiments of the present application.

[0161] As shown in FIG. 1, the electronic device 100 can display an image frame after being powered on. Figure 4 The image frame can be an initial image frame displayed by the electronic device after being powered on, for example, the initial image frame is an image frame corresponding to a system main interface.

[0162] In fact, the image frame can also be an image frame displayed by the electronic device during use, and the image frame is usually an image frame corresponding to an application interface (which will be described in subsequent embodiments of the present application). The specific type of the image frame is not limited in the embodiments of the present application.

[0163] The embodiments of the present application can be applied to the process of image frame rendering and display involved in the above-mentioned scenarios to save the resources of the in-chip memory in the process.

[0164] It should be noted that the above-mentioned scenarios in which the electronic device 100 displays the image frame are only used for illustrative purposes. The change process of the display screen of the electronic device after the screen is turned on involves the rendering of the image frame, the updating of the image frame, and the display of the image frame. The memory usage method provided by the embodiments of the present application is not limited to being applied to the above-mentioned scenarios, but can also be applied to other scenarios of UI interface display. The embodiments of the present application are not limited in this regard.

[0165] The memory usage method provided by the embodiments of the present application will be described taking the process of displaying the initial image frame as an example.

[0166] Figure 5 is the first flowchart of the memory usage method provided by the embodiments of the present application.

[0167] Figure 6 is the first interaction schematic diagram of the memory usage method provided by the embodiments of the present application.

[0168] As shown in Figure 5 and Figure 6 , in some embodiments, the method includes the following steps S101-S106.

[0169] Step S101, the electronic device is turned on.

[0170] The user can perform a long press operation on the power-on key of the electronic device, and the electronic device performs a system startup operation in response to the long press operation on the power-on key. After the system startup is completed, the electronic device triggers a screen-on instruction and sends the screen-on instruction to the PMS module. In this way, the PMS module can control the display screen to turn on.

[0171] The user can also perform a restart operation on the electronic device when the electronic device is turned on, and the electronic device restarts the system in response to the restart operation. After the system restart is completed, the electronic device triggers a screen-on instruction and sends the screen-on instruction to the PMS module. In this way, the PMS module can control the display screen to turn on. The user can also perform a wrist lifting operation, a touch operation on the display screen, or a wrist turning operation, etc. when the electronic device is turned off, and the electronic device can respond to such operations to trigger a screen-on instruction and send the screen-on instruction to the PMS module. In this way, the PMS module can control the display screen to turn on.

[0172] In the embodiments of the present application, the user can operate the electronic device based on multiple operation modes to make the electronic device brighten the screen. The embodiments of the present application do not limit the screen brightening mode of the electronic device.

[0173] In step S102, in response to the electronic device brightening the screen, the display screen in the hardware layer generates a first interrupt signal and sends the first interrupt signal to the kernel layer.

[0174] In an implementation manner, the interrupt signal can include at least one of a clock (CLK) signal, a horizontal synchronization (HSYNC) signal, a vertical synchronization (VSYNC) signal, and a data enable (DE) signal.

[0175] For example, the first interrupt signal can be a VSYNC signal. The VSYNC signal can be used to indicate that the display screen completes the display of one frame of image in the vertical direction, that is, the display of the next frame will be performed. The first interrupt signal can be generated periodically. The embodiments of the present application do not limit the specific signal type of the first interrupt signal.

[0176] The first interrupt signal can be generated periodically with a first preset time length. For example, the first preset time length is 16.6 ms. Specifically, the frame rate of the electronic device needs to be at least 60 hertz (Hz) to ensure that the user will not have a visual experience of lag, so that the refresh rate of the image frame is at least 16.6 ms / time. Therefore, the display screen needs to generate a first interrupt signal every 16.6 ms. The embodiments of the present application do not limit the specific value of the first preset time length.

[0177] Specifically, step S102 includes step S1021.

[0178] In step S1021, in response to the electronic device brightening the screen, the display screen generates a first interrupt signal and sends the first interrupt signal to a first interrupt interface in the kernel layer.

[0179] The interrupt interface can be a part of the operating system or the HAL layer, which is used to manage the communication between the interrupt signal of the hardware and the interrupt handling program (ISR) of the software.

[0180] In the display driver corresponding to the display screen, the developer can define one or more callback functions, and register the callback functions in the interrupt interface when initializing the hardware configuration of the display screen. Thus, when the interrupt interface receives an interrupt signal, the interrupt interface can call the callback function to execute the logic program predefined in the callback function for processing the interrupt event. The predefined logic program can be used to update the display buffer, process user input, and the like.

[0181] In an embodiment of the present application, the first interrupt interface can include a callback function for updating the display buffer.

[0182] At step S103, in response to receiving the first interrupt signal, the kernel layer renders a first image frame in the first framebuffer. The first image frame includes a non-rectangular region corresponding to the shape of the display screen, and the non-rectangular region includes a first part and a second part. The second part is a region of the non-rectangular region other than the first part.

[0183] It should be noted that the first image frame itself corresponds to a rectangular display region, and the non-rectangular region is used in a subsequent copying process.

[0184] Figure 7 FIG. 1 is a schematic diagram of a non-rectangular region according to an embodiment of the present application.

[0185] As shown in FIG. 1, in an embodiment of the present application, the first image frame can be an initial image frame. The type of the first image frame is not limited in the embodiments of the present application. Figure 7

[0186] The non-rectangular region in the first image frame can be a circular region. The first part can be the upper half of the circular region, and the second part can be the lower half of the circular region. In fact, the first part and the second part can also have other division manners. For example, the first part can be the lower half of the circular region, and the second part can be the upper half of the circular region. The specific division manner of the first part and the second part is not limited in the embodiments of the present application.

[0187] In an implementation manner, step S103 includes steps S1031-S1032.

[0188] At step S1031, in response to receiving the first interrupt signal, the first interrupt interface sends a first trigger message to the first thread.

[0189] For example, the first thread is an interface refresh thread. The type of the first thread is not limited in the embodiments of the present application.

[0190] ​The interface refresh thread is a thread used to update and refresh user interface elements in a graphical user interface (GUI) system.

[0191] The interface refresh thread can be run after the electronic device is powered on. The interface refresh thread has a first message queue for storing messages to be processed in the interface refresh thread. When there are messages in the first message queue, the interface refresh thread can process the messages in the first message queue in turn according to the first-in first-out principle. When there are no messages in the first message queue, it indicates that there is no message that needs to be processed by the interface refresh thread at present, and at this time, the interface refresh thread is in a suspended state to wait for the first message queue to receive a message.

[0192] After the first interrupt interface receives the first interrupt signal, a first trigger message is stored in the first message queue of the interface refresh thread. The first trigger message is used to instruct the interface refresh thread to perform a rendering operation.

[0193] In step S1032, the first thread calls the GPU to render the first image frame in the first framebuffer in response to the first trigger message.

[0194] For example, after the electronic device is turned on, the desktop application sends interface parameters related to the main interface to the interface refresh thread. The interface parameters may include, for example, parameters corresponding to controls, text, and images.

[0195] In this way, the interface refresh thread can respond to the first trigger message and call the GPU to render the image frame corresponding to the main interface in the first framebuffer.

[0196] In an implementation manner, before the kernel layer renders the first image frame in the first framebuffer, the memory capacity of the second framebuffer needs to be modified first to match the memory capacity of the second framebuffer with the memory occupied by the non-rectangular region. In this way, after the first thread renders the first image frame, subsequent steps can be quickly performed.

[0197] Figure 8 is a schematic diagram of the modified memory capacity of the second framebuffer provided by an embodiment of the present application.

[0198] As shown in Figure 8 The initial memory capacity of the second framebuffer can be the same as that of the first framebuffer, for example, referring to the foregoing embodiment, the initial memory capacity of the first framebuffer and the second framebuffer can each be 466x466x3 bytes, about 650KB.

[0199] When the non-rectangular region is circular, the modified memory capacity of the second framebuffer can be, for example, π×(466 / 2). 2 ×3, approximately 509538 bytes, approximately 510KB.

[0200] It should be noted that the modified memory capacity of the second Framebuffer is based on the ability to accommodate non-rectangular areas. This memory capacity can have a certain error range and should be adjusted according to the actual situation. This application embodiment does not limit the specific value of the modified memory capacity of the second Framebuffer.

[0201] It is important to note that the circular area in the second framebuffer in the attached diagram is only used to indicate the memory size occupied by the data, and does not necessarily represent the actual memory area occupied. The actual memory area occupied in the second framebuffer can be a region with contiguous physical addresses.

[0202] In this way, the modified memory capacity of the second framebuffer can be used for subsequent processes. Compared to the case where the initial memory capacity of the second framebuffer is fully utilized, the embodiments of this application save approximately 20% of the memory capacity in the second framebuffer.

[0203] Specifically, the kernel layer can modify the memory capacity of the second framebuffer based on the Graphics Memory Management Unit (GFXMMU).

[0204] GFXMMU can optimize memory usage based on the shape of the display screen, allowing the microcontroller to store only the non-rectangular areas corresponding to the visible portion of the display screen in a contiguous physical storage area, thereby saving memory space in the frame buffer.

[0205] GFXMMU can manage memory using a virtual buffer. The virtual buffer allows users to define a memory region that contains only the actual content to be displayed, rather than an entire rectangular area. In this embodiment, the virtual buffer can define a memory region corresponding to a circular display area.

[0206] The virtual buffer can be configured with a lookup table (LUT), which defines the mapping rules for each pixel. Users can define the first and last visible pixels of each row, as well as the color depth of the frame buffer, within the LUT. Based on this, GFXMMU can generate the LUT configuration and calculate the required memory footprint. This simplifies the development process and ensures efficient memory management.

[0207] GFXMMU determines the address and memory size in the second framebuffer based on the LUT configuration, so that the second framebuffer uses the modified memory capacity to actually store image frames.

[0208] It should be noted that the memory capacity of the second framebuffer can also be modified using other methods such as virtual memory technology in this application embodiment. This application embodiment does not limit the specific method for modifying the memory capacity of the second framebuffer.

[0209] In step S104, after the first image frame is rendered, the kernel layer copies the first part to the first memory so that the first memory receives the first copied part, and the kernel layer copies the second part to the second framebuffer so that the second framebuffer receives the second copied part.

[0210] In one implementation, the kernel layer copies a first portion to a first memory based on a first channel, so that the first memory receives the first copied portion, and copies a second portion to a second framebuffer based on a second channel, so that the second framebuffer receives the second copied portion, wherein the first channel and the second channel have different copying devices.

[0211] For example, the copy device in the first channel can be a GPU, and the copy device in the second channel can be DMA2D.

[0212] It should be noted that since the first memory does not have its memory capacity modified in advance, the kernel layer can copy the upper half of the rectangular region in the first image frame to the first memory, rather than just copying the first part to the first memory. This can improve the memory region compatibility between the first framebuffer and the first memory.

[0213] Figure 9 This is a schematic diagram of the copying process of the first image frame provided in the embodiments of this application.

[0214] like Figure 9 As shown, in one implementation, step S104 includes steps S1041-S1042.

[0215] In step S1041, in response to the next receipt of the first interrupt signal, the first interrupt interface sends a second trigger message to the second thread.

[0216] For example, the second thread is the display thread. This application does not limit the thread type of the second thread in its embodiments.

[0217] The display thread is a thread in the GUI system used to manage display output and refresh display content.

[0218] The display thread can run after the electronic device is powered on. The display thread has a second message queue for storing messages to be processed in the display thread. When there is a message in the second message queue, the display thread can process each message in the second message queue in turn according to the first-in first-out principle. When there is no message in the second message queue, it indicates that there is currently no message that needs to be processed by the display thread, and at this time, the display thread is in a suspended state to wait for the second message queue to receive a message.

[0219] Based on the foregoing embodiment, the first interrupt interface executes step S1031 when it first receives the first interrupt signal. Since the first interrupt signal is periodically generated, the first interrupt interface can periodically receive subsequent first interrupt signals after it first receives the first interrupt signal. In this way, the first interrupt interface stores a second trigger message in the second message queue of the display thread when it receives the first interrupt signal for the second time. The second trigger message is used to instruct the display thread to perform a copy operation.

[0220] It should be noted that the first interrupt interface can determine whether it is currently receiving the first interrupt signal for the first time. If the first interrupt interface determines that it is currently receiving the first interrupt signal for the first time, the first interrupt interface executes step S1031. If the first interrupt interface determines that it is currently receiving the first interrupt signal for the second time, the first interrupt interface executes step S1041.

[0221] In step S1042, in response to the second trigger message, the second thread copies the first part based on the GPU corresponding to the first channel to the first memory, so that the first memory obtains the first copy part, and the second thread copies the second part based on the DMA2D corresponding to the second channel to the second framebuffer.

[0222] The DMA2D is used to process 2D graphics acceleration tasks, such as processing color filling, image copying, color format conversion, and transparency mixing tasks.

[0223] The DMA2D can perform color filling in a specified area. For example, the DMA2D performs color filling in the second framebuffer after modifying the memory capacity.

[0224] The DMA2D can also copy pixels in one memory area to another memory area, and supports different color formats to ensure image quality during copying. For example, the DMA2D copies pixels in the first framebuffer to the second framebuffer.

[0225] The DMA2D can also convert image data from one color format to another color format, for example, from RGB888 format to RGB565 format. In the RGB888 format, each pixel is composed of three bytes, representing the red component, the green component and the blue component respectively, and the value range of each component is 0-255 (i.e. 28). 8- 1). Therefore, the RGB888 format can represent about 16.77 million colors. In the RGB565 format, each pixel is composed of two bytes, in which the red component occupies 5-bit, the green component occupies 6-bit, and the blue component occupies 5-bit. Thus, the value range of the red component is 0-31 (i.e. 25 5 -1), the value range of the blue component is 0-31 (i.e. 25 5 -1), and the value range of the green component is 0-63 (i.e. 26 6- 1). The DMA2D can reduce the red component in the image frame of the RGB888 format from 8-bit to 5-bit, reduce the green component from 8-bit to 6-bit, and reduce the blue component from 8-bit to 5-bit. In this way, the DMA2D can complete the conversion from the RGB888 format to the RGB565 format.

[0226] It should be noted that in the embodiments of the present application, the DMA2D can support conversion between multiple color formats, for example, the DMA2D can support conversion between RGB, YUV, RGB888, RGB565 and other different color formats. The color format conversion manner involved in the process of copying the image frame by the DMA2D is not limited in the embodiments of the present application.

[0227] In the embodiments of the present application, the DMA2D can perform color format conversion on the first image frame in the process of copying the first image frame, or can copy the first image frame in the original color format. The specific manner of copying the first image frame by the DMA2D is not limited in the embodiments of the present application.

[0228] For example, the first image frame can be in the RGB888 color format. When the first thread copies the second part in the first image frame based on the DMA2D corresponding to the second channel to the second framebuffer, the second part can still have the RGB888 color format.

[0229] It should be noted that the embodiment of the present application adopts DMA2D to copy the second part, instead of GPDMA, because DMA2D can automatically calculate the copy address and automatically perform line-by-line copying when copying the local area (such as the second part) in the first framebuffer, which is the ability that GPDMA does not have. That is, GPDMA does not support copying the rectangular image frame in the first framebuffer to the area after the modified memory capacity of the second framebuffer, and only DMA2D can realize this copying process.

[0230] However, if only DMA2D is used to copy the first image frame in the first framebuffer, the copying time is about 2.8 ms. As known from the foregoing, in the process of copying the image frame based on GPDMA, the use of the on-chip memory consumes about 1.8 ms. Therefore, based on DMA2D, copying the first image frame in the first framebuffer to the second framebuffer after modifying the memory capacity can save memory, but increases the copying time by nearly 1 ms, which affects the frame rate performance of the electronic device.

[0231] Since the embodiment of the present application copies the first image frame based on different copying devices on two channels, the copying processes on the two channels can be performed simultaneously, that is, the first thread copies the upper half of the rectangular area in the first framebuffer to the first memory off-chip, and at the same time, the first thread copies the lower half of the non-rectangular area in the first framebuffer to the second framebuffer. In this way, the amount of data copied on each channel is reduced by half, which can reduce the copying time. In the process of copying the first part to the first memory and copying the second part to the second framebuffer, the copying time only needs about 1.6 ms. Compared with the image copying process based on GPDMA, the embodiment of the present application does not increase the copying time and does not affect the frame rate performance of the electronic device.

[0232] Specifically, the display thread can first send a request to the channel allocation interface to request the channel allocation interface to allocate the available first channel and the second channel to the display thread. Further, the display thread can call the initialization interface to initialize the first channel and the second channel. After the initialization of the first channel and the second channel is completed, the display thread can call the data copying interface to send data copying parameters to the GPU and the DMA2D, so that the GPU copies the first part to the first memory based on the corresponding data copying parameters, and the DMA2D copies the second part to the second framebuffer based on the corresponding data copying parameters.

[0233] The data copy parameter is used for the copy device to determine the image frame area of the required copy, so as to determine the copy position of the image frame. For example, the data copy parameter includes the data amount corresponding to the first part, the data amount corresponding to the second part, the starting address information of the first part corresponding to the first framebuffer, the starting address information of the first part in the first memory, the starting address information of the second part corresponding to the first framebuffer, and the starting address information of the second part corresponding to the second framebuffer. The embodiment of the present application does not limit the specific parameter type included in the data copy parameter.

[0234] It should be noted that, since the DMA2D can independently complete the data copy, the display thread needs to monitor the first channel and the second channel, so as to determine whether the GPU and the DMA2D complete the data copy, and further determine whether the subsequent operation needs to be performed.

[0235] The copy process in the second channel is exemplarily described.

[0236] In an implementation manner, the DMA2D generates a DMA interrupt signal after completing each data copy operation, and the display thread can create a target object for monitoring the DMA interrupt signal, so as to determine whether the DMA2D completes the data copy based on the target object. The display thread can also create a target object for monitoring the GPU in the first channel, and the embodiment of the present application does not repeat the description.

[0237] In an implementation manner, the step S104 further includes a step S104a.

[0238] In the step S104a, the kernel layer renders a second image frame in the first framebuffer, and the second image frame is a next frame image frame of the first image frame.

[0239] In an implementation manner, the step S104a includes a step S1041a.

[0240] In the step S1041a, the second thread sends a rendering instruction to the first thread, so that the first thread calls the GPU to render the second image frame in the first framebuffer in response to the rendering instruction.

[0241] It should be noted that, before the display thread sends the rendering instruction to the interface refresh thread, it is necessary to determine whether the first image frame in the first framebuffer is completed.

[0242] In an implementation manner, the display thread can obtain the data amount copied by the DMA2D, and compare the data amount with a preset data amount threshold, so as to determine whether the copy is completed.

[0243] In an implementation, the display thread can determine whether the DMA interrupt signal is received within a preset time length to determine whether the copying is completed.

[0244] The embodiments of the present application do not limit the specific manner in which the display thread determines whether the first image frame in the first framebuffer has been copied.

[0245] Further, the display thread can release the first channel and the second channel in the case that the display thread determines that the first image frame in the first framebuffer has been copied, so that the first channel and the second channel can be reused.

[0246] After the display thread completes the above process, step S1041a can be performed.

[0247] That is, after the first image frame in the first framebuffer is copied out by the GPU and the DMA2D, the first framebuffer can start rendering the next frame without waiting for the second framebuffer to completely obtain the first part and the second part. In this way, the frame rate performance of the electronic device is ensured.

[0248] In step S105, the kernel layer copies the first copy part in the first memory to the second framebuffer, so that the second framebuffer obtains a third copy part, wherein the second copy part and the third copy part form a target image frame corresponding to the shape of the display screen.

[0249] It should be noted that step S105 and step S104a can be executed synchronously or asynchronously, and the embodiments of the present application do not limit the execution order of step S105 and step S104a.

[0250] In an implementation, the kernel layer copies the first copy part to the second framebuffer based on a third channel, so that the second framebuffer obtains a third copy part, wherein the third channel and the first channel have different copying devices.

[0251] For example, in the case that the copying device in the first channel is the GPU, the copying device in the third channel is the DMA2D.

[0252] It should be noted that the third channel can be the reused second channel or a newly enabled channel, and the embodiments of the present application do not limit this.

[0253] In an implementation, step S105 includes step S1051.

[0254] Step S1051, the second thread copies the first copy part to the second framebuffer based on the third channel corresponding DMA2D, so that the second framebuffer obtains the third copy part.

[0255] The second copy part and the third copy part can form a target image frame. The target image frame can be adapted to the display area of the non-rectangular display screen.

[0256] Since the first copy part is located in the first memory, the second thread further needs to copy the first copy part to the second framebuffer, so that the second framebuffer obtains the target image frame adapted to the display area of the non-rectangular display screen.

[0257] This copying process can be performed simultaneously with the rendering process in step S1041, so as not to affect the frame rate performance of the electronic device.

[0258] Step S106, the kernel layer displays the target image frame based on the second framebuffer, so as to display the target image frame on the display screen.

[0259] In an implementation manner, step S106 includes steps S1061-S1062.

[0260] Step S1061, in response to next receiving the first interrupt signal, the first interrupt interface sends a third trigger message to the second thread.

[0261] Step S1062, in response to the third trigger message, the second thread displays the target image frame based on the second framebuffer, so as to display the target image frame on the display screen.

[0262] In this way, the method provided by the present application is applied to the electronic device with the non-rectangular display screen, which can not only perform the region-based copying on the non-rectangular area corresponding to the shape of the display screen in the first image frame to save the resources of the on-chip memory, but also will not affect the frame rate performance of the electronic device.

[0263] Figure 10 FIG. 2 is a second scene diagram of the electronic device displaying an image frame provided by an embodiment of the present application.

[0264] As Figure 10As shown, the electronic device 100 is a wearable device that supports independent music playback. The electronic device 100 can display a main interface 1, and on the main interface 1, display a music application icon 2 corresponding to the music application. The user can click on the music application icon 2. In response to the click, the electronic device 100 renders an image frame corresponding to the music application interface 3 based on an interface refresh thread. After the image frame is rendered, it sends the rendered image frame to the display screen based on a display thread, so that the display screen displays the music application interface 3 corresponding to the image frame.

[0265] The music application can update the image frames, thus changing the music application interface 3. For example, the music application can call the audio service module to obtain a Lyric Resource Chart (LRC) file and decode it to obtain timestamps and lyrics data. Then, when the audio service module detects a time change event in the audio player, it can send the lyrics data to the GPU via the interface refresh thread based on the timestamp corresponding to the time change event, triggering an update of the image frames.

[0266] The embodiments of this application can be applied to the process of image frame rendering and display in the above-mentioned scenarios, so as to save on-chip memory resources in the process.

[0267] It should be noted that the scenario of the electronic device 100 displaying image frames is only used for illustrative purposes. The application involves the rendering, updating and displaying of image frames. The memory usage method provided in this application is not limited to the above scenario, but can also be applied to other UI display scenarios. This application does not limit this.

[0268] Taking the display of an image frame corresponding to the application interface as an example, the memory usage method provided in this application embodiment will be described.

[0269] Figure 11 This is the second flowchart of the memory usage method provided in the embodiments of this application.

[0270] Figure 12 This is the second interactive schematic diagram of the memory usage method provided in the embodiments of this application.

[0271] like Figure 11 and Figure 12 As shown, in some embodiments, the method includes the following steps S201-S206.

[0272] Step S201: The electronic device displays the first application interface.

[0273] Step S202, in response to the click operation of the user on the first application interface, the display screen in the hardware layer generates a first interrupt signal and sends the first interrupt signal to the kernel layer.

[0274] In an implementation manner, step S202 includes step S2021.

[0275] Step S2021, in response to the click operation of the user on the first application interface, the display screen generates a first interrupt signal and sends the first interrupt signal to a first interrupt interface in the kernel layer.

[0276] Step S203, in response to receiving the first interrupt signal, the kernel layer renders a first image frame in the first Framebuffer, wherein the first image frame includes a non-rectangular region corresponding to the shape of the display screen, and the non-rectangular region includes a first part and a second part, and the second part is a region of the non-rectangular region except the first part.

[0277] In an implementation manner, step S203 includes steps S2031-S2032.

[0278] Step S2031, in response to receiving the first interrupt signal, the first interrupt interface sends a first trigger message to the first thread.

[0279] The first trigger message includes a message triggered after the display screen completes the display of a previous image frame.

[0280] Step S2032, in response to the first trigger message, the first thread calls the GPU to render the first image frame in the first Framebuffer.

[0281] In the embodiment, the first image frame can be an image frame updated by the application program.

[0282] Step S204, after the first image frame is rendered, the kernel layer copies the first part to the first memory so that the first memory obtains a first copied part, and the kernel layer copies the second part to the second Framebuffer so that the second Framebuffer obtains a second copied part.

[0283] In an implementation manner, step S204 includes steps S2041-S2042.

[0284] Step S2041, in response to next receiving the first interrupt signal, the first interrupt interface sends a second trigger message to the second thread.

[0285] Step S2042, in response to the second trigger message, the second thread copies the first part to the first memory based on the GPU corresponding to the first channel, so that the first memory obtains the first copy part, and the second thread copies the second part to the second framebuffer based on the DMA2D corresponding to the second channel.

[0286] In an implementation manner, after the step S204, the method further includes a step S204a.

[0287] Step S204a, the kernel layer renders a second image frame in the first framebuffer, the second image frame being a next frame image frame of the first image frame.

[0288] In an implementation manner, the step S204a includes a step S2041a.

[0289] Step S2041a, the second thread sends a rendering instruction to the first thread, so that the first thread renders the second image frame in the first framebuffer in response to the rendering instruction.

[0290] Step S205, the kernel layer copies the first copy part in the first memory to the second framebuffer, so that the second framebuffer obtains a third copy part, wherein the second copy part and the third copy part form a target image frame corresponding to a shape of the display screen.

[0291] In an implementation manner, the step S205 includes a step S2051.

[0292] Step S2051, the second thread copies the first copy part to the second framebuffer based on the DMA2D corresponding to the third channel, so that the second framebuffer obtains the third copy part.

[0293] Step S206, the kernel layer displays the target image frame in the display screen based on the second framebuffer.

[0294] In an implementation manner, the step S206 includes steps S2061-S2062.

[0295] Step S2061, in response to next receiving the first interrupt signal, the first interrupt interface sends a third trigger message to the second thread.

[0296] Step S2062, in response to the third trigger message, the second thread displays the target image frame in the display screen based on the second framebuffer.

[0297] It should be noted that in this embodiment, the triggering method of the first interrupt signal and the triggering method of the first trigger message are different from those in the previous embodiments. The specific implementation methods of the other steps can be referred to in the previous embodiments, and this application will not elaborate on them.

[0298] Figure 13 This is the third flowchart of the memory usage method provided in the embodiments of this application.

[0299] like Figure 13 As shown, in some embodiments, the method includes the following steps S301-S304.

[0300] Step S301: Render a first image frame in the first framebuffer. The first image frame includes a non-rectangular region corresponding to the shape of the display screen. The non-rectangular region includes a first part and a second part. The second part is the area in the non-rectangular region other than the first part.

[0301] Step S302: After the first image frame is rendered, the first part is copied to the first memory so that the first memory has the first copied part, and the second part is copied to the second framebuffer so that the second framebuffer has the second copied part.

[0302] Step S303: Copy the first copy portion in the first memory to the second framebuffer so that the second framebuffer obtains a third copy portion, wherein the second copy portion and the third copy portion form a target image frame corresponding to the shape of the display screen.

[0303] Step S304: The target image frame is sent to the display based on the second Framebuffer to display the target image frame on the display screen.

[0304] The memory usage method provided in this application can be applied to electronic devices with non-rectangular displays. This method can copy non-rectangular regions in an image frame that correspond to the shape of the display screen in sections. In this way, compared with the conventional method of copying rectangular image frames, this application can save on-chip memory resources while ensuring frame rate performance.

[0305] The embodiments provided in this application above describe various schemes of the memory usage method provided by this application from the perspective of an electronic device. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0306] Figure 14 This is a schematic diagram of the memory usage device provided in the embodiments of this application.

[0307] like Figure 14 As shown, the memory usage device 200 provided in this application embodiment can be applied to an electronic device 100. The electronic device 100 includes a display screen, a memory, and a processor. The display screen, memory, and processor are coupled together. The shape of the display screen includes shapes other than rectangles. The memory includes a first framebuffer, a second framebuffer, and a first memory. The first framebuffer and the second framebuffer are both disposed on the processor chip, and the first memory is disposed outside the processor chip and connected to the processor. The device includes:

[0308] The rendering module 201 is used to render a first image frame in a first framebuffer. The first image frame includes a non-rectangular region corresponding to the shape of the display screen. The non-rectangular region includes a first part and a second part. The second part is the area in the non-rectangular region other than the first part.

[0309] The first copy module 202 is used to copy a first part to a first memory after the first image frame is rendered, so that the first memory obtains the first copied part, and to copy a second part to a second framebuffer, so that the second framebuffer obtains the second copied part.

[0310] The second copy module 203 is used to copy the first copy portion in the first memory to the second framebuffer so that the second framebuffer obtains the third copy portion, wherein the second copy portion and the third copy portion form a target image frame corresponding to the shape of the display screen.

[0311] The display module 204 is used to send the target image frame to the display screen based on the second framebuffer so as to display the target image frame on the display screen.

[0312] The apparatus provided in this application embodiment can be applied to electronic devices with non-rectangular displays. The apparatus can perform region-by-region copying of non-rectangular areas in an image frame that correspond to the shape of the display screen. In this way, compared with the conventional method of copying rectangular image frames, this application can save on-chip memory resources while ensuring frame rate performance.

[0313] Figure 15 This is a schematic diagram of the structure of a memory usage device provided in another embodiment of this application.

[0314] like Figure 15 As shown, the memory usage device provided in this application embodiment may include: a display screen 1001, a memory 1002, a processor 1003, and a communication module 1004. These devices can be connected via one or more communication buses 1005. The display screen 1001 may include a display panel 10011 and a touch sensor 10012. The display panel 10011 is used to display images, and the touch sensor 10012 can transmit detected touch operations to the application processor to determine the touch event type, providing visual output related to the touch operation through the display panel 10011. The processor 1003 may include one or more processing units, such as an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. Different processing units may be independent devices or integrated into one or more processors. The memory 1002 is coupled to the processor 1003 and is used to store various software programs and / or computer instructions. The memory 1002 may include volatile memory and / or non-volatile memory. When the processor executes computer instructions, the memory-using device can perform various functions or steps performed by the electronic device 100 in the above method embodiments.

[0315] In some embodiments, when the software programs and / or sets of instructions in the memory 1002 are run by the processor 1003, the device is caused to perform the following method steps: rendering a first image frame in a first framebuffer, the first image frame including a non-rectangular region corresponding to the shape of the display screen, the non-rectangular region including a first portion and a second portion, the second portion being the region of the non-rectangular region other than the first portion; after the first image frame is rendered, copying the first portion to a first memory so that the first memory obtains a first copied portion, and copying the second portion to a second framebuffer so that the second framebuffer obtains a second copied portion; copying the first copied portion in the first memory to the second framebuffer so that the second framebuffer obtains a third copied portion, wherein the second copied portion and the third copied portion form a target image frame corresponding to the shape of the display screen; and sending the target image frame based on the second framebuffer for display, so as to display the target image frame on the display screen.

[0316] The embodiments of the present application further provide an electronic device, which can include a touch screen, a memory and one or more processors. The touch screen, the memory and the processors are coupled. The memory is configured to store computer program codes, which include computer instructions. When the processors execute the computer instructions, the electronic device can perform each function or step performed by the electronic device in the above-mentioned method embodiments. The structure of the electronic device can refer to the structure of the electronic device 100 shown in Figure 2

[0317] The embodiments of the present application further provide a chip system, for example, a System on a Chip (SoC), which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a circuit. For example, the interface circuit can be configured to receive signals from other devices, for example, the memory of the electronic device. For another example, the interface circuit can be configured to send signals to other devices, for example, the processor or the touch screen of the electronic device. Illustratively, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform each step in the above-mentioned embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0318] The embodiments of the present application further provide a computer readable storage medium, which includes computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device is caused to perform each function or step performed by the electronic device in the above-mentioned method embodiments.

[0319] ​The embodiment of the present application further provides a computer program product, which, when running on an electronic device, causes the electronic device to perform each function or step performed by the electronic device in the method embodiment.

[0320] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0321] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiment described above is only schematic, for example, the division of the module or unit is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0322] The unit described as a separate component can be or can not be physically separated, and the component shown as a unit can be one physical unit or a plurality of physical units, that is, can be located in one place, or can be distributed to a plurality of different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0323] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0324] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0325] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A memory usage method characterized by, The application is applied to an electronic device, the electronic device comprising a display screen, a memory and a processor, the display screen, the memory and the processor being coupled, wherein the shape of the display screen comprises a shape other than a rectangle, the memory comprises a first frame buffer, a second frame buffer and a first memory, the first frame buffer and the second frame buffer are both arranged on a chip of the processor, and the first memory is arranged outside the chip of the processor and connected with the processor, and the method comprises the following steps: rendering a first image frame in the first frame buffer, the first image frame comprising a non-rectangular region corresponding to the shape of the display screen, the non-rectangular region comprising a first part and a second part, and the second part being a region other than the first part in the non-rectangular region; after the rendering of the first image frame is completed, copying the first part to the first memory so that the first memory obtains a first copied part, and copying the second part to the second frame buffer so that the second frame buffer obtains a second copied part; copying the first copied part in the first memory to the second frame buffer so that the second frame buffer obtains a third copied part, wherein the second copied part and the third copied part form a target image frame corresponding to the shape of the display screen; based on the second frame buffer, sending the target image frame to be displayed so as to display the target image frame on the display screen.

2. The memory usage method of claim 1, wherein, Before the step of rendering a first image frame in the first frame buffer, the method further comprises the following step: modifying the memory capacity of the second frame buffer so that the memory capacity matches the memory occupied by the non-rectangular region.

3. The memory usage method of claim 2, wherein, The step of modifying the memory capacity of the second frame buffer comprises the following step: based on a graphics memory management unit (GFXMMU), modifying the memory capacity.

4. The memory usage method of claim 1, wherein, The step of, after the rendering of the first image frame is completed, copying the first part to the first memory so that the first memory obtains a first copied part, and copying the second part to the second frame buffer so that the second frame buffer obtains a second copied part, comprises the following steps: based on a first channel, copying the first part to the first memory so that the first memory obtains the first copied part, and based on a second channel, copying the second part to the second frame buffer so that the second frame buffer obtains the second copied part, wherein the first channel and the second channel have different copying devices.

5. The memory usage method of claim 4, wherein, The step of copying the first copied part in the first memory to the second frame buffer so that the second frame buffer obtains a third copied part, comprises the following step: based on a third channel, copying the first copied part to the second frame buffer so that the second frame buffer obtains the third copied part, wherein the third channel and the first channel have different copying devices.

6. The memory usage method of claim 5, wherein, The copy device corresponding to the first channel includes a graphic processing unit (GPU), and the copy device corresponding to the second channel includes a direct memory access two-dimensional graphic accelerator (DMA2D), the copying of the first part to the first memory based on the first channel to make the first memory obtain the first copy part, and the copying of the second part to the second frame buffer based on the second channel to make the second frame buffer obtain the second copy part include: copying the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copy part, and copying the second part to the second frame buffer based on the DMA2D corresponding to the second channel to make the second frame buffer obtain the second copy part.

7. The memory usage method of claim 6, wherein, The copy device corresponding to the third channel includes the DMA2D, and the copying of the first copy part to the second frame buffer based on the third channel to make the second frame buffer obtain the third copy part include: copying the first copy part to the second frame buffer based on the DMA2D corresponding to the third channel to make the second frame buffer obtain the third copy part.

8. The memory usage method of claim 7, wherein, After the copying of the first part to the first memory to make the first memory obtain the first copy part, and the copying of the second part to the second frame buffer to make the second frame buffer obtain the second copy part, further include: rendering a second image frame in the first frame buffer, the second image frame being a next frame image frame of the first image frame.

9. The memory usage method of claim 8, wherein, The rendering of the first image frame in the first frame buffer include: The kernel layer renders the first image frame in the first frame buffer.

10. The memory usage method of claim 9, wherein, The copying of the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copy part, and the copying of the second part to the second frame buffer based on the DMA2D corresponding to the second channel to make the second frame buffer obtain the second copy part include: The kernel layer copies the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copy part, and the kernel layer copies the second part to the second frame buffer based on the DMA2D corresponding to the second channel to make the second frame buffer obtain the second copy part.

11. The memory usage method of claim 10, wherein, The copying of the first copy part to the second frame buffer based on the DMA2D corresponding to the third channel to make the second frame buffer obtain the third copy part include: The kernel layer copies the first copy part to the second frame buffer based on the DMA2D corresponding to the third channel to make the second frame buffer obtain the third copy part.

12. The memory usage method of claim 11, wherein, The sending of the target image frame based on the second frame buffer to display the target image frame on the display screen include: The kernel layer sends the target image frame for display based on the second frame buffer to display the target image frame in the display screen.

13. The memory usage method of claim 12, wherein, The rendering of the second image frame in the first frame buffer comprises: The kernel layer renders the second image frame in the first frame buffer.

14. The memory usage method of claim 13, wherein, The kernel layer comprises a first thread, and the kernel layer renders the first image frame in the first frame buffer comprises: The first thread calls a graphic processing unit (GPU) to render the first image frame in the first frame buffer.

15. The memory usage method of claim 14, wherein, The kernel layer further comprises a second thread, and the kernel layer copies the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copy part, and copies the second part to the second frame buffer based on the DMA2D corresponding to the second channel to make the second frame buffer obtain the second copy part, comprising: The second thread copies the first part to the first memory based on the GPU corresponding to the first channel to make the first memory obtain the first copy part, and copies the second part to the second frame buffer based on the DMA2D corresponding to the second channel.

16. The memory usage method of claim 15, wherein, The kernel layer copies the first copy part to the second frame buffer based on the DMA2D corresponding to the third channel to make the second frame buffer obtain the third copy part, comprising: The second thread copies the first copy part to the second frame buffer based on the DMA2D corresponding to the third channel to make the second frame buffer obtain the third copy part.

17. The memory usage method of claim 16, wherein, The kernel layer sends the target image frame for display based on the second frame buffer to display the target image frame in the display screen, comprising: The second thread sends the target image frame for display based on the second frame buffer to display the target image frame in the display screen.

18. The memory usage method of claim 17, wherein, The kernel layer renders the second image frame in the first frame buffer, comprising: The second thread sends a rendering instruction to the first thread to make the first thread render the second image frame in the first frame buffer in response to the rendering instruction.

19. An electronic device, comprising: The electronic device comprises a display screen, a memory and a processor, the display screen, the memory and the processor are coupled, wherein the shape of the display screen comprises a shape other than a rectangle, the memory stores computer program code, the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the memory usage method as claimed in any one of claims 1-18.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises computer instructions, and when the computer instructions are executed on the electronic device, the electronic device executes the memory usage method as claimed in any one of claims 1-18.

Images