Memory management method, memory management device and electronic equipment

Abstract

The embodiment of the invention provides a memory management method, a memory management device and electronic equipment. The memory management method comprises the following steps: receiving a first input of a user, wherein the first input is used for enabling a first application to enter a background state; intercepting an interface related to access of caches in the first cache set, wherein the caches in the first cache set belong to caches related to decoding of a first video played by the first application; and recovering the physical memory corresponding to the cache in the first cache set. According to the memory management method, the memory management device and the electronic equipment provided by the embodiment of the invention, the physical memory corresponding to the cache related to video decoding can be recycled, so that the available physical memory of the whole machine is increased, and the response speed of the electronic equipment and the experience feeling of a user are improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of electronic technology, and more specifically, to a memory management method, a memory management device, and an electronic device. Background Art

[0002] When the application is running in the background, the operating system can reclaim memory in various ways when memory is tight, such as page discarding, page swapping, page writeback, etc., to relieve memory pressure.

[0003] In the video playback scenario, a lot of input and output caches are involved. After the video enters the background, the related caches will not be destructed and continue to occupy a lot of physical memory, resulting in the electronic device having less and less memory as the number of video applications opened by users increases. In addition, the hardware decoders called by the video application and the media module usually share physical memory, and the operating system does not have a memory recovery mechanism for shared memory. Summary of the invention

[0004] The embodiments of the present application provide a memory management method, a memory management device, and an electronic device, which can recycle the cache involved in video decoding when the video playback enters the background. If the application playing the video subsequently returns to the foreground, it will not affect the normal playback of the video, thereby improving the user experience.

[0005] In a first aspect, a memory management method is provided for an electronic device, comprising: receiving a first input from a user, the first input being used to cause a first application to enter a background state; intercepting an interface related to access to a cache in a first cache set, the cache in the first cache set being a cache related to decoding of a first video played by the first application; and reclaiming physical memory corresponding to the cache in the first cache set.

[0006] When the first video enters the background, the first video will be paused. In this embodiment, the physical memory corresponding to the cache related to the decoding of the first video can be recovered and the interface related to the corresponding cache access can be intercepted when the first video is in the background. In this way, even if multiple video applications are running in the background, the memory usage of the whole machine will be reduced, the response speed of the electronic device to the user will be faster, and the user experience will be improved.

[0007] In combination with the first aspect, in some implementations of the first aspect, the method also includes: receiving a second input from the user, wherein the second input is used to return the first application to the foreground state; reallocating physical memory for the cache in the first cache set; and unblocking the interface related to access to the cache in the first cache set.

[0008] In this embodiment, after the first application returns to the foreground, the first video will automatically start playing or continue playing after user operation. At this time, it is necessary to reallocate physical memory for the cache of reclaimed physical memory, and release the interception of related interfaces after reallocating the physical memory, so as not to affect the continued playback of the first video.

[0009] Optionally, after the first video enters the background, the first application may be cleared by the Android memory clearing mechanism or manually by the user, and the first application needs to be returned to the foreground by a cold start. At this time, the cache of the physical memory does not need to be reallocated.

[0010] In combination with the first aspect, in some implementations of the first aspect, the first cache set includes at least one of a second cache, a third cache, a fourth cache, and a fifth cache, wherein the second cache is an empty input cache held by a media module and in an input cache queue, the third cache is an output cache held by the media module and being filled with decoding results, the fourth cache is an output cache held by the media module and in an output cache queue and filled with decoding results to be used by the first application, the fifth cache is an output cache filled with decoding results held by the first application, and the media module is used to decode the first video.

[0011] In this embodiment, the cache in a specific state may be reclaimed according to the state of the cache related to the first video decoding, and physical memory reclaim may be performed according to different reclaiming strategies for caches in different states.

[0012] In some embodiments, the physical memory corresponding to the cache can be recycled according to the memory state classification of the electronic device. For example, the electronic device is divided into a first memory tension state and a second memory tension state, and the memory tension degree of the first memory tension state is lower than that of the second memory tension state. When the electronic device is in the first memory tension state, only the physical memory corresponding to the cache held by the media module in the second cache to the fifth cache is recycled; when the electronic device is in the second memory tension state, the fifth cache held by the first application is recycled. Other strategies can also be used, such as dividing the electronic device into more levels according to the memory tension state, and recycling different memories for different levels, or recycling the physical memory corresponding to the same cache for different levels, but different recycling methods can be used. For example, for the third cache mentioned above, if the memory is very tight, the physical memory can be recycled by re-decoding. If the memory state is not particularly tight, the decoding result can be stored and recycled.

[0013] In combination with the first aspect, in some implementations of the first aspect, intercepting the interface related to access to the cache in the first cache set includes: when at least one process or thread of the first application uses the interface related to access to the cache in the first cache set, freezing at least one process or thread of the first application; and releasing the interception of the interface related to access to the cache in the first cache set includes: unfreezing at least one process or thread of the first application.

[0014] In this embodiment, the electronic device can monitor the interface that needs to be intercepted, for example, by setting a code at the corresponding interface to organize the process or thread to use the corresponding interface. When the corresponding process or thread of the first application accesses the corresponding interface, the process or thread is directly frozen, and after the physical memory is reallocated, the corresponding process or thread can be unfrozen.

[0015] Optionally, for interception of some applications, an error code (for example, -EAGAIN) may be directly returned when the application accesses the corresponding interface. These applications will not regard the media module as having an error upon receiving the error code and will continue to use the media module.

[0016] In combination with the first aspect, in some implementations of the first aspect, when the first cache set includes a third cache, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: recording the corresponding decoding position of the decoding result filled in the third cache, and reclaiming the physical memory corresponding to the third cache; and the reallocation of physical memory for the cache in the first cache set includes: reallocating physical memory for the third cache, and filling the reallocated physical memory with the data corresponding to the decoding result after re-decoding the data according to the decoding position; or, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: compressing the decoding position filled in the third cache. As a result, the physical memory corresponding to the third cache is reclaimed; and the reallocation of physical memory for the caches in the first cache set includes: reallocating physical memory for the third cache, decompressing the compressed decoding result and filling it into the reallocated physical memory; or, the reclaiming physical memory corresponding to the caches in the first cache set includes: swapping out the decoding result filled in the third cache through page swapping, and reclaiming the physical memory corresponding to the third cache; and the reallocation of physical memory for the caches in the first cache set includes: reallocating physical memory for the third cache, swapping the swapped-out decoding result into the reallocated physical memory.

[0017] In this embodiment, the third cache is an output cache held by the media module and is being filled with decoding results. The cache can be recycled by directly discarding the decoding results and re-decoding them during reconstruction, or by storing the decoding results and filling the stored decoding results into the reconstructed physical memory after the physical memory is reconstructed. The decoding results can be stored in other locations in the memory, in a memory swap area (or swap space), or in an external storage medium, such as a hard disk or disk of an electronic device, in a compressed or uncompressed manner.

[0018] In combination with the first aspect, in some implementations of the first aspect, when the first cache set includes a fourth cache, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: compressing the decoding results filled in the fourth cache, and reclaiming the physical memory corresponding to the fourth cache; and the reallocation of physical memory for the cache in the first cache set includes: reallocating physical memory for the fourth cache, decompressing the compressed decoding results and filling them into the reallocated physical memory; or, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: swapping out the decoding results in the fourth cache through page swapping, and reclaiming the physical memory corresponding to the fourth cache; and the reallocation of physical memory for the cache in the first cache set includes: reallocating physical memory for the fourth cache, and swapping the swapped-out decoding results into the reallocated physical memory.

[0019] In this embodiment, the fourth cache is a cache filled with decoding results and to be consumed by the first application. The decoding results stored in the fourth cache cannot be decoded again after the memory is rebuilt, so it is necessary to recycle the physical memory after storage. It can also be stored in a compressed or uncompressed manner and then refilled into the physical memory after the physical memory is rebuilt.

[0020] In combination with the first aspect, in some implementations of the first aspect, when the first cache set includes a fifth cache, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: discarding the decoding results filled in the fifth cache, and reclaiming the physical memory corresponding to the fifth cache; or, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: compressing the decoding results filled in the fifth cache, and reclaiming the physical memory corresponding to the fifth cache; and, the reallocation of physical memory for the cache in the first cache set includes: reallocating physical memory for the fifth cache, decompressing the compressed decoding results and filling them into the reallocated physical memory; or, the reclaiming of the physical memory corresponding to the cache in the first cache set includes: swapping out the decoding results in the fifth cache through page swapping, and reclaiming the physical memory corresponding to the fifth cache; and, the reallocation of physical memory for the cache in the first cache set includes: reallocating physical memory for the fifth cache, and swapping the swapped-out decoding results into the reallocated physical memory.

[0021] In this embodiment, for the output cache filled with decoding results held by the first application, physical memory can be recovered by directly discarding the data in the output cache, and the data will not be restored after the memory is rebuilt. In this way, when the first video continues to play, the first application passes the fifth cache to the media module for display. The data sent for display is lost, which may cause temporary freezes in the first video. However, since the discarded data is not much, it has little impact on the user's video viewing experience. It can be implemented when the memory of the electronic device is relatively tight, thereby freeing up more physical memory.

[0022] Optionally, the decoding result in the fifth cache may be stored in a compressed or uncompressed manner, and the data may be restored after the physical memory is rebuilt, thereby reducing the impact on the user's video experience.

[0023] In combination with the first aspect, in some implementations of the first aspect, the interface related to accessing the cache in the first cache set includes a first interface and a second interface, the first interface is used by the first application to obtain the second cache, and the second interface is used by the first application to obtain the fourth cache.

[0024] Optionally, the interface related to cache access in the first cache set also includes a third interface, and the third interface is used for the first application to release the fifth cache.

[0025] In this embodiment, different interfaces can be intercepted according to the cache of the physical memory to be recovered. When the physical memory corresponding to the fifth cache needs to be recovered, the third interface can be intercepted. When the physical memory corresponding to the fifth cache does not need to be recovered, only the first interface and the second interface can be recovered.

[0026] In combination with the first aspect, in some implementations of the first aspect, when the first application uses the media module in synchronous mode, the first interface is used by the first application to actively obtain the second cache, and the second interface is used by the first application to actively obtain the fourth cache; when the first application uses the media module in asynchronous mode, the first interface is used by the first application to obtain the second cache actively released by the media module, and the second interface is used by the first application to obtain the fourth cache actively released by the media module.

[0027] In combination with the first aspect, in some implementations of the first aspect, before reclaiming the physical memory corresponding to the cache in the first cache set, the method further includes: waiting for a preset time period.

[0028] In this embodiment, the preset duration can be the duration between the moment when the media module learns that the first application enters the background state and determines the cache of the physical memory to be reclaimed. Due to the waiting preset duration, the media module can continue to process for a period of time, such as 200ms, so that more caches can be recycled. For example, during the preset duration, the data to be decoded in the sixth cache can be consumed by the media module, and the sixth cache is the input cache filled with data to be decoded and passed to the media module by the first application. Thus, the sixth cache becomes an empty input cache, which can increase the number of caches of the physical memory that can be recycled, reclaim more physical memory, and improve the user experience.

[0029] In a second aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the memory management method described in the first aspect and any possible implementation thereof to be executed.

[0030] In a third aspect, a memory management device is provided, comprising a module for implementing the memory management method as described in the first aspect and any possible implementation manner thereof.

[0031] In a fourth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the memory management method described in the first aspect and any possible implementation thereof is executed.

[0032] In a fifth aspect, a readable storage medium is provided, wherein instructions are stored in the readable storage medium. When the instructions are executed on a device, the memory management method as described in the first aspect and any possible implementation thereof is executed.

[0033] In a sixth aspect, a program product is provided, the program product comprising a program code, and when the program code runs on a device, the memory management method as described in the first aspect and any possible implementation thereof is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0035] Figure 2 It is a schematic diagram of the software structure of the electronic device provided in the embodiment of the present application.

[0036] Figure 3 It is a schematic diagram of the caching cycle process when the first application plays a video.

[0037] Figure 4 It is a schematic flowchart of a memory management method provided in an embodiment of the present application.

[0038] Figure 5 It is a flowchart of reclaiming physical memory and rebuilding physical memory.

[0039] Figure 6 It is a schematic diagram of the cache involved in the first video decoding process provided in an embodiment of the present application.

[0040] Figure 7 It is a schematic diagram of an interface used by a first application to acquire and release an input and output buffer in synchronous mode and asynchronous mode.

[0041] Figure 8 It is a schematic flow chart of the interface interception process.

[0042] Fig. 9 It is a schematic block diagram of a memory management device provided in an embodiment of the present application.

[0043] Fig.10 is a schematic block diagram of a memory management device provided in an embodiment of the present application DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0045] Figure 11 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0046] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0047] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0048] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0049] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0050] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0051] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL).

[0052] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170.

[0053] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface.

[0054] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160.

[0055] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .

[0056] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc.

[0057] The USB interface 130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device.

[0058] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0059] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0060] The power management module 141 is used to connect the battery 142 , the charging management module 140 and the processor 110 .

[0061] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0062] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the electronic device 100 .

[0063] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0064] The wireless communication module 160 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0065] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0066] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0067] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), or a display panel made of one of the materials such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0068] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0069] The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture still images or videos.

[0070] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0071] The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs.

[0072] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 .

[0073] The internal memory 121 may be used to store computer executable program codes, wherein the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121 .

[0074] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0075] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0076] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0077] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0078] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0079] The earphone jack 170D is used to connect a wired earphone.

[0080] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0081] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .

[0082] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0083] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).

[0084] The distance sensor 180F is used to measure the distance.

[0085] The fingerprint sensor 180H is used to collect fingerprints.

[0086] The touch sensor 180K is also called a “touch panel.” The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen.”

[0087] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 180M can also contact the human body's pulse to receive blood pressure beating signals.

[0088] Buttons 190 include a power button, a volume button, and the like.

[0089] Motor 191 can generate vibration prompts.

[0090] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0091] The SIM card interface 195 is used to connect a SIM card.

[0092] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0093] Figure 2 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

[0094] The first application in the embodiment of the present application may be located at the application layer.

[0095] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0096] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0097] The media module in the embodiment of the present application may be located at the application framework layer.

[0098] like Figure 2 As shown, the application framework layer may include window manager service (WMS), content provider, activity manager service (AMS), package manager service (PMS), view system, phone manager, resource manager, notification manager, etc.

[0099] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0100] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0101] Activity management services are used to manage activities, such as activity lifecycle management.

[0102] Package management services can be used to manage the installation and uninstallation of applications.

[0103] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0104] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0105] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0106] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0107] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0108] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0109] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0110] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0111] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0112] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0113] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0114] A 2D graphics engine is a drawing engine for 2D drawings.

[0115] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0116] Before introducing the technical solution of the present application, some professional terms that may be involved in the present application solution are briefly introduced below.

[0117] Activity: A component that can contain a user interface and is mainly used to interact with users, such as making calls, taking photos, sending emails, and viewing maps. Most of the content that users can see in an application is provided by the activity component. In an application, multiple activities can be included.

[0118] Task: usually refers to a collection of related activities, which are arranged in a stack in the order in which they are opened. The home screen of the electronic device is usually the starting point of the task.

[0119] Foreground: A foreground service performs actions that the user notices. For example, an audio app uses a foreground service to play an audio track. A foreground service often needs to display a notification. A foreground service continues to run even if the user stops interacting with the app.

[0120] Background: Background services perform actions that are not directly noticed by the user. For example, if an app uses a service to compress its storage space, this service is usually a background service.

[0121] To facilitate understanding, the following first introduces concepts related to the memory management mechanism of the electronic system that may be involved in the embodiments of the present application.

[0122] 1. Page

[0123] In the memory management process, a page is the smallest unit of the address space, and a page is usually 4 kilobytes (KB) or 64KB.

[0124] 2. Logical memory and physical memory

[0125] The memory space of a process is only virtual memory, or "logical memory", while the program needs real physical memory, or random access memory (RAM). When necessary, the operating system can map the logical memory requested during program operation to RAM, allowing the process to use physical memory.

[0126] 3. Memory Recycling

[0127] Regarding memory recycling, the operating system needs to perform recycling actions under the premise of ensuring that the application data is not lost. There are generally three ways to recycle pages: page writeback, page swapping, and page discarding: 1) Page writeback, if the backup storage of a physical page is a block device (such as file mapping), the memory can be directly synchronized to the block device, and the freed page can be reused; 2) Page swapping, if the physical page has no backup storage (such as anonymous mapping), the stored data can be swapped to a specific swap partition (the system usually allocates a storage space on the disk as a memory cache, and this part of the storage space is generally called the swap area), and then swapped back to the memory when it is accessed again; 3) Page discarding, if the backup storage of the physical page is a file, but the file content cannot be modified in the memory (such as an executable file), then it can be directly discarded if it is not needed at the moment.

[0128] After the application enters the background, when the memory is tight, the operating system can reclaim memory in various ways, such as page discarding, page swapping, page writeback, etc., to alleviate memory pressure.

[0129] In the video playback scenario, more cache is involved. Figure 3A schematic diagram of a cache cycle process when a first application plays a video is shown. The first application may be a video playback application. When playing a video, the first application first obtains an empty input buffer from a media module (S301), and fills the empty input buffer with data to be decoded; the first application returns the buffer filled with data to be decoded to the media module for processing (S302); after the media module obtains the data to be decoded, it fills the decoding result into the output buffer and returns it to the first application (S303); after the first application obtains and consumes the decoding result, it releases the output buffer to the media module (S304).

[0130] When the first application enters the background, the cache involved in playing the video by the first application will not be destructed, occupying the physical memory, so that the memory resources of the electronic device become less and less as the number of video applications used by users increases.

[0131] When playing a video, the first application usually shares memory with a hardware decoder driven by a media module. The media module is a software module that is used to call a hardware decoder to decode the video played by the first application. Exemplarily, the media module may include a MediaCodec component, an Acodec component, and an OpenMAX component. During the video decoding process, the MediaCodec component is used to connect to the application; the Acodec component and the OpenMAX component are specific implementations. The Acodec component is responsible for the entire decoding process, for example, the status of each cache can be marked, and the OpenMAX component is used to call the hardware decoder to decode the data to be decoded in the cache.

[0132] The media module will call the hardware decoder to directly access the physical memory, and the location of the physical memory is fixed. If the application does not actively inform that the memory can be released, the operating system does not have a mechanism to reclaim the physical memory for the shared memory of the application and the device (hardware decoder), and the application will not actively inform that the physical memory can be released when it exits the background. If the operating system actively reclaims the memory through page swapping and other methods when the application exits the background, when the media module subsequently calls the hardware decoder to access the physical memory, the hardware decoder will still access the original physical memory. At this time, the physical memory may be occupied by other processes, causing problems with the operating system. In addition, for shared memory, the shared memory is allocated and released by a dedicated module (such as the general memory manager ion). After the application exits the background, it will not inform the module which physical memory is no longer used, and the module cannot identify which physical memory needs to be reclaimed.

[0133] The embodiment of the present application provides a memory management method, which can recycle the physical memory related to video playback when the video enters the background, and when the video returns to the foreground, it will not affect the normal playback of the video, thereby improving the fluency of the operating system and the user experience. The memory management method includes:

[0134] S410, receiving a first input from a user, where the first input is used to cause a first application to enter a background state.

[0135] S420, intercepting an interface related to access to a cache in a first cache set, where the cache in the first cache set is a cache related to decoding of a first video played by a first application.

[0136] S430: reclaim the physical memory corresponding to the cache in the first cache set.

[0137] When the electronic device receives the first input from the user, it may be in a state of playing the first video on the interface of the first application. The first application may be a video application, and the first input is an input for causing the first application to enter the background. For example, it may be an input in which the user returns to the desktop or main interface by swiping up at the bottom of the display screen or by clicking the home button (virtual button, floating button, or physical button) on the video interface. The first input may also be an input in which the user brings up the multitasking interface on the current video playback interface so that the first application enters the background. For example, the user may bring up the multitasking interface by swiping up at the bottom of the display screen and pausing. The user may click on the windows of other applications on the multitasking interface to bring other applications to the foreground, and the first application enters the background and runs.

[0138] In an embodiment of the present application, the physical memory corresponding to part of the cache in the cache related to the video decoding played by the application can be reclaimed when the application enters the background state. When the first application enters the background, the electronic device can determine that the physical memory of the cache related to the first application can be reclaimed through the technical solution of the embodiment of the present application.

[0139] In some embodiments, the activity management service (AMS) can monitor the status of each application and determine that the first application is included in the application that calls the media module. When the first application enters the background, the AMS can continue to monitor the running status of the first application. The AMS needs to determine that there is no foreground activity when the first application enters the background. For example, when the first application enters the background, the first video will not continue to be played through the foreground floating window, and it is determined that the playback of the first video will not continue to be played in the background through audio. This can be determined, for example, by the priority of the first application, whether it is in the playback state, etc. When there is no foreground activity when the first application enters the background, the AMS can determine that the first application is an application that can reclaim physical memory according to the technical solution provided in the embodiment of the present application.

[0140] In other embodiments, AMS can monitor the status of applications and determine that the applications that call the media module to play the video include the first application, and can determine in advance based on the configuration information of the first application that when the first application enters the background, it will not continue to play through the foreground floating window, background audio playback, etc. For example, a first list can be set, and the first list includes the identifiers of applications that can implement the technical solution of the embodiment of the present application to recycle physical memory after entering the background. Therefore, when the first application enters the background state, the media module is directly called through the first application and the identifier of the first application is in the list, and the technical solution of the embodiment of the present application is executed.

[0141] After AMS determines that the first application has entered the background and the first application can reclaim physical memory, AMS can instruct the media module to reclaim the physical memory corresponding to the cache related to the first video decoding. For example, AMS notifies the media module through a first message, so that the media module can intercept the corresponding interface according to S420 and S430 and reclaim the physical memory corresponding to the cache in the first cache set, and the first cache set belongs to the cache related to the first video decoding played by the first application. The first cache set may include a portion of the cache held by the media module. Exemplarily, the first cache set may include at least one of the second cache, the third cache, and the fourth cache, the second cache is an empty input cache held by the media module in the input cache queue, the third cache is an output cache held by the media module that is being filled with decoding results, and the fourth cache is an output cache held by the media module that is in the output cache queue and is filled with decoding results to be used by the first application.

[0142] In addition to the cache held by the media module, the first cache set may also include a fifth cache held by the first application, the fifth cache being an output cache filled with decoding results held by the first application. That is, the first cache set includes at least one of the second to fifth caches.

[0143] The second cache can correspond to Figure 3 The fourth cache may correspond to the output cache 317 filled with decoding results held by the media module to be consumed by the first application, and the fifth cache may be the output cache 313 or 314 filled with decoding results held by the first application. The recycling process of the second cache to the fifth cache will be described in detail later.

[0144] The interface intercepted in S420 may be related to the type of cache corresponding to the physical memory to be reclaimed, which will be described in detail later in conjunction with the introduction of the cache type. Specifically, the intercepted interface may monitor the access to the corresponding interface. When the process corresponding to the first application calls or uses the interface, the corresponding process or thread of the first application calling or using the interface may be frozen, thereby avoiding the occurrence of use after free (UAS) causing system problems. After the corresponding interface is intercepted, the physical memory corresponding to the first cache set may be reclaimed.

[0145] When the electronic device receives the user's second input, the AMS can determine that the second input is used to return the first application to the foreground state, so that the media module can be notified to rebuild the physical memory, for example, by notifying the media module through a second message, so that the media module can reallocate physical memory for the recovered cache, and after the allocation, release the interception of the corresponding interface, so that the first video re-enters the normal playback state.

[0146] Of course, after the physical memory corresponding to the cache in the first cache set is reclaimed, if the first application is cleaned up by the operating system through a memory cleaning mechanism, such as the Android preset memory cleaning mechanism (LMK, low memory killer) or is manually cleaned up by the user on the multitasking interface, the first application will not return to the foreground before the next cold start, and there is no need to perform the physical memory reconstruction step.

[0147] In this way, through the above steps, part of the cached physical memory involved in the first video decoding process can be recovered. If the video returns to the foreground again, it will not affect the normal playback of the video, so that the memory of the entire machine is increased and the system fluency is improved.

[0148] Figure 5 FIG. 1 shows a flowchart of the entire process of reclaiming physical memory and rebuilding physical memory. Figure 5 As shown, including:

[0149] S510, the AMS notifies the media module that the first application has exited the background.

[0150] S520, the media module intercepts the interface.

[0151] In S520, the media module intercepts the corresponding interface according to the recycling strategy. The corresponding details of the intercepted interface will be described in detail later.

[0152] S530: The media module identifies and records a cache list of reclaimed physical memory.

[0153] S540: The media module reclaims physical memory according to the cache list.

[0154] S550: The media module feeds back to the AMS that the physical memory is successfully reclaimed.

[0155] If memory recycling fails, the physical memory may not be recycled or, in a memory shortage state, memory cleaning may be performed according to a preset memory cleaning mechanism.

[0156] S560: The AMS notifies the media module that the first application returns to the foreground.

[0157] S570: The media module performs physical memory reconstruction.

[0158] S580: The media module releases the interception of the interface.

[0159] S590: The media module feeds back to the AMS that the physical memory reconstruction is successful.

[0160] In order to specifically introduce the specific details of the technical solution provided by the embodiments of the present application, Figure 6 The specific situation of the cache involved in the first video decoding process is shown.

[0161] like Figure 6 As shown, before Android Q, the media module includes three components, namely MediaCodec component, Acodec component, and OpenMAX component. During the video decoding process, the MediaCodec component calls the OpenMAX component through the Acodec component, thereby calling the hardware decoder to implement hardware encoding and decoding. Codec2.0 is launched on Android Q, replacing the Acodec component and OpenMAX component. The following only introduces the specific situation of the cache involved in the first video decoding process based on the media module including MediaCodec component, Acodec component, and OpenMAX component.

[0162] The cache involved in the first video decoding process can be divided into a cache held by the first application, a cache held by the media module, and a cache held by the display module according to their subordination. Figure 6 The schematic diagram shows that the first application holds two caches (cache 501 and cache 502), the media module holds five caches (cache 503, cache 504, cache 505, cache 506 and cache 507), and the display module holds one cache (cache 508). The cache held by the display module is usually transferred from the media module to the display module for rendering and display. After the first application obtains the cache decoded by the media module, it can transfer the decoded cache to the media module at a certain speed, and the media module sends it to the display, thereby controlling the video playback speed.

[0163] The MediaCodec component of the media module can mark who uses or owns the cache. The cache can be marked as owned by the first application through "mOwnedByClient=true".

[0164] The Acodec component of the media module marks the status of each cache. The cache can be marked as an input cache held by the MediaCodec component or an input cache held by the first application through "OWNED_BY_UPSTREAM", the cache can be marked as an output cache held by the MediaCodec component or an output cache held by the first application through "OWNED_BY_DOWNSTREAM", the output cache is filled with decoding results, the cache is marked as being held by the OpenMAX component through "OWNED_BY_COMPONENT", for example, being decoded by the OpenMAX component, the cache is marked as being processed by the Acodec component through "OWNED_BY_US", and the cache is marked as being held by the display module through "OWNED_BY_NATIVE_WINDOW".

[0165] The cache held by the first application may include an input cache and an output cache. Figure 6 Schematic diagram of an input cache (cache 501) and an output cache (cache 502, a fifth cache) held by a first application. The fifth cache may be as follows: Figure 3 313 or 314. Cache 501 is marked as "input buffer mOwnedByClient = true" and "OWNED_BY_UPSTREAM", indicating that the cache is an input buffer and is owned by the first application, and the cache has been obtained by the first application through the "dequeueInputBuffer" interface. Cache 502 is marked as "output buffer mOwnedByClient = true" and "OWNED_BY_DOWNSTREAM", indicating that the cache is an output buffer and is owned by the first application, and the first application has obtained the cache through the "dequeueOutputBuffer" interface.

[0166] The display module holds the cache such as Figure 6 As shown in the cache 508, it is marked as "OWNED_BY_NATIVE_WINDOW" and can be passed by the media module to the graphics processing unit (GPU) for rendering and display.

[0167] The buffer held by the media module may include an input buffer in the input queue, and the buffer in the input buffer queue may include an empty input buffer (second buffer). The empty input buffer may be as follows: Figure 6 Cache 503 as shown (or can also be as shown Figure 3 315), the buffer 503 is marked as "input buffer available" and "OWNED_BY_UPSTREAM", indicating that the buffer is an available empty input buffer, which is held by the media module. The first application can obtain the buffer through the "dequeueInputBuffer" interface and fill it with data to be decoded. The input buffer can also include a buffer filled with data to be decoded (a sixth buffer, such as Figure 3 shown in 316), Figure 6 Not shown is the cache when the media module has just received the cache filled with the data to be decoded by the first application but has not yet used the data to be decoded. The cache will be quickly passed to the Acodec component for processing.

[0168] The buffer held by the media module may also include an output buffer. The output buffer may include an output buffer (fourth buffer) filled with decoding results in the output buffer array to be acquired by the first application, such as Figure 6 504 (or as shown in Figure 3 317), cache 504 is marked as "output buffer available" and "OWNED_BY_DOWNSTREAM", indicating that the cache is an output buffer filled with decoding results and is held by the media module. The cache can be obtained by the first application through the "dequeueOutputBuffer" interface to obtain the decoding results. The output buffer can also include an output buffer (third cache) that is being filled with decoding results. This type of output buffer can be held by an OpenMAX component, as shown in cache 506 and cache 507, which are marked as "output buffer" and "OWNED_BY_COMPONENT", indicating that the cache is an output buffer and is held by an OpenMAX component. The output cache can also include a cache (seventh cache) that is returned to the media module after being obtained by the first application. Figure 6 Not shown, it can be Figure 3 As shown in 318, after the cache is obtained by the MediaCodec component of the media module, it will be quickly passed to the Acodec component and then sent to the display module for rendering and display.

[0169] The buffer held by the media module may also include the buffer being processed by the Acodec component, such as Figure 6 As shown in cache 505, cache 505 is marked as "OWNED_BY_US", indicating that the cache is being processed by the Acodec component.

[0170] The first cache set may include one or more of the second cache, the third cache, the fourth cache, and the fifth cache, and the interface intercepted by the electronic device is related to the cache in the first cache set for reclaiming the physical memory.

[0171] In an embodiment of the present application, when the first application enters the background and the first video stops playing, the decoding activity of the first video stops, and the OpenMAX component no longer passes the input cache or output cache to the Acodec component. Therefore, the intercepted interface may include an interface for the first application to obtain the input cache or output cache, thereby reducing the possibility that an error occurs when the first application requests the input or output cache due to most of the caches being held by the OpenMAX component (there is no input cache or output cache in the input and output cache queue), thereby causing the first application to no longer use the media module, resulting in problems with video playback.

[0172] Specifically, when the first cache set includes the second cache, the second cache is an empty input cache in the input queue, that is, an empty input cache held by the media module, such as Figure 6 As shown in the schematic cache 503. Since the cache in the input queue does not store any data, the physical memory corresponding to the cache can be directly reclaimed. When the first application returns to the foreground and needs to rebuild the physical memory corresponding to the second cache, it does not involve data recovery, and the physical memory can be directly allocated.

[0173] When the first cache set also includes a third cache, the third cache is a cache that is being filled with decoding results, or can also be called a cache held by an OpenMAX component or a cache that is partially filled with decoding results. Since the third cache is filled with decoding results, the subsequent recovery of the decoding results needs to be considered when reclaiming the physical memory.

[0174] In one embodiment, the decoding result can be saved and then the physical memory corresponding to the third cache can be recovered. For example, the decoding result can be compressed, and the compressed decoding result can be swapped to the swap area in the memory or the external storage medium through the page swap process, or the compressed decoding result can be transferred to other locations in the memory (for example, if the decoding result is compressed from 4KB to 2KB, the compressed decoding result can be transferred from the original physical memory location corresponding to the cache to the physical memory of 2KB in the memory), and then the physical memory corresponding to the third cache can be recycled; after the physical memory is reallocated to the third cache, the compressed decoding result can be filled into the reallocated physical memory, and the compressed decoding result can be decompressed. For another example, the decoding result can be not compressed, but directly swapped to the swap area of ​​the memory or the external storage medium through the form of page swap, and after the physical memory is reallocated to the third cache, the decoding result can be swapped into the reallocated physical memory.

[0175] In another embodiment, the decoding result in the third cache may be discarded, and instead the decoding position of the third cache may be recorded. After reallocating the physical memory, the OpenMAX component may re-decode the data according to the decoding position and fill the decoding result into the reallocated physical memory.

[0176] When the first cache set can include the fourth cache, the fourth cache is a cache in the output cache queue, which is a cache filled with decoding results to be acquired or used by the first application. Before reclaiming the physical memory corresponding to the fourth cache, it is also necessary to consider restoring the original data after rebuilding the physical memory. Since the decoding result cannot be decoded again after the first application returns to the foreground, the decoding result in the fourth cache needs to be saved.

[0177] The decoding result can be directly saved after compression or without compression, just like the third cache. For example, the decoding result is compressed, and the physical memory corresponding to the fourth cache is reclaimed after being stored in other locations; after the physical memory is reallocated to the fourth cache, the compressed data can be decompressed and filled into the reallocated physical memory. For another example, the decoding result can be directly swapped out of the current physical memory by page swapping, such as swapping to a memory swap area or an external storage medium, and then swapped into the allocated physical memory after the physical memory is reallocated.

[0178] When the first cache set includes the fifth cache, in one embodiment, the decoding result in the fifth cache can be directly discarded and the physical memory corresponding to the fifth cache can be recycled. After the first application returns to the foreground, the physical memory is reallocated for the fifth cache. In this way, when the first video continues to play, the decoding result in the fifth cache held by the first application is discarded and will not be displayed, and some frames of the first video are discarded, which may cause a sense of lag when the video just resumes playing.

[0179] In another embodiment, the fifth cache may be recycled by compressed or uncompressed page swapping, similar to the fourth cache. The specific process may refer to the above description and will not be repeated here.

[0180] When the first cache set includes the fifth cache, it is necessary to intercept the interface of the first application releasing the fifth cache, so as to avoid the possibility that after the first application releases the fifth cache, the media module accesses the fifth cache and causes system problems.

[0181] In an embodiment of the present application, a recycling strategy can be determined according to the current memory occupancy. For example, when the electronic device is in a first memory stress state (for example, the occupancy rate is greater than a threshold, the memory occupancy is greater than a threshold, the remaining memory is less than a threshold, and the memory remaining rate is less than a threshold), the first cache set only includes the cache held by the media module in the second cache to the fifth cache. When the electronic device is in a second memory stress state, the first cache set also includes the cache held by the first application in the second cache to the fifth cache, wherein the memory in the second memory stress state is more tense than the first memory stress state. In this way, a suitable physical memory recycling strategy can be selected according to the memory state of the electronic device, and more physical memory can be recycled when the memory is more tense. Of course, the above strategy is only schematically shown. For example, more levels of memory stress states can be set for the electronic device, such as adding a third memory stress state, and the memory stress degree of the third memory stress state is lower than that of the first memory stress state and the second memory stress state, and when the first memory stress state is in the second memory stress state, only the second cache can be recycled, and the second cache does not need to recover data when recycling, making the implementation of the memory recycling strategy simpler. For another example, different recycling methods can be used to recycle the cache in the same state under different memory shortage conditions. Taking the fifth cache as an example, in the first memory shortage state, the decoding result in the fifth cache can be directly discarded for recycling, while in the second memory shortage state, the decoding result in the fifth cache can be stored in, for example, a memory swap area for recycling. The specific strategy used for memory recycling can be determined according to actual conditions, and will not be described in detail here.

[0182] Before reclaiming the physical memory corresponding to the cache, the media module can determine the status of each cache, and set a cache reclaim list according to the cache status, and after reclaiming, perform a physical memory reconstruction process of the cache of the reclaimed physical memory according to the list.

[0183] In addition, in an embodiment of the present application, when the first application enters the background, the media module can first start to intercept the corresponding interface according to the recycling strategy, and wait for a preset time before reclaiming the memory. The preset time can be determined based on experience or historical processing data of the media module. For example, 200ms, so that the media module continues to process for part of the time. It is also possible to wait for a preset time before determining the cache status. Within the preset time, for example, the media module can consume the data in the cache (sixth cache) filled with data to be decoded that is passed by the first application, so that the cache becomes an empty input cache; for example, within the preset time, the Acodec component can pass the processed output cache filled with decoding results to the MediaCodec component, so that the cache can become a cache that can reclaim physical memory (fourth cache). In this way, the categories of caches for which physical memory needs to be reclaimed are reduced, and more physical memory corresponding to caches can be reclaimed.

[0184] The above introduces the cache categories that can be recycled by the technical solution of the embodiment of the present application. For different recycling strategies, the media module needs to intercept the corresponding interface to prevent the first application from accessing the address of the cache of the released physical memory.

[0185] Specifically, when the first application interacts with the media editing module, the input buffer or the output buffer may be obtained through the synchronous mode and the asynchronous mode. Figure 7 (a) shows the interface used by the first application to obtain and release the input and output buffers in synchronous mode. In synchronous mode, the first application actively obtains the input buffer and output buffer from the media module. The first application obtains the empty input buffer (second buffer) from the media module through the dequeueInputBuffer() interface, and obtains the output buffer (fourth buffer) filled with decoding results from the media module through the dequeueOutputBuffer() interface. The media module returns a buffer identifier (buffer index) to the first application so that the first application can obtain the corresponding input or output buffer. After the first application fills the empty input buffer with the data to be decoded, it can pass the data to be decoded to the media module through the queueInputBuffer() interface; and after the first application uses the output buffer filled with decoding results, it can release the output buffer (fifth cache) through the releaseOutputBuffer() interface.

[0186] Figure 7 (b) shows the interface used by the first application to obtain and release the input and output buffers in asynchronous mode. In asynchronous mode, the media module actively releases the available buffer to the first application. When the media module determines that there is an idle input buffer, it can release the input buffer to the first application through the onInputBufferAvailable() interface. After the first application obtains the input buffer, it fills it with the data to be decoded and passes the buffer to the media module through the queueInputBuffer() interface. After the media module completes decoding, the output buffer filled with the decoding result is released to the first application through the onOutputBufferAvailable() interface. After the first application uses the decoding result, it can release the output buffer through the releaseOutputBuffer() interface.

[0187] In synchronous mode, the dequeueInputBuffer() interface and the dequeueOutnputBuffer() interface may be intercepted to prevent the first application from actively acquiring the input and output buffers.

[0188] In asynchronous mode, you can intercept the onInputBufferAvailable() interface and onOutputBufferAvailable() interface to prevent the media module from actively releasing the input and output buffers.

[0189] When the first cache set includes (the fifth cache), the interface for the first application to release the cache can be intercepted. Specifically, the releaseOutputBuffer() interface can be intercepted to prevent the media module from accessing the corresponding address after the first application releases the output buffer.

[0190] As mentioned above, the above interception corresponding interface can freeze the process or thread of the first application using the interface, so as to avoid directly returning an error code and causing some applications to think that the media module has an error and stop using it. Of course, if the application does not think that the media module has an error due to the return of an error code and continues to use it, the error code can be directly returned to the relevant application without freezing the corresponding process or thread of the application.

[0191] Figure 8 The interception interface process includes the following steps:

[0192] S1, the first application uses the corresponding interface to request or release the cache.

[0193] The corresponding interface can be determined according to the strategy adopted by the first application to reclaim physical memory, and the interface includes an interface for the first application to obtain input or output buffer, and can also include an interface for the first application to release output buffer. For example, when the first application reclaims physical memory and does not need to reclaim the fifth buffer mentioned above, the corresponding interface does not include the releaseOutputBuffer() interface.

[0194] S2: Determine whether the first application is in the background.

[0195] S3: When the first application is in the background, freeze the process or thread using the interface.

[0196] When the first application is running in the background, the physical memory corresponding to the cache of the first application has been reclaimed, and the process or thread using the interface can be frozen.

[0197] S4, the first application returns to the foreground.

[0198] S5, unfreeze the corresponding process or thread.

[0199] S6: When the first application is not in the background, allocate a cache or allow the first application to release the cache according to the interface used by the first application.

[0200] Fig. 9A memory management device 600 provided in an embodiment of the present application is shown, and the memory management device 600 includes: a first application 610 and a media module 620; optionally, the memory management device 600 may also include an activity management service.

[0201] Specifically, the memory management device 600 includes: a first application 610, used to receive a first input from a user, and the first input is used to make the first application 610 enter a background state; a media module 620, used to: intercept an interface related to access to a cache in a first cache set, and the cache in the first cache set belongs to a cache related to decoding of a first video played by the first application 610; and reclaim the physical memory corresponding to the cache in the first cache set.

[0202] In an embodiment of the present application, the first application 610 is also used to receive a second input from the user, and the second input is used to return the first application 610 to the foreground state; the media module 620 is also used to reallocate physical memory for the cache in the first cache set; and release the interception of the interface related to the access to the cache in the first cache set.

[0203] In an embodiment of the present application, the first cache set includes at least one of a second cache, a third cache, a fourth cache, and a fifth cache, wherein the second cache is an empty input cache held by the media module 620 and in an input cache queue, the third cache is an output cache held by the media module 620 and being filled with decoding results, the fourth cache is an output cache held by the media module 620 and in an output cache queue and filled with decoding results to be used by the first application 610, the fifth cache is an output cache filled with decoding results held by the first application 610, and the media module 620 is used to decode the first video.

[0204] In an embodiment of the present application, the media module 620 is specifically used to: freeze at least one process or thread of the first application 610 when at least one process or thread of the first application 610 uses an interface related to accessing a cache in a set in the first cache; and unfreeze at least one process or thread of the first application 610.

[0205] In an embodiment of the present application, when the first cache set includes a third cache, the media module 620 is specifically used to: record the corresponding decoding position of the decoding result filled in the third cache, and reclaim the physical memory corresponding to the third cache; and reallocate physical memory for the third cache, and according to the decoding position, re-decode the data corresponding to the decoding result and fill it into the reallocated physical memory; or, compress the decoding result filled in the third cache, and reclaim the physical memory corresponding to the third cache; and, reallocate physical memory for the third cache, and decompress the compressed decoding result and fill it into the reallocated physical memory; or, swap out the decoding result filled in the third cache through page swapping, and reclaim the physical memory corresponding to the third cache; and, reallocate physical memory for the third cache, and swap the swapped out decoding result into the reallocated physical memory.

[0206] In an embodiment of the present application, when the first cache set includes a fourth cache, the media module 620 is specifically used to: compress the decoding results filled in the fourth cache, and reclaim the physical memory corresponding to the fourth cache; and reallocate physical memory for the fourth cache, and decompress the compressed decoding results and fill them into the reallocated physical memory; or, swap out the decoding results in the fourth cache through page swapping, and reclaim the physical memory corresponding to the fourth cache; and reallocate physical memory for the fourth cache, and swap the swapped out decoding results into the reallocated physical memory.

[0207] In an embodiment of the present application, when the first cache set includes the fifth cache, the media module 620 is specifically used to: discard the decoding results filled in the fifth cache, and reclaim the physical memory corresponding to the fifth cache; or, compress the decoding results filled in the fifth cache, and reclaim the physical memory corresponding to the fifth cache; and, reallocate physical memory for the fifth cache, decompress the compressed decoding results and fill them into the reallocated physical memory; or, swap out the decoding results in the fifth cache through page swapping, and reclaim the physical memory corresponding to the fifth cache; and, reallocate physical memory for the fifth cache, and swap the swapped out decoding results into the reallocated physical memory.

[0208] In an embodiment of the present application, the interface related to accessing the cache in the first cache set includes a first interface and a second interface, the first interface is used by the first application 610 to obtain the second cache, and the second interface is used by the first application 610 to obtain the fourth cache.

[0209] In an embodiment of the present application, when the first application 610 uses the media module 620 in synchronous mode, the first interface is used by the first application 610 to actively obtain the second cache, and the second interface is used by the first application 610 to actively obtain the fourth cache; when the first application 610 uses the media module 620 in asynchronous mode, the first interface is used by the first application 610 to obtain the second cache actively released by the media module 620, and the second interface is used by the first application 610 to obtain the fourth cache actively released by the media module 620.

[0210] In an embodiment of the present application, before reclaiming the physical memory corresponding to the cache in the first cache set, the media module 620 is also used to: wait for a preset time length, where the preset time length is used for the media module 620 to consume data in a sixth cache, where the sixth cache is an input cache filled with data to be decoded and passed to the media module 620 by the first application 610.

[0211] In an embodiment of the present application, the device also includes an activity management service, which is used to: send a first message to the media module based on the first input, and the first message is used to indicate that the cache related to the first video decoding can be reclaimed; based on the second input, send a second message to the media module 620, and the second message is used to indicate that the first application returns to the foreground.

[0212] Fig.10 FIG. 7 shows a memory management device 700 provided in an embodiment of the present application. The memory management device 700 may be Fig. 9 A specific example of a memory management device in the embodiment of the present application. The memory management device 700 includes: a processor 720. In an embodiment of the present application, the processor 720 is used to implement corresponding control management operations, for example, the processor 720 is used to support the memory management device 700 to perform the method or operation or function of the aforementioned embodiment. Optionally, the memory management device 700 may also include: a memory 710 and a communication interface 730; the processor 720, the communication interface 730 and the memory 710 may be interconnected or interconnected through a bus 740. Among them, the communication interface 730 is used to support the electronic device to communicate with other devices, etc., and the memory 710 is used to store program codes and data of the electronic device. The processor 720 calls the code or data stored in the memory 710 to implement the corresponding operation. The memory 710 may be coupled to the processor or not. The coupling in the embodiment of the present application is an indirect coupling or communication connection between electronic devices, units or modules, which may be electrical, mechanical or other forms, and is used for information interaction between electronic devices, units or modules.

[0213] Among them, the processor 720 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in combination with the contents disclosed in this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The communication interface 730 can be a transceiver, circuit, bus, module or other type of communication interface. The bus 740 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0214] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0215] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0216] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0218] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0219] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0220] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A memory management method for an electronic device, characterized in that: include: receiving a first input from a user, where the first input is used to cause the first application to enter a background state; Intercepting an interface related to access to a cache in a first cache set, where the cache in the first cache set belongs to a cache related to decoding of a first video played by the first application; The physical memory corresponding to the cache in the first cache set is reclaimed.

2. The method according to claim 1, characterized in that The method further comprises: receiving a second input from a user, where the second input is used to return the first application to a foreground state; reallocating physical memory for caches in the first cache set; Unblocking the interface related to the access to the cache in the first cache set.

3. The method according to claim 2, characterized in that The first cache set includes at least one of a second cache, a third cache, a fourth cache, and a fifth cache, wherein the second cache is an empty input cache held by a media module and in an input cache queue, the third cache is an output cache held by the media module and being filled with decoding results, the fourth cache is an output cache held by the media module and in an output cache queue and filled with decoding results to be used by the first application, the fifth cache is an output cache filled with decoding results held by the first application, and the media module is used to decode the first video.

4. The method according to claim 2 or 3, characterized in that: The intercepting of the interface related to the access to the cache in the first cache set includes: freezing at least one process or thread of the first application when the at least one process or thread of the first application uses the interface related to the access to the cache in the first cache set; and / or, The unblocking of the interface related to the access to the cache in the first cache set includes: unfreezing at least one process or thread of the first application.

5. The method according to claim 3, characterized in that When the first cache set includes a third cache, The reclaiming of the physical memory corresponding to the cache in the first cache set includes: recording the decoding position corresponding to the decoding result filled in the third cache, and reclaiming the physical memory corresponding to the third cache; and the reallocation of the physical memory for the cache in the first cache set includes: reallocating the physical memory for the third cache, and re-decoding the data corresponding to the decoding result according to the decoding position and filling it into the reallocated physical memory; or, The reclaiming of the physical memory corresponding to the cache in the first cache set includes: compressing the decoding result filled in the third cache, and reclaiming the physical memory corresponding to the third cache; and the reallocating of the physical memory for the cache in the first cache set includes: reallocating the physical memory for the third cache, and decompressing the compressed decoding result and filling it into the reallocated physical memory; or, The reclaiming of the physical memory corresponding to the cache in the first cache set includes: swapping out the decoding results filled in the third cache through page swapping, and reclaiming the physical memory corresponding to the third cache; and the reallocating of physical memory for the cache in the first cache set includes: reallocating physical memory for the third cache, and swapping the swapped out decoding results into the reallocated physical memory.

6. The method according to claim 3 or 5, characterized in that When the first cache set includes a fourth cache, The reclaiming of the physical memory corresponding to the cache in the first cache set includes: compressing the decoding result filled in the fourth cache, and reclaiming the physical memory corresponding to the fourth cache; and the reallocating of the physical memory for the cache in the first cache set includes: reallocating the physical memory for the fourth cache, and decompressing the compressed decoding result and filling it into the reallocated physical memory; or, The reclaiming of the physical memory corresponding to the cache in the first cache set includes: swapping out the decoding result in the fourth cache through page swapping, and reclaiming the physical memory corresponding to the fourth cache; and the reallocating of physical memory for the cache in the first cache set includes: reallocating physical memory for the fourth cache, and swapping the swapped out decoding result into the reallocated physical memory.

7. The method according to any one of claims 1 to 6, characterized in that The interface related to access to the cache in the first cache set includes a first interface and a second interface, the first interface is used by the first application to obtain the second cache, and the second interface is used by the first application to obtain the fourth cache.

8. The method according to claim 7, characterized in that When the first application uses the media module in a synchronous mode, the first interface is used by the first application to actively obtain the second cache, and the second interface is used by the first application to actively obtain the fourth cache; When the first application uses the media module in asynchronous mode, the first interface is used by the first application to obtain the second cache actively released by the media module, and the second interface is used by the first application to obtain the fourth cache actively released by the media module.

9. A memory management device, characterized in that: include: A first application is used to receive a first input from a user, where the first input is used to make the first application enter a background state; Media module for: Intercepting an interface related to access to a cache in a first cache set, where the cache in the first cache set belongs to a cache related to decoding of a first video played by the first application; The physical memory corresponding to the cache in the first cache set is reclaimed.

10. The device according to claim 9, characterized in that The first application is further used to receive a second input from a user, where the second input is used to return the first application to a foreground state; The media module is also used for: reallocating physical memory for caches in the first cache set; Unblocking the interface related to the access to the cache in the first cache set.

11. The device according to claim 10, characterized in that The first cache set includes at least one of a second cache, a third cache, a fourth cache, and a fifth cache, wherein the second cache is an empty input cache held by a media module and in an input cache queue, the third cache is an output cache held by the media module and being filled with decoding results, the fourth cache is an output cache held by the media module and in an output cache queue and filled with decoding results to be used by the first application, the fifth cache is an output cache filled with decoding results held by the first application, and the media module is used to decode the first video.

12. The device according to claim 10 or 11, characterized in that: The media module is specifically used for: When at least one process or thread of the first application uses the interface related to accessing the cache in the first cache set, freezing at least one process or thread of the first application; and / or At least one process or thread of the first application is unfrozen.

13. The device according to claim 11, characterized in that When the first cache set includes a third cache, the media module is specifically configured to: Recording the decoding position corresponding to the decoding result filled in the third cache, reclaiming the physical memory corresponding to the third cache; and reallocating the physical memory for the third cache, and re-decoding the data corresponding to the decoding result according to the decoding position and filling it into the reallocated physical memory; or, Compressing the decoding result filled in the third cache, reclaiming the physical memory corresponding to the third cache; and reallocating the physical memory for the third cache, decompressing the compressed decoding result and filling it into the reallocated physical memory; or, The decoding results filled in the third cache are swapped out through page swapping to reclaim the physical memory corresponding to the third cache; and the physical memory is reallocated for the third cache, and the swapped out decoding results are swapped into the reallocated physical memory.

14. The device according to claim 11 or 13, characterized in that When the first cache set includes a fourth cache, the media module is specifically configured to: Compressing the decoding result filled in the fourth cache, reclaiming the physical memory corresponding to the fourth cache; and reallocating the physical memory for the fourth cache, decompressing the compressed decoding result and filling it into the reallocated physical memory; or, The decoding results in the fourth cache are swapped out through page swapping to reclaim the physical memory corresponding to the fourth cache; and the physical memory is reallocated to the fourth cache, and the swapped out decoding results are swapped into the reallocated physical memory.

15. The device according to any one of claims 9 to 14, characterized in that The interface related to access to the cache in the first cache set includes a first interface and a second interface, the first interface is used by the first application to obtain the second cache, and the second interface is used by the first application to obtain the fourth cache.

16. The device according to claim 15, characterized in that When the first application uses the media module in a synchronous mode, the first interface is used by the first application to actively obtain the second cache, and the second interface is used by the first application to actively obtain the fourth cache; When the first application uses the media module in asynchronous mode, the first interface is used by the first application to obtain the second cache actively released by the media module, and the second interface is used by the first application to obtain the fourth cache actively released by the media module.

17. The device according to claim 11, characterized in that The apparatus also includes an activity management service for: Sending a first message to the media module according to the first input, where the first message is used to indicate that the cache associated with the first video decoding can be reclaimed; A second message is sent to the media module according to the second input, where the second message is used to instruct the first application to return to the foreground.

18. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more computer programs, the one or more computer programs include instructions, when the instructions are executed by one or more processors, the memory management method as described in any one of claims 1 to 8 is executed.

19. A chip, characterized in that: include: A processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the memory management method according to any one of claims 1 to 8 is executed.

20. A readable storage medium, characterized in that: The readable storage medium stores instructions, and when the instructions are executed on a computer, the memory management method according to any one of claims 1 to 8 is executed.

Images