Audio and video resource processing method and device, terminal and storage medium

Abstract

The present disclosure relates to a kind of audio and video resource processing method, device, terminal and storage medium, and it is related to terminal technical field.It includes: determine that application is located in background;In response to the application is located in background, and there is any one of the following conditions, release the audio and video resource occupied by the application: the application is in abnormal decoding state, the application is in soundless video playback state, the application is in soundless audio keep-alive state.The audio and video resource processing method proposed in the present disclosure can release the audio and video resource occupied by background application, save terminal power consumption.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, and in particular to a method, apparatus, terminal and storage medium for processing audio and video resources. Background Technology

[0002] Currently, with the development of application functions, more and more applications are using various means to maintain their long-term activity, thereby facilitating their business development.

[0003] In related technologies, some application developers set their applications to play audio silently after they are opened and enter the background, in order to keep the application active. However, playing audio silently in an application will waste the terminal's power consumption, leading to faster power consumption. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, terminal and storage medium for processing audio and video resources.

[0005] According to a first aspect of the present disclosure, a method for processing audio and video resources is provided, comprising:

[0006] Make sure the application is running in the background;

[0007] In response to the application being in the background and existing in any of the following states, release the audio and video resources occupied by the application:

[0008] The application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state.

[0009] Optionally, the audio and video resources include a video decoder; in response to the application being in the background and the application being in an abnormal decoding state, releasing the audio and video resources occupied by the application includes:

[0010] In response to the application being in the background and in an abnormal decoding state, the video decoder occupied by the application during the abnormal decoding is released.

[0011] Optionally, the audio and video resources include a video decoder and an audio channel. In response to the application being in the background and in a silent video playback state, the audio and video resources occupied by the application are released, including:

[0012] In response to the application being in the background and the application being in a silent video playback state, the video decoder and audio channel occupied by the application when playing the video silently are released.

[0013] Optionally, the audio and video resources include an audio channel. In response to the application being in the background and in a silent audio keep-alive state, releasing the audio and video resources occupied by the application includes:

[0014] In response to the application being in the background and in a silent audio keep-alive state, the audio channel occupied by the application during keep-alive is released.

[0015] Optionally, the method further includes:

[0016] The audio channels occupied when the application is kept alive are marked to obtain the audio channel identifier to be processed;

[0017] If the audio channel identifier to be processed exists and the application does not request the release of the audio channel corresponding to the audio channel identifier to be processed, the audio channel corresponding to the audio channel identifier to be processed is restored.

[0018] Optionally, the method further includes:

[0019] The timer starts when the application requests a video decoder, the video decoder is in use, and the application does not request an audio channel.

[0020] If a preset duration is reached, the application is in the background, and the video decoder is in use, the application is determined to be in an abnormal decoding state.

[0021] Optionally, the method further includes:

[0022] If the application requests a video decoder and the video decoder is in use, and the application requests an audio channel and the audio channel is in silent playback mode, then the application is determined to be in a silent video playback state.

[0023] Optionally, the method further includes:

[0024] If the application does not request a video decoder but does request an audio channel, and the audio channel is in silent playback mode, then the application is determined to be in a silent audio keep-alive state.

[0025] Optionally, the method further includes:

[0026] The audio channel is determined to be in silent playback state if at least one of the following conditions is met:

[0027] The ratio between the duration of writing empty data to the preset period within the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than the human ear perception threshold.

[0028] According to a second aspect of the present disclosure, an audio and video resource processing apparatus is provided, comprising:

[0029] The determination module is configured to determine whether the application is running in the background;

[0030] The response module is configured to release the audio and video resources occupied by the application when the application is in the background and any of the following states exist:

[0031] The application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state.

[0032] According to a third aspect of the present disclosure, a terminal is provided, comprising:

[0033] processor;

[0034] Memory used to store processor-executable instructions;

[0035] The processor is configured as follows:

[0036] The steps of performing the audio and video resource processing method provided in the first aspect of this disclosure.

[0037] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the audio and video resource processing method provided in the first aspect of the present disclosure.

[0038] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0039] First, by releasing abnormally occupied audio and video resources, the released audio and video resources can be used by other normally functioning applications.

[0040] Second, once the audio and video resources occupied by background applications are released, the background applications will stop playing audio and video files, thereby avoiding the power waste caused by playing audio and video files and saving terminal power.

[0041] Third, after releasing audio and video resources, the audio and video resources will be marked. When the application returns to the foreground, the marked and released audio and video resources will be actively restored, thereby ensuring the normal use of the application.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0044] Figure 1 This is a flowchart illustrating a method for processing audio and video resources according to an exemplary embodiment.

[0045] Figure 2 This is a schematic diagram illustrating an audio / video file decoding method according to an exemplary embodiment.

[0046] Figure 3 This is a schematic diagram illustrating the release of audio and video resources according to an exemplary embodiment.

[0047] Figure 4 This is a block diagram illustrating an audio / video resource processing apparatus according to an exemplary embodiment.

[0048] Figure 5 This is a block diagram illustrating an audio / video resource processing apparatus according to an exemplary embodiment. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0050] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0051] Figure 1 This is a flowchart illustrating a method for processing audio and video resources according to an exemplary embodiment, such as... Figure 1 As shown, the method for processing audio and video resources in a terminal includes the following steps.

[0052] In step S11, it is determined that the application is running in the background.

[0053] Please see Figure 3As shown, the terminal has a built-in process management service, multimedia resource service, and keep-alive behavior determination program. The process management service sends application status signals to the keep-alive behavior determination program, which determines whether the application is in the background or foreground based on the application status signals.

[0054] The process management service is used to maintain the startup and termination of various processes in the terminal system, and to record the application corresponding to each process. A process can have multiple threads, and a thread is the basic unit of resource allocation. The process management service is also used to send application status signals to the keep-alive behavior judge, which indicate whether the application is in the foreground or background.

[0055] For example, when an application changes from being visible on the screen to being invisible on the screen, it indicates that the application has switched from the foreground to the background. At this time, the process management service will send an application status signal to the keep-alive behavior judgment program. The application status signal includes at least: the unique identifier (uid) of the process attribute corresponding to the application, and the application status is visible.

[0056] For example, when an application switches from being invisible to being visible on the screen, it indicates that the application has switched from the background to the foreground. At this time, the process management service will send an application status signal to the keep-alive behavior judgment program. The application status signal includes at least: the unique identifier (uid) of the process attribute corresponding to the application, and the application status is invisible.

[0057] The multimedia resource service distinguishes between silent audio files and normally playing audio files based on the audio channel, and sends a usage status signal of the audio and video resources to the keep-alive behavior judgment program. The usage status signal is used to indicate the status of audio and video resource processing of audio and video files.

[0058] Audio and video files refer to files containing audio and video data awaiting playback. Audio and video resources refer to the computational or transmission resources used to process audio and video files, such as audio channels, video decoders, and audio decoders. Computational resources include, for example, the computational resources used by video decoders and audio decoders to process audio and video files. Transmission resources include, for example, the transmission resources used by audio channels to transmit audio files. Please refer to [link / reference]. Figure 2As shown, when an application plays audio or video files, it loads the files into a multimedia resource service. Audio files are decoded by the MediaCodec Audio decoder. The decoded audio data is input to an audio channel, and then the AudioFlinger integrates the audio data from multiple audio channels before outputting it to external speakers or other audio playback devices. For example, background music and character music in games are transmitted through multiple audio channels; the AudioFlinger integrates the audio data from these channels before outputting it to the external speakers. Video files are decoded by the MediaCodec Video decoder. The decoded video data is sent to the Surface Flinger rendering engine for rendering. The video data is packaged with other non-image information such as windows, and the packaged image information is sent to the GPU. The GPU then displays the image information on the terminal screen. Figure 2 The synchronization program in the software is used to upload and play audio and video synchronously when playing audio and video at the same time.

[0059] Usage status signals include audio transmission resource usage status signals and video transmission resource usage status signals.

[0060] Audio transmission resource usage status signals include: start transmitting audio data, stop transmitting audio data, and silent audio playback. Audio transmission resources include audio channels and audio decoders.

[0061] For example, when an audio file is split into audio data, and the application's thread controls the audio channel to start writing audio data, the multimedia resource service sends an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: the process attribute identifier, the thread attribute unique identifier (pid), and the audio channel unique identifier (aid). The status of the audio channel is that it has started transmitting audio data.

[0062] For example, when the application's thread controls the audio channel to stop writing audio data, the multimedia resource service will send an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is that it has stopped transmitting audio data.

[0063] For example, assuming the time period interval is T1, the duration of writing empty data to the audio channel within period T1 is t1, and the duration of writing non-empty data is t2. When the ratio is greater than the preset value K, it is determined that the duration of playing empty data in the audio channel is relatively long. Then, the audio channel is marked to perform silent playback. At this time, the multimedia resource service will send an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is silent playback of audio data.

[0064] For example, when the average frequency f of outputting non-empty data within the time interval T1 is less than the human hearing perception threshold F, the audio channel is marked to perform silent playback. At this time, the multimedia resource service will send an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is silent playback of audio data.

[0065] Video transmission resource usage status signals include: in use and not in use.

[0066] For example, when an application thread requests a video decoder, the multimedia resource service sends a video transmission resource usage status signal to the keep-alive behavior judgment program. The video transmission resource usage status signal includes: a unique identifier for the process attribute, a unique identifier for the thread attribute, a unique identifier for the video decoder (vid), and the video decoder's status is "in use".

[0067] For example, when an application thread is deregistered and releases the video decoder, the multimedia resource service sends a video transmission resource usage status signal to the keep-alive behavior judgment program. The video transmission resource usage status signal includes: a unique identifier for the process attribute, a unique identifier for the thread attribute, a unique identifier for the video decoder, and the video decoder's status is "not in use".

[0068] The application runs on a process management service and plays audio or video using a multimedia resource service, as shown in the following details:

[0069] 1) For audio resources only, an application thread can request one or more audio channels for audio playback. The audio file is decoded into audio data by an audio decoder, and then the audio data is divided into data segments and written into the buffer of the audio channel. Each audio channel is independent of each other.

[0070] For example, when playing stereo or 5.1 channel audio, a music player will use a corresponding number of audio channels to play different audio tracks. Similarly, for audio data containing surround sound or special sound effects, in order to ensure that users have a better audio experience, the music player will also use multiple audio channels to present these mixing effects.

[0071] 2) For video files containing audio files, one thread of the application can request one or more video decoders for video playback, and one thread of the application can request one audio channel for audio playback.

[0072] For example, a video playback software thread can request one or more video decoders for video playback, but usually it requests an audio channel for audio playback so that the audio and video can be played synchronously.

[0073] In step S12, in response to the application being in the background and existing in any of the following states, the audio and video resources occupied by the application are released: the application is in an abnormal decoding state, the application is in a silent video playback state, or the application is in a silent audio keep-alive state.

[0074] When the terminal determines whether the application is in the background based on the status signal indicated by the application status signal and the audio / video resource usage status signal indicated by the multimedia resource service through the keep-alive behavior, it can also determine whether the application is in an abnormal decoding state, a silent video playback state, or a silent audio keep-alive state based on the status indicated by the usage status signal. In any of these three abnormal states, the terminal releases the audio / video resources occupied by the application.

[0075] An abnormal decoding state refers to a situation where the audio or video decoder used by the application is unable to decode audio or video files for an extended period of time, resulting in decoding abnormalities.

[0076] Silent video playback means that the video is playing but there is no sound. For example, the sound is inaudible to the human ear while the video is playing, or the sound of the video itself is adjusted to a volume that is inaudible to the human ear by the application.

[0077] Silent audio keep-alive state refers to an application playing audio silently in the background in order to maintain its own activity.

[0078] The above technical solution allows us to address the issue that when an application is in the background, it means the user is not using it. When the application is in any of the following states: abnormal decoding, silent video playback, or silent audio keep-alive, it indicates that the application is consuming additional audio and video resources. Therefore, even when the user is not using the application and it is consuming additional audio and video resources on the terminal, the audio and video resources occupied by the application can be released.

[0079] On the one hand, it allows the released audio and video resources to be used by other normally used applications. On the other hand, after the audio and video resources occupied by background applications are released, the applications will stop playing, thereby avoiding the power consumption waste caused by playback and saving terminal power.

[0080] The optional and specific embodiments involved in steps S11 and S12 above will be described below.

[0081] The terminal dynamically records usage status signals from multimedia resource services through keep-alive behavior, recording the usage status of audio and video resources in the terminal system. For example, the unique identifier of the audio channel is denoted as aid, with a default aid value of -1. When aid is -1, no audio channel is requested. The audio channel status (aid_status) can be playing, silent playing, or not playing, with the default being not playing. The time of the most recent silent playing of the audio channel is denoted as lat_mute, with a default lat_mute value of -1. When lat_mute is -1, there is no most recent silent playing time. The unique identifier of the video decoder is denoted as vid, with a default vid value of -1. When vid is -1, no video decoder is requested. The video decoder usage status (vid_status) can be used or not used, with the default being not used.

[0082] If the terminal determines that an application has never been in the background by judging its keep-alive behavior, then when the application is in the background, it will not judge whether the application is in any of the following states: abnormal decoding state, silent video playback state, or silent audio keep-alive state. If it is determined that the application has been in the background, then the application will be marked, and then it will be judged whether the application is in any of the following states: abnormal decoding state, silent video playback state, or silent audio keep-alive state.

[0083] In one specific embodiment, the audio and video resources include a video decoder, so releasing the audio and video resources includes: in response to the application being in the background and the application being in an abnormal decoding state, releasing the video decoder occupied by the application during abnormal decoding.

[0084] If the application requests a video decoder and the video decoder is in use, and the application does not request an audio channel, a timer is started. If a preset time is reached, and the application is in the background and the video decoder is in use, it is determined that the application is in an abnormal decoding state without any audio channel occupation. When the application is in an abnormal decoding state without any audio channel occupation, the video decoder is released.

[0085] For example, when vid is not -1 (there is a request for the video decoder) and the corresponding vid_status is in use (the video decoder is being used), and all aids are -1 (there is no request for the audio channel), start timing for duration T2. ​​If within T2 time, the application status signal sent by the process management service indicates that the application status is invisible, and the usage status signal sent by the multimedia resource service indicates that the video decoder is not in use, it is determined that the video decoder cannot decode the video file for a long time, and the application will continue to occupy the video decoder in the background. In this case, the video decoder occupied by the application can be released.

[0086] Of course, if the application requests a video decoder and the video decoder is in use, and the application requests an audio channel but the audio channel is playing silently, a timer starts. If a preset duration is reached, and the application is in the background while the video decoder is in use, it is determined that the application is in an abnormal decoding state, accompanied by the application occupying the audio channel for silent playback. If the application is in an abnormal decoding state and the audio channel is playing silently, and it is determined that the video decoder cannot decode the video file for an extended period, and the application will continue to occupy the video decoder in the background, then the video decoder occupied by the application can be released. The reason for not releasing the audio channel here is that some video playback software also plays audio while playing video. When the video decoder is abnormally decoding, the audio channel does not need to be released, thus ensuring that the video playback software can occupy the audio channel to play audio while releasing the video decoder and not playing video.

[0087] By using the above solution, when the video decoder fails to decode due to abnormal code logic, it is determined that the video decoder is being used improperly. At this time, the video decoder occupied by the background application can be released, which can prevent the video decoder from being continuously occupied and reduce the power consumption caused by the application's thread requesting the video decoder for continuous decoding.

[0088] In one specific embodiment, the audio and video resources include a video decoder and an audio channel. Therefore, releasing the audio and video resources includes: in response to the application being in the background and the application being in a silent video playback state, releasing the video decoder and audio channel occupied by the application when the application is playing the video silently.

[0089] If the application requests a video decoder and the video decoder is in use, and the application requests an audio channel and the audio channel is in silent playback mode, then the application is determined to be in a silent video playback state.

[0090] For example, if vid is not -1 (there is a request for a video decoder), vid_status is in use (the video decoder is being used), aid is not -1 (there is a request for at least one audio channel out of multiple audio channels), and aid_status is silent playback (the audio channel is in silent playback), it can be determined that the application is playing video in the background and the video has no sound. In this case, the video decoder and audio channel occupied by the application can be released.

[0091] By implementing the above solution, when the application is playing video silently, the video decoder and audio channel occupied by the application are released, which can prevent the video decoder and audio channel from being continuously occupied and also reduce the power consumption caused by the application's threads continuously transmitting audio and video files after requesting the video decoder and audio channel.

[0092] In one specific embodiment, the audio and video resources include an audio channel. Therefore, releasing the audio and video resources includes: in response to the application being in the background and the application being in a silent audio keep-alive state, releasing the audio channel occupied by the application during the keep-alive state.

[0093] If the application does not request a video decoder but does request an audio channel, and the audio channel is in silent playback mode, then the application is determined to be in a silent audio keep-alive state.

[0094] For example, if vid is not -1 (there is no request for a video decoder), aid is not -1 (there is a request for at least one audio channel among multiple audio channels), and aid_status is silent playback (the audio channel status is silent playback), it is determined that the application is playing audio in the background and there is no sound. At this time, the audio channel occupied by the application can be released.

[0095] After releasing the audio channel occupied by the application, the audio channel occupied when the application was kept alive can be marked to obtain the audio channel identifier to be processed; if the audio channel identifier to be processed exists and the application does not request to release the audio channel corresponding to the audio channel identifier to be processed, the audio channel corresponding to the audio channel identifier to be processed is restored.

[0096] The pending audio channel identifier refers to the audio channel that is being released for silent playback. An application actively requests the release of the audio channel corresponding to the pending audio channel identifier when the user clears the application from the background; in this case, the application will actively request the release of the audio channel. If the application does not actively request the release of the audio channel corresponding to the pending audio channel identifier, it means that the application has returned to the foreground, and the user needs to use the application normally, so the application will not actively request the release of the audio channel.

[0097] When an application is kept alive, it may occupy part of the audio channel to perform silent playback. At this time, the occupied audio channel can be released to save terminal power consumption. If it is confirmed that the application has not actively requested to release the audio channel used for silent playback, the released audio channel can be restored, thereby ensuring that the application runs normally after returning to the foreground, and users can use the application normally without noticing.

[0098] In related technologies, to prevent applications from consuming excessive audio and video resources during silent playback, the application is often closed directly when silent playback occurs. However, this logic may mistakenly classify a normal application as abnormal, leading to its abnormal closure and a degraded user experience. For example, a music player might use multiple audio channels during playback. If one of these channels becomes silent, the music player will be permanently deleted, causing it to stop working abnormally.

[0099] This disclosure releases audio channels that are used for silent playback among multiple audio channels. Since each audio channel is independent of the others, the remaining audio channels will not be released, thus ensuring that the application can be used normally and preventing accidental closure.

[0100] Furthermore, after releasing the audio channel that was performing silent playback, a pending audio channel identifier will be added to that audio channel. If it is determined that the application did not actively request the release of the audio channel that was performing silent playback, the audio channel corresponding to the pending audio channel identifier can be restored according to the pending audio channel identifier, that is, the silent playback audio channel can be restored. This further ensures that the application can be used normally by the user after entering the foreground, and that the application will not be closed unexpectedly.

[0101] In the above scheme, after the audio channel is released, the audio decoder will naturally be released accordingly.

[0102] Figure 4 This is a block diagram illustrating an audio / video resource processing apparatus according to an exemplary embodiment. (Refer to...) Figure 4The audio and video resource processing device 400 includes a determining module 410 and a responding module 420.

[0103] Module 410 is configured to determine that the application is running in the background.

[0104] The response module 420 is configured to release the audio and video resources occupied by the application when the application is in the background and any of the following states exist:

[0105] The application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state.

[0106] Optionally, the audio and video resources include a video decoder; the response module 420 includes:

[0107] The first response submodule is configured to release the video decoder used by the application during abnormal decoding when the application is in the background and the application is in an abnormal decoding state.

[0108] Optionally, the audio and video resources include a video decoder and an audio channel, and the response module 420 includes:

[0109] The second response submodule is configured to release the video decoder and audio channel occupied by the application when the application is in the background and the application is playing video silently.

[0110] Optionally, the audio and video resources include an audio channel, and the response module 420 includes:

[0111] The third response submodule is configured to release the audio channel occupied by the application during its keep-alive state when the application is in the background and the application is in a silent audio keep-alive state.

[0112] Optionally, the audio / video resource processing device 400 includes:

[0113] The marking module is configured to mark the audio channels occupied when the application is kept alive, and obtain the audio channel identifier to be processed;

[0114] The recovery module is configured to restore the audio channel corresponding to the audio channel identifier to be processed when the audio channel identifier to be processed exists and the application does not request to release the audio channel corresponding to the audio channel identifier to be processed.

[0115] Optionally, the audio / video resource processing device 400 includes:

[0116] The timing module is configured to start timing when the application requests a video decoder and the video decoder is in use, and the application does not request an audio channel.

[0117] An abnormal decoding determination module is configured to determine that the application is in an abnormal decoding state when a preset duration is reached, the application is in the background, and the video decoder is in use.

[0118] Optionally, the audio / video resource processing device 400 includes:

[0119] The silent video playback determination module is configured to determine that the application is in a silent video playback state when the application requests a video decoder and the video decoder is in use, and the application requests an audio channel and the audio channel is in a silent playback state.

[0120] Optionally, the audio / video resource processing device 400 includes:

[0121] The silent audio keep-alive determination module is configured to determine that the application is in a silent audio keep-alive state when the application does not request a video decoder but does request an audio channel and the audio channel is in silent playback mode.

[0122] Optionally, the audio / video resource processing device 400 includes:

[0123] The silent playback determination module is configured to determine that the audio channel is in silent playback state if at least one of the following conditions exists:

[0124] The ratio between the duration of writing empty data to the preset period within the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than the human ear perception threshold.

[0125] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0126] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the audio and video resource processing method provided in this disclosure.

[0127] Figure 5This is a block diagram illustrating an apparatus 500 for processing audio and video resources according to an exemplary embodiment. For example, apparatus 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0128] Reference Figure 5 The device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output interface 512, sensor component 514, and communication component 516.

[0129] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-described audio and video resource processing method. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0130] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of such data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] Power supply component 506 provides power to various components of device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.

[0132] Multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0133] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0134] Input / output interface 512 provides an interface between processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0135] Sensor assembly 514 includes one or more sensors for providing status assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0136] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0137] In an exemplary embodiment, the apparatus 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described audio and video resource processing method.

[0138] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by the processor 520 of the device 500 to complete the above-described audio and video resource processing method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0139] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0140] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for processing audio and video resources, characterized in that, include: Make sure the application is running in the background; In response to the application being in the background and existing in any of the following states, release the audio and video resources occupied by the application: The application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state. The audio and video resources include video decoders and / or audio channels; In response to the application being in the background and in a silent audio keep-alive state, the audio and video resources occupied by the application are released, including: In response to the application being in the background and the application being in a silent audio keep-alive state, release the audio channels occupied by the application during the keep-alive process among multiple audio channels; The method further includes: The audio channels occupied when the application is kept alive are marked to obtain the audio channel identifier to be processed; If the pending audio channel identifier exists and the application does not request to release the audio channel corresponding to the pending audio channel identifier, the audio channel corresponding to the pending audio channel identifier is restored.

2. The method according to claim 1, characterized in that, In response to the application being in the background and in an abnormal decoding state, the audio and video resources occupied by the application are released, including: In response to the application being in the background and in an abnormal decoding state, the video decoder occupied by the application during the abnormal decoding is released.

3. The method according to claim 1, characterized in that, In response to the application being in the background and in a silent video playback state, the audio and video resources occupied by the application are released, including: In response to the application being in the background and the application being in a silent video playback state, the video decoder and audio channel occupied by the application when playing the video silently are released.

4. The method according to claim 1, characterized in that, The method further includes: The timer starts when the application requests a video decoder, the video decoder is in use, and the application does not request an audio channel. If a preset duration is reached, the application is in the background, and the video decoder is in use, the application is determined to be in an abnormal decoding state.

5. The method according to claim 1, characterized in that, The method further includes: If the application requests a video decoder and the video decoder is in use, and the application requests an audio channel and the audio channel is in silent playback mode, then the application is determined to be in a silent video playback state.

6. The method according to claim 1, characterized in that, The method further includes: If the application does not request a video decoder but does request an audio channel, and the audio channel is in silent playback mode, then the application is determined to be in a silent audio keep-alive state.

7. The method according to claim 5 or 6, characterized in that, The method further includes: The audio channel is determined to be in silent playback state if at least one of the following conditions is met: The ratio between the duration of writing empty data to the preset period within the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than the human ear perception threshold.

8. An audio and video resource processing apparatus, characterized in that, include: The determination module is configured to determine whether the application is running in the background; The response module is configured to release the audio and video resources occupied by the application when the application is in the background and any of the following states exist: The application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state. The audio and video resources include video decoders and / or audio channels; The response module includes: The third response submodule is configured to release the audio channels occupied by the application during its keep-alive state when the application is in the background and the application is in a silent audio keep-alive state. The audio and video resource processing device includes: The marking module is configured to mark the audio channels occupied when the application is kept alive, and obtain the audio channel identifier to be processed; The recovery module is configured to restore the audio channel corresponding to the audio channel identifier to be processed when the audio channel identifier to be processed exists and the application does not request to release the audio channel corresponding to the audio channel identifier to be processed.

9. A terminal, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 7.

Images