Audio and video resource processing method and device, terminal and storage medium

Abstract

The invention relates to an audio and video resource processing method and device, a terminal and a storage medium, and relates to the technical field of terminals. The method includes: determining that an application is background; and in response to the situation that the application program is located in the background and any one of the following states exists, releasing audio and video resources occupied by the application program: the application program is in an abnormal decoding state, the application program is in a silent video playing state, and the application program is in a silent audio keep-alive state. By using the audio and video resource processing method provided by the invention, the audio and video resources occupied by the background application program can be released, and the power consumption of the terminal is saved.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a method, device, terminal and storage medium for processing audio and video resources. Background Art

[0002] At present, with the development of application functions, more and more applications will maintain their long-term active status through various means to facilitate the development of their own business.

[0003] In the related art, some application developers will set the application to play audio silently after it is opened and enters the background, so as to keep the application active. However, the application playing audio silently will cause a waste of terminal power consumption and cause the terminal to consume power faster. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, terminal and storage medium for processing audio and video resources.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for processing audio and video resources is provided, including:

[0006] Make sure the app is in the background;

[0007] In response to the application being in the background and in any of the following states, releasing the audio and video resources occupied by the application:

[0008] The application is in an abnormal decoding state, the application is in a silent video playing state, and the application is in a silent audio keep-alive state.

[0009] Optionally, the audio and video resources include a video decoder; in response to the application being in the background and the application being in an abnormal decoding state, releasing the audio and video resources occupied by the application, comprising:

[0010] In response to the application being in the background and being in an abnormal decoding state, releasing a video decoder occupied by the application during abnormal decoding.

[0011] Optionally, the audio and video resources include a video decoder and an audio channel, and in response to the application being in the background and the application being in a silent video playing state, releasing the audio and video resources occupied by the application includes:

[0012] In response to the application being in the background and being in a state of silently playing a video, the video decoder and the audio channel occupied by the application when the video is silently played are released.

[0013] Optionally, the audio and video resources include an audio channel, and in response to the application being in the background and being in a silent audio keep-alive state, releasing the audio and video resources occupied by the application includes:

[0014] In response to the application being in the background and being in a silent audio keep-alive state, releasing the audio channel occupied by the application when it is kept alive.

[0015] Optionally, the method further comprises:

[0016] Marking the audio channel occupied when releasing the application program to keep it alive, and obtaining an identifier of the audio channel to be processed;

[0017] In a case where the to-be-processed audio channel identifier exists and the application does not request to release the audio channel corresponding to the to-be-processed audio channel identifier, the audio channel corresponding to the to-be-processed audio channel identifier is restored.

[0018] Optionally, the method further comprises:

[0019] When the application has applied for a video decoder, the video decoder is in use, and the application has not applied for an audio channel, the timing starts;

[0020] When the preset time duration is reached, the application is in the background, and the video decoder is in the use state, it is determined that the application is in an abnormal decoding state.

[0021] Optionally, the method further comprises:

[0022] When the application has applied for a video decoder, and the video decoder is in use, and the application has applied for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent video playback state.

[0023] Optionally, the method further comprises:

[0024] When the application does not apply for a video decoder but applies for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent audio keep-alive state.

[0025] Optionally, the method further comprises:

[0026] In at least one of the following cases, it is determined that the use state of the audio channel is silent playback:

[0027] The ratio of the duration of writing empty data to the audio channel within a preset period to the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than a human ear perception threshold.

[0028] According to a second aspect of an embodiment of the present disclosure, there is provided a device for processing audio and video resources, including:

[0029] a determination module configured to determine that the application is in the background;

[0030] The response module is configured to release the audio and video resources occupied by the application in response to the application being in the background and in any of the following states:

[0031] The application is in an abnormal decoding state, the application is in a silent video playing state, and the application is in a silent audio keep-alive state.

[0032] According to a third aspect of an embodiment of the present disclosure, a terminal is provided, including:

[0033] processor;

[0034] a memory for storing processor-executable instructions;

[0035] Wherein, the processor is configured to:

[0036] Execute the steps of the method for processing audio and video resources provided in the first aspect of the present disclosure.

[0037] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for processing audio and video resources provided in the first aspect of the present disclosure are implemented.

[0038] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0039] 1. By releasing the audio and video resources that are abnormally occupied, the released audio and video resources can be used by other normally used applications.

[0040] 2. After the audio and video resources occupied by the background application are released, the background application will stop playing the audio and video files, thereby avoiding the power waste caused by playing the audio and video files and saving terminal power.

[0041] 3. After releasing the audio and video resources, they will be marked. When the application returns to the foreground, the marked and released audio and video resources will be actively restored to ensure the normal use of the application.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0044] Figure 1 The present invention is a flowchart of a method for processing audio and video resources according to an exemplary embodiment.

[0045] Figure 2 The figure is a schematic diagram showing decoding of an audio and video file according to an exemplary embodiment.

[0046] Figure 3 The diagram is a schematic diagram showing releasing audio and video resources according to an exemplary embodiment.

[0047] Figure 4 The present invention is a block diagram of a device for processing audio and video resources according to an exemplary embodiment.

[0048] Figure 5 The present invention is a block diagram of a device for processing audio and video resources according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0050] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0051] Figure 1 is a flowchart of a method for processing audio and video resources according to an exemplary embodiment. Figure 1 As shown, the method for processing audio and video resources is used in a terminal and includes the following steps.

[0052] In step S11 , it is determined that the application is in the background.

[0053] See also Figure 3As shown, the terminal has built-in process management service, multimedia resource service and keep-alive behavior judgment program. The process management service sends an application status signal to the keep-alive behavior judgment program, and the keep-alive behavior judgment program determines whether the application is in the background or the foreground based on the application status signal.

[0054] The process management service is used to maintain the startup and termination of each process in the terminal system, and record the applications corresponding to each process in the terminal system. A process can have multiple threads, and threads are the basic unit of resource allocation. The process management service is also used to send application status signals to the keep-alive behavior judgment program. The application status signal is used to indicate whether the application is in the foreground or background.

[0055] For example, when an application switches from being visible on the screen to being invisible on the screen, it indicates that the application switches from the foreground to the background. At this time, the process management service will send an application status signal to the keep-alive behavior judgment program. The application status signal includes at least: the unique identifier (uid) of the process attribute corresponding to the application, and the application status is visible.

[0056] For example, when an application switches from being invisible on the screen to being visible on the screen, it indicates that the application switches from the background to the foreground. At this time, the process management service will send an application status signal to the keep-alive behavior judgment program. The application status signal includes at least: the unique identifier (uid) of the process attribute corresponding to the application, and the application status is invisible.

[0057] The multimedia resource service distinguishes between silently played audio files and normally played audio files based on the audio channel, and sends a usage status signal of the audio and video resources to the keep-alive behavior judgment program. The usage status signal is used to indicate the status of the audio and video resources processing audio and video files.

[0058] Audio and video files refer to audio and video files waiting to be played. Audio and video resources refer to computing resources or transmission resources such as audio channels, video decoders, and audio decoders used to process audio and video files. Computing resources include computing resources such as video decoders and audio decoders that perform computing processing on audio and video files. Transmission resources include transmission resources such as audio channels that are used to transmit audio files. Figure 2As shown, when the application plays audio and video files, the application loads the audio and video files into the multimedia resource service, the audio files are decoded by the audio decoder (MediaCodec Audio), the decoded audio data is input into the audio channel, and then the audio engine (AudioFlinger) integrates the audio data of multiple audio channels, and outputs the audio to an audio playback device such as a peripheral speaker. For example, the background music and character music when playing games are transmitted through multiple audio channels. At this time, the audio data of multiple audio channels can be integrated by the audio engine and then output to the peripheral speaker; the video file is decoded by the video decoder (MediaCodecVideo), and the decoded video data is sent to the rendering engine (Surface Flinger) for rendering, the video data and other non-image information such as windows are packaged, and the packaged image information is sent to the image processor (GPU), and the image processor displays the image information on the terminal screen. Figure 2 The synchronization program in is used to upload and play audio and video synchronously when playing audio and video at the same time.

[0059] The usage status signal includes an audio transmission resource usage status signal and a video transmission resource usage status signal.

[0060] The audio transmission resource usage status signal includes: starting to transmit audio data, stopping to transmit audio data, playing audio data silently, etc. The audio transmission resources include audio channels and audio decoders.

[0061] For example, an audio file is split into audio data. When the application's thread controls the audio channel to start writing audio data, the multimedia resource service sends an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: the process attribute identifier, the thread attribute unique identifier (pid), and the audio channel unique identifier (aid). The status of the audio channel is to start transmitting audio data.

[0062] For example, when the application's thread-controlled audio channel stops writing audio data, the multimedia resource service sends an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is to stop transmitting audio data.

[0063] For example, assuming that the time period interval is T1, the duration of the empty data written by the audio channel in the T1 period is t1, and the duration of writing non-empty data is t2. When the ratio is greater than the preset value K, it is determined that the audio channel plays empty data for a long time, then the audio channel is marked to perform silent playback. At this time, the multimedia resource service will send an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is to play audio data silently.

[0064] For example, when the average frequency f of non-empty data output within the time period interval T1 is less than the human ear perception threshold F, the audio channel is marked to perform silent playback. At this time, the multimedia resource service will send an audio transmission resource usage status signal to the keep-alive behavior judgment program. The audio transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier of the audio channel. The status of the audio channel is to play audio data silently.

[0065] The video transmission resource usage status signal includes: used and not used.

[0066] For example, the application thread applies for a video decoder, and the multimedia resource service sends a video transmission resource usage status signal to the keep-alive behavior judgment program. The video transmission resource usage status signal includes: a unique identifier of the process attribute, a unique identifier of the thread attribute, and a unique identifier (vid) of the video decoder. The status of the video decoder is in use.

[0067] For example, the thread deregistration of the application releases the video decoder, and the multimedia resource service sends a video transmission resource usage status signal to the keep-alive behavior judgment program. The video transmission resource usage status signal includes: the unique identifier of the process attribute, the unique identifier of the thread attribute, and the unique identifier of the video decoder. The status of the video decoder is not in use.

[0068] The application relies on the process management service to run and the multimedia resource service to play audio or video, as shown below:

[0069] 1) For audio resources only, a thread of the application can apply for one or more audio channels for audio playback. The audio file is decoded into audio data by an audio decoder, and then the audio data is divided into data segments and written into the buffer of the audio channel. Each audio channel is independent of each other.

[0070] For example, for a certain music player software, when playing multi-channel audio such as stereo or 5.1 channels, the music player software will use the corresponding number of audio channels to play audio tracks of different channels; for another example, for audio data containing mixed effects such as surround sound or special sound effects, in order to ensure that users can have a better audio experience, the music player software will also use multiple audio channels to present these mixed effects.

[0071] 2) For a video file containing an audio file, one thread of the application can apply for one or more video decoders for video playback, and one thread of the application can apply for an audio channel for audio playback.

[0072] For example, for a certain video playback software, the thread of the video playback software can apply for one or more video decoders to play the video, but usually applies for an audio channel to play the audio so that the audio and video can be played synchronously.

[0073] In step S12, in response to the application being in the background and in any of the following states, the audio and video resources occupied by the application are released: the application is in an abnormal decoding state, the application is in a silent video playback state, and the application is in a silent audio keep-alive state.

[0074] When the terminal receives the application status signal sent by the process management service and the usage status signal of the audio and video resources sent by the multimedia resource service through the keep-alive behavior judgment program, it can determine whether the application is in the background according to the status indicated by the application status signal, and determine whether the application is in the abnormal decoding state, the silent video playback state, and the silent audio keep-alive state according to the status indicated by the usage status signal; and when in any of these three abnormal states, release the audio and video resources occupied by the application.

[0075] The abnormal decoding state means that the audio decoder or video decoder occupied by the application cannot decode the audio or video file for a long time, resulting in decoding abnormality.

[0076] The silent video playback state means that the video is playing but there is no sound. For example, the sound cannot be heard by human ears when the video is playing or the sound of the video itself is adjusted to a volume that cannot be heard by human ears by the application.

[0077] The silent audio keep-alive state means that the application plays audio silently in the background in order to maintain its own activity.

[0078] Through the above technical solution, when the application is in the background, it means that the user will not use the application at this time. When the application is in any of the abnormal decoding state, silent video playback state or silent audio keep-alive state, it means that the application will occupy additional audio and video resources. At this time, when the user does not use the application and the application will additionally occupy the audio and video resources of the terminal, the audio and video resources occupied by the application can be released.

[0079] On the one hand, the released audio and video resources can be occupied by other normally used applications. On the other hand, after the audio and video resources occupied by the background application are released, the application will stop playing, thereby avoiding the power waste caused by the playback behavior and saving terminal power.

[0080] The optional embodiments and specific embodiments involved in the above-mentioned step S11 and step S12 will be introduced below.

[0081] The terminal dynamically records the usage status signal from the multimedia resource service through the keep-alive behavior judgment program, and records the usage status of the audio and video resources in the terminal system. For example, the unique identifier of the audio channel is recorded as aid, and the default aid is -1. When aid is -1, there is no application for the audio channel. The status of the audio channel (aid_status) includes play, silent play and non-play, and the default is non-play; the time of the most recent silent play of the audio channel is recorded as lat_mute, and the default lat_mute is -1. When lat_mute is -1, there is no most recent silent play time; the unique identifier of the video decoder is recorded as vid, and the default vid is -1. When vid is -1, there is no application for the video decoder. The usage status (vid_status) of the video decoder is used and not used, and the default is not used.

[0082] The terminal determines through the keep-alive behavior judgment program that the application has never been in the background. Then, for these applications, when these applications are in the background, it will not be determined whether these applications are in any of the abnormal decoding state, silent video playback state and silent audio keep-alive state; if it is determined that the application has been in the background, then for these applications, it will be marked, and then it will be determined whether these applications are in any of the abnormal decoding state, silent video playback state and silent audio keep-alive state.

[0083] In a specific embodiment, the audio and video resources include a video decoder, and thus releasing the audio and video resources includes: in response to the application being in the background and the application being in an abnormal decoding state, releasing the video decoder occupied by the application during abnormal decoding.

[0084] When the application has applied for a video decoder, the video decoder is in use, and the application has not applied for an audio channel, start timing; when a preset time is reached and the application is in the background and the video decoder is in use, determine that the application is in an abnormal decoding state and is not accompanied by the occupation of any audio channel; when the application is in an abnormal decoding state and is not accompanied by the occupation of any audio channel, release the video decoder.

[0085] For example, when vid is not -1 (there is an action of applying for a video decoder) and the corresponding vid_status is in use (the video decoder is being used), and all aid are -1 (there is no action of applying for an audio channel), the timing duration T2 starts. If the application status signal sent by the process management service is not received within T2, indicating that the application status is invisible, and the usage status signal sent by the multimedia resource service is not received, indicating that the usage status of the video decoder is not in use, it is determined that the video decoder cannot decode the video file for a long time, and the application will continue to occupy the video decoder when it is in the background, then the video decoder occupied by the application can be released.

[0086] Of course, when the application has applied for a video decoder, and the video decoder is in use, and the application has applied for an audio channel, and the audio channel is playing silently, the timing starts; when the preset time is reached, the application is in the background, and the video decoder is in use, it is determined that the application is in an abnormal decoding state, and the application occupies the audio channel to perform silent playback; when the application is in an abnormal decoding state and the audio channel performs silent playback, it is determined that the video decoder cannot decode the video file for a long time, and the application will continue to occupy the video decoder in the background, then the video decoder occupied by the application can be released. The reason for not releasing the audio channel here is that for some video playback software, it also plays audio when playing video. When the video decoder is abnormally decoded, the audio channel does not need to be released, thereby ensuring that the video playback software can occupy the audio channel to play audio while releasing the video decoder to stop playing video.

[0087] Through the above scheme, when the video decoder cannot decode due to code logic abnormalities, it is determined that the video decoder is used unreasonably. At this time, the video decoder occupied by the background application can be released, which can avoid the video decoder being continuously occupied and reduce the power consumption caused by the continuous decoding after the application thread applies for the video decoder.

[0088] In a specific embodiment, the audio and video resources include a video decoder and an audio channel, so releasing the audio and video resources includes: in response to the application being in the background and the application being in a silent video playback state, releasing the video decoder and the audio channel occupied by the application when the application is silently playing the video.

[0089] When the application has applied for a video decoder, and the video decoder is in use, and the application has applied for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent video playback state.

[0090] For example, when vid is not -1 (there is an action of applying for a video decoder), and vid_status is in use (the video decoder is being used), and aid is not -1 (there is an action of applying for at least one of multiple audio channels), and aid_status is silent play (the status of the audio channel is silent play), it is determined that the application is playing a video in the background and the video has no sound. At this time, the video decoder and audio channel occupied by the application can be released.

[0091] Through the above solution, when the application plays the video silently, the video decoder and audio channel occupied by the application are released, which can avoid the video decoder and audio channel from being continuously occupied, and can also reduce the power consumption caused by the application thread applying for the video decoder and audio channel and continuously transmitting audio and video files.

[0092] In a specific embodiment, the audio and video resources include an audio channel, and therefore, releasing the audio and video resources includes: in response to the application being in the background and the application being in a silent audio keep-alive state, releasing the audio channel occupied by the application when it is kept alive.

[0093] When the application does not apply for a video decoder but applies for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent audio keep-alive state.

[0094] For example, when vid is not -1 (there is no behavior of applying for a video decoder), aid is not -1 (there is a behavior of applying for at least one of multiple audio channels), and aid_status is silent playback (the status of the audio channel is silent playback), it is determined that the application is playing audio in the background and the audio has no sound. At this time, the audio channel occupied by the application can be released.

[0095] After releasing the audio channel occupied by the application, the audio channel occupied when the application is kept alive can be marked to obtain an identifier of the audio channel to be processed; if the identifier of the audio channel to be processed exists and the application does not request to release the audio channel corresponding to the identifier of the audio channel to be processed, the audio channel corresponding to the identifier of the audio channel to be processed is restored.

[0096] The pending audio channel identifier refers to the audio channel that performs silent playback and is released. The application actively requests to release the audio channel corresponding to the pending audio channel identifier, which means that the user clears the application from the background, and the application will actively request to release the audio channel; the application does not actively request to release the audio channel corresponding to the pending audio channel identifier, which means that the application returns to the foreground, and the user needs to use the application normally, so the application will not actively request to release the audio channel.

[0097] When the application is kept alive, some audio channels will be occupied to perform silent playback. At this time, the occupied audio channels can be released to save terminal power consumption. When it is confirmed that the application has not actively requested to release the audio channels for silent playback, the released audio channels can be restored to ensure that the application runs normally after returning to the foreground, and users can use the application normally without noticing.

[0098] In the related art, in order to prevent the application from occupying extra audio and video resources when playing silently, the application will be directly closed when the application performs silent playback. However, this logic may mistakenly judge a normal application as an abnormal application, which will cause the normal application to be abnormally closed, reducing the user experience. For example, a music player software will occupy multiple audio channels when playing music. If one of the audio channels performs silent playback, the music player software will be directly cleared, causing the music player software to stop being used abnormally.

[0099] The present disclosure can release the audio channel for silent playback among multiple audio channels by releasing the audio channel. Since the audio channels are independent of each other, the remaining audio channels will not be released, thereby ensuring that the application can be used normally and avoiding accidental closure.

[0100] Moreover, after releasing the audio channel for silent playback, the audio channel will be marked with a pending audio channel identifier. After determining that the application has not actively requested to release the audio channel for silent playback, the audio channel corresponding to the pending audio channel identifier can be restored based on the pending audio channel identifier, that is, the audio channel for silent playback is restored, thereby further ensuring that the application can be used normally by the user after entering the foreground, and the application will not be accidentally closed.

[0101] In the above scheme, after the audio channel is released, the audio decoder will naturally be released accordingly.

[0102] Figure 4 is a block diagram of a device for processing audio and video resources according to an exemplary embodiment. Figure 4The audio and video resource processing device 400 includes a determination module 410 and a response module 420.

[0103] A determination module 410 is configured to determine that the application is in the background;

[0104] The response module 420 is configured to release the audio and video resources occupied by the application in response to the application being in the background and in any of the following states:

[0105] The application is in an abnormal decoding state, the application is in a silent video playing state, and the application is in a silent audio keep-alive state.

[0106] Optionally, the audio and video resources include a video decoder; the response module 420 includes:

[0107] The first response submodule is configured to release the video decoder occupied by the application during abnormal decoding in response to the application being in the background and the application being in an abnormal decoding state.

[0108] Optionally, the audio and video resources include a video decoder and an audio channel, and the response module 420 includes:

[0109] The second response submodule is configured to release the video decoder and audio channel occupied by the application when the application plays the video silently in response to the application being in the background and the application playing the video silently.

[0110] Optionally, the audio and video resources include an audio channel, and the response module 420 includes:

[0111] The third response submodule is configured to release the audio channel occupied by the application when it is kept alive in response to the application being in the background and the application being in a silent audio keep-alive state.

[0112] Optionally, the audio and video resource processing device 400 includes:

[0113] A marking module is configured to mark the audio channel occupied when releasing the application program to keep it alive, and obtain an identifier of the audio channel to be processed;

[0114] The recovery module is configured to recover the audio channel corresponding to the to-be-processed audio channel identifier if the to-be-processed audio channel identifier exists and the application does not request to release the audio channel corresponding to the to-be-processed audio channel identifier.

[0115] Optionally, the audio and video resource processing device 400 includes:

[0116] A timing module is configured to start timing when the application has applied for a video decoder, the video decoder is in use, and the application has not applied for an audio channel;

[0117] The abnormal decoding determination module is configured to determine that the application is in an abnormal decoding state when a preset time period is reached, the application is in the background, and the video decoder is in the use state.

[0118] Optionally, the audio and video resource processing device 400 includes:

[0119] The silent video playback determination module is configured to determine that the application is in the silent video playback state when the application has applied for a video decoder, the video decoder is in use, the application has applied for an audio channel, and the use state of the audio channel is silent playback.

[0120] Optionally, the audio and video resource processing device 400 includes:

[0121] The silent audio keep-alive determination module is configured to determine that the application is in the silent audio keep-alive state when the application does not apply for a video decoder but applies for an audio channel and the use status of the audio channel is silent playback.

[0122] Optionally, the audio and video resource processing device 400 includes:

[0123] The silent play determination module is configured to determine that the use state of the audio channel is silent play when at least one of the following conditions exists:

[0124] The ratio of the duration of writing empty data to the audio channel within a preset period to the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than a human ear perception threshold.

[0125] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0126] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for processing audio and video resources provided by the present disclosure are implemented.

[0127] Figure 51 is a block diagram of an apparatus 500 for processing audio and video resources according to an exemplary embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or other terminals.

[0128] Reference Figure 5 , the device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output interface 512 , a sensor component 514 , and a communication component 516 .

[0129] The processing component 502 generally controls the overall operation of the device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned audio and video resource processing method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0130] The memory 504 is configured to store various types of data to support operations on the device 500. Examples of such data include instructions for any application or method operating on the device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0131] The power supply component 506 provides power to the various components of the device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 500.

[0132] The multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0133] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the device 500 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0134] The input / output interface 512 provides an interface between the processing component 502 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0135] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the device 500. For example, the sensor assembly 514 can detect the open / closed state of the device 500, the relative positioning of components, such as the display and keypad of the device 500, the sensor assembly 514 can also detect the position change of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration / deceleration of the device 500, and the temperature change of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0136] The communication component 516 is configured to facilitate wired or wireless communication between the device 500 and other devices. The device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0137] In an exemplary embodiment, the device 500 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned audio and video resource processing method.

[0138] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by the processor 520 of the device 500 to complete the above-mentioned audio and video resource processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0139] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0140] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing audio and video resources, characterized in that: include: Make sure the app is in the background; In response to the application being in the background and in any of the following states, releasing the audio and video resources occupied by the application: The application is in an abnormal decoding state, the application is in a silent video playing state, and the application is in a silent audio keep-alive state.

2. The method according to claim 1, characterized in that The audio and video resources include a video decoder; in response to the application being in the background and the application being in an abnormal decoding state, releasing the audio and video resources occupied by the application, including: In response to the application being in the background and being in an abnormal decoding state, releasing a video decoder occupied by the application during abnormal decoding.

3. The method according to claim 1, characterized in that The audio and video resources include a video decoder and an audio channel. In response to the application being in the background and the application being in a silent video playing state, releasing the audio and video resources occupied by the application includes: In response to the application being in the background and being in a state of silently playing a video, the video decoder and the audio channel occupied by the application when the video is silently played are released.

4. The method according to claim 1, characterized in that: The audio and video resources include an audio channel, and in response to the application being in the background and the application being in a silent audio keep-alive state, releasing the audio and video resources occupied by the application, including: In response to the application being in the background and being in a silent audio keep-alive state, releasing the audio channel occupied by the application when it is kept alive.

5. The method according to claim 4, characterized in that The method further comprises: Marking the audio channel occupied when releasing the application program to keep it alive, and obtaining an identifier of the audio channel to be processed; In a case where the to-be-processed audio channel identifier exists and the application does not request to release the audio channel corresponding to the to-be-processed audio channel identifier, the audio channel corresponding to the to-be-processed audio channel identifier is restored.

6. The method according to claim 1, characterized in that The method further comprises: When the application has applied for a video decoder, the video decoder is in use, and the application has not applied for an audio channel, the timing starts; When the preset time duration is reached, the application is in the background, and the video decoder is in the use state, it is determined that the application is in an abnormal decoding state.

7. The method according to claim 1, characterized in that The method further comprises: When the application has applied for a video decoder, and the video decoder is in use, and the application has applied for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent video playback state.

8. The method according to claim 1, characterized in that: The method further comprises: When the application does not apply for a video decoder but applies for an audio channel, and the use status of the audio channel is silent playback, it is determined that the application is in a silent audio keep-alive state.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: In at least one of the following cases, it is determined that the use state of the audio channel is silent playback: The ratio of the duration of writing empty data to the audio channel within a preset period to the preset period is greater than a preset value, and the output frequency of writing non-empty data to the audio channel is less than a human ear perception threshold.

10. A device for processing audio and video resources, characterized in that: include: a determination module configured to determine that the application is in the background; The response module is configured to release the audio and video resources occupied by the application in response to the application being in the background and in any of the following states: The application is in an abnormal decoding state, the application is in a silent video playing state, and the application is in a silent audio keep-alive state.

11. A terminal, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Execute the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images