Method and device for detecting silent audio

Abstract

Methods and apparatus for detecting silent audio are disclosed. The method is applied to the electronic equipment and comprises the following steps: under the condition that a specific application program has an active audio track, determining an average energy value of audio data of the specific application program within a preset duration; in response to the fact that the average energy value is smaller than a preset threshold value, it is determined that the specific application program plays silent audio; and determining that the specific application program plays the silent audio in response to continuous preset times, and performing preset processing on the specific application program.

Description

Technical Field

[0001] The present disclosure generally relates to the field of electronic technology, and more particularly, to a method and apparatus for detecting silent audio. Background Art

[0002] As an electronic device, mobile terminals have become a necessity in people's daily lives. Many applications (i.e., APPs) can be installed on mobile terminals. In order to save resources and protect user privacy, the operating system of the mobile terminal usually restricts or even shuts down APPs running in the background. Some APPs use various keep-alive strategies to continue running in the background.

[0003] For example, in order to provide users with an immersive and friendly audio experience, mobile terminals generally allow audio apps to continue playing audio while running in the background. Therefore, some apps (including the most commonly used apps or the most popular apps) will use this feature to keep them alive by playing silent audio while running in the background.

[0004] However, keeping the phone alive by playing silent audio will cause the battery to be continuously consumed, especially after the mobile terminal turns off the screen. The APP playing silent audio will cause the mobile terminal to be unable to enter sleep mode. At the same time, the APP will occupy system resources and cause system freezes and other problems if it keeps running in the background. Summary of the invention

[0005] Embodiments of the present disclosure provide a method and apparatus for detecting silent audio, which can effectively solve the problem that an electronic device cannot normally enter a sleep mode.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for detecting silent audio is provided, the method being applied to an electronic device, the method comprising: in the case where a specific application has an active audio track, determining an average energy value of audio data of the specific application within a preset duration; in response to the average energy value being less than a preset threshold, determining that the specific application plays silent audio; and in response to determining that the specific application plays silent audio for a predetermined number of consecutive times, performing preset processing on the specific application.

[0007] Optionally, the specific application is an application that is switched to run in the background, and / or the specific application is an application that is active after the screen of the electronic device is turned off.

[0008] Optionally, when a specific application has an active audio track, the step of determining the average energy value of the audio data of the specific application within a preset duration includes: in response to the existence of an active audio track for the specific application, creating an instance for performing silent audio detection on the specific application in a pre-built unordered mapping container, wherein the instance includes the following information: identification information of the specific application, an energy superposition value, the number of energy superpositions, and instance creation time; based on the information included in the instance, determining the average energy value of the audio data of the specific application within the preset duration.

[0009] Optionally, the step of determining the average energy value of the audio data of the specific application within a preset duration includes: obtaining identification information of the application contained in each audio track in the electronic device and the energy value of the audio data; updating the energy superposition value and the number of energy superpositions of the instance based on the identification information of the application contained in each audio track; and determining the average energy value of the audio data of the specific application within the preset duration based on the updated energy superposition value and the number of energy superpositions after the preset duration.

[0010] Optionally, each audio track is pre-set with an energy value variable; wherein the step of obtaining the energy value of the audio data contained in each audio track in the electronic device includes: in response to each audio track obtaining audio data from the shared memory corresponding to it, respectively calculating the energy value of the audio data obtained by each audio track, and saving the calculated energy value in the energy value variable of each audio track respectively; obtaining the energy value of the audio data of each audio track from the energy value variable.

[0011] Optionally, the step of calculating the energy value of the audio data acquired for each audio track includes: selecting data of at least one sampling point from the acquired audio data, and calculating the energy value of the data of the at least one sampling point as the energy value of the acquired audio data.

[0012] Optionally, based on the identification information of the application contained in each audio track, the step of updating the energy superposition value and the number of energy superpositions of the instance includes: based on the identification information of the application contained in each audio track, identifying the energy value of the audio data of each audio track containing the identification information of the specific application; whenever a audio track containing the identification information of the specific application is identified, adding the energy value of its audio data to the energy superposition value of the instance, and increasing the number of energy superpositions of the instance by 1.

[0013] Optionally, the method further includes: in response to the time elapsed from the instance creation time of the instance exceeding a preset time threshold, destroying the instance; or, in response to determining that the specific application plays silent audio, destroying the instance; or, in response to the specific application being closed, destroying the instance.

[0014] Optionally, the preset duration used to determine the average energy value when determining for the first time that the specific application plays silent audio is less than the preset duration used to determine the average energy value when determining for the other times that the specific application plays silent audio.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided a device for detecting silent audio, wherein the device is arranged in an electronic device, and the device comprises: an energy value determination unit, configured to: in the case where a specific application has an active audio track, determine the average energy value of the audio data of the specific application within a preset duration; a silent audio detection unit, configured to: in response to the average energy value being less than a preset threshold, determine that the specific application plays silent audio; and a processing unit, configured to: in response to determining that the specific application plays silent audio for a predetermined number of consecutive times, perform preset processing on the specific application.

[0016] Optionally, the specific application is an application that is switched to run in the background, and / or the specific application is an application that is active after the screen of the electronic device is turned off.

[0017] Optionally, the energy value determination unit is configured to: in response to the existence of an active audio track of a specific application, create an instance for silent audio detection for the specific application in a pre-built unordered mapping container, wherein the instance includes the following information: identification information of the specific application, an energy superposition value, the number of energy superpositions and the instance creation time; based on the information contained in the instance, determine the average energy value of the audio data of the specific application within a preset duration.

[0018] Optionally, the energy value determination unit is configured to: obtain identification information of the application contained in each audio track in the electronic device and the energy value of the audio data; based on the identification information of the application contained in each audio track, update the energy superposition value and the number of energy superpositions of the instance; based on the updated energy superposition value and the number of energy superpositions after the preset time length, determine the average energy value of the audio data of the specific application within the preset time length.

[0019] Optionally, each audio track is pre-set with an energy value variable; wherein the energy value determination unit is configured to: in response to each audio track obtaining audio data from the shared memory corresponding to it, calculate the energy value of the audio data obtained by each audio track respectively, and save the calculated energy value in the energy value variable of each audio track respectively; obtain the energy value of the audio data of each audio track from the energy value variable.

[0020] Optionally, the energy value determining unit is configured to: select data of at least one sampling point from the acquired audio data, and calculate the energy value of the data of the at least one sampling point as the energy value of the acquired audio data.

[0021] Optionally, the energy value determination unit is configured to: identify the energy value of the audio data of each audio track containing the identification information of the specific application based on the identification information of the application contained in each audio track; whenever a audio track containing the identification information of the specific application is identified, add the energy value of its audio data to the energy superposition value of the instance, and increase the energy superposition times of the instance by 1.

[0022] Optionally, the processing unit is configured to: destroy the instance in response to the time elapsed from the instance creation time of the instance exceeding a preset time threshold; or, destroy the instance in response to determining that the specific application plays silent audio; or, destroy the instance in response to the specific application being closed.

[0023] Optionally, the preset duration used to determine the average energy value when determining for the first time that the specific application plays silent audio is less than the preset duration used to determine the average energy value when determining for the other times that the specific application plays silent audio.

[0024] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided. When the computer program is executed by a processor, the method for detecting silent audio as described above is implemented.

[0025] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory storing a computer program, wherein when the computer program is executed by the processor, the method for detecting silent audio as described above is implemented.

[0026] According to the method and device for detecting silent audio in the embodiments of the present disclosure, it is possible to accurately detect an APP that plays silent audio in real time without substantially affecting the system performance of the electronic device, and by processing the detected APP that plays silent audio, it is possible to ensure that the electronic device enters sleep mode normally, thereby reducing device energy consumption and saving system resources.

[0027] Additional aspects and / or advantages of the present general inventive concept will be set forth in part in the following description and in part will be apparent from the description or may be learned through practice of the present general inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects and features of exemplary embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings which exemplarily illustrate the embodiments.

[0029] Figure 1 is a diagram showing a framework for executing a method for detecting silent audio according to an embodiment of the present disclosure;

[0030] Figure 2 is a flowchart illustrating a method for detecting silent audio according to an embodiment of the present disclosure;

[0031] Figure 3 is a schematic diagram showing the transmission of audio data between AudioTrack and Track according to an embodiment of the present disclosure;

[0032] Figure 4 is a diagram showing an example of acquiring audio data for each shared memory according to an embodiment of the present disclosure;

[0033] Figure 5 is a block diagram showing an apparatus for detecting silent audio according to an embodiment of the present disclosure;

[0034] Figure 6 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Hereinafter, various embodiments of the present disclosure are described with reference to the accompanying drawings, wherein the same reference numerals are used to represent the same or similar elements, features and structures. However, it is not intended that the present disclosure be limited to specific embodiments by the various embodiments described herein, and it is intended that: the present disclosure covers all modifications, equivalents and / or substitutes of the present disclosure, as long as they are within the scope of the attached claims and their equivalents. The terms and words used in the following specification and claims are not limited to their dictionary meanings, but are only used to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only, and not for the purpose of limiting the present disclosure defined by the attached claims and their equivalents.

[0036] It should be understood that the singular includes the plural unless the context clearly indicates otherwise.The terms "include", "comprising" and "having" used herein indicate the presence of disclosed functions, operations or elements, but do not exclude other functions, operations or elements.

[0037] For example, the expression "A or B" or "at least one of A and / or B" may indicate A and B, A or B. For example, the expression "A or B" or "at least one of A and / or B" may indicate (1) A, (2) B, or (3) both A and B.

[0038] In various embodiments of the present disclosure, it is intended that when a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component) or being "coupled" or "connected" to another component (e.g., the second component), the component may be directly connected to the other component or may be connected through another component (e.g., a third component). In contrast, when a component (e.g., a first component) is referred to as being "directly coupled" or "directly connected" to another component (e.g., the second component) or being directly coupled to or directly connected to another component (e.g., the second component), there is no other component (e.g., the third component) between the component and the other component.

[0039] The expression "configured to" used in describing various embodiments of the present disclosure may be used interchangeably with expressions such as "suitable for", "having the ability to", "designed to", "suitable for", "manufactured to", and "capable of", for example, depending on the circumstances. The term "configured to" may not necessarily indicate that it is "specially designed to" in terms of hardware. On the contrary, the expression "a device configured to..." in some cases may indicate that the device and another device or part "can..." For example, the expression "a processor configured to perform A, B, and C" may indicate a dedicated processor (e.g., an embedded processor) for performing the corresponding operations or a general-purpose processor (e.g., a central processing unit CPU or an application processor (AP)) for performing the corresponding operations by executing at least one software program stored in a memory.

[0040] The terms used herein are to describe certain embodiments of the present disclosure, but are not intended to limit the scope of other embodiments. Unless otherwise noted herein, all terms used herein (including technical or scientific terms) may have the same meaning as those generally understood by those skilled in the art. Typically, the terms defined in the dictionary should be considered to have the same meaning as the contextual meaning in the relevant field, and, unless clearly defined herein, should not be understood differently or understood to have an overly formal meaning. In any case, the terms defined in the present disclosure are not intended to be interpreted as excluding embodiments of the present disclosure.

[0041] Refer to the following Figures 1 to 6 A method and apparatus for detecting silent audio according to an embodiment of the present disclosure are described in detail.

[0042] Figure 1is a diagram showing a framework for executing a method for detecting silent audio according to an embodiment of the present disclosure. The method for detecting silent audio according to an embodiment of the present disclosure can be executed in an application layer (Application) and an application framework layer (FW) of an electronic device. Specifically, the method for detecting silent audio according to an embodiment of the present disclosure can be mainly executed by a system application (System APP) in the application layer and an AudioFlinger module in the application framework layer.

[0043] First refer to Figure 1 A framework for implementing a method for detecting silent audio is explained.

[0044] like Figure 1 As shown, when an APP (e.g., APP X) creates an AudioTrack to play audio, a Track (i.e., the audio track in this disclosure) that is bound to the AudioTrack one by one is created in AudioFlinger. APP X can support multiple audio formats such as MP3, WAV, FLAC, PCM, etc. at the same time, but AudioTrack can only process PCM audio streams. Therefore, it is necessary to decode the audio streams in MP3, WAV, FLAC, etc. formats of APP X into PCM streams through the MediaPlayerService module, and then provide the decoded PCM streams to the corresponding AudioTrack. One audio format can correspond to one AudioTrack. Figure 1 4 AudioTracks of APP X are shown in FIG. 1 , namely AudioTrack1, AudioTrack2, AudioTrack3 and AudioTrack4, but the number of AudioTracks is not limited thereto. Meanwhile, four Tracks, namely Track1, Track2, Track3 and Track4, are created in AudioFlinger and are respectively bound to the four AudioTracks.

[0045] As a submodule in AudioFlinger, PlaybackThread is responsible for transmitting playback data (i.e., audio data) from the application layer to the Audio HAL layer (audio hardware abstraction layer). According to an embodiment of the present disclosure, PlaybackThread can obtain the corresponding energy value while obtaining the playback data, and can provide the energy value to SilentAudio.

[0046] SilentAudio is a submodule specially constructed in the AudioFlinger module, which is used to store an unordered map container (i.e., mSilentAudioMap), and can update the relevant data in the unordered map container according to the energy value provided by PlaybackThread. The unordered map container (mSilentAudioMap) will be explained in detail later.

[0047] The Audio HAL layer is usually responsible for interacting with hardware devices (speakers, headphones, microphones, etc.) and is used to provide playback data to the hardware devices for playback.

[0048] The AudioFlinger module may also include a SecAudioParamFlinger submodule. The SecAudioParamFlinger submodule is used to parse and execute instructions issued by the system application through the setParameters interface, thereby creating an instance of silent audio detection for APP X in the unordered mapping container (mSilentAudioMap) stored in SilentAudio. On the other hand, the SecAudioParamFlinger submodule can respond to the instructions issued by the system application through the getParameters interface, and provide the detection result of silent audio detection for APP X (for example, APP X plays silent audio or APP X does not play silent audio) to the system application, so that the system application performs preset processing on APP X, and the preset processing will be explained later.

[0049] Combine the following Figure 1 and Figure 2 A method for detecting silent audio and its execution framework are further described.

[0050] Figure 2 is a flow chart showing a method for detecting silent audio according to an embodiment of the present disclosure. The method for detecting silent audio according to an embodiment of the present disclosure can be applied to an electronic device, which may be a smart phone, a tablet device, a personal digital assistant, or other mobile terminals.

[0051] Reference Figure 1 and Figure 2 In step S201, when there is an active audio track for a specific application (e.g., APP X), the average energy value of the audio data of APP X within a preset duration is determined. According to an embodiment of the present disclosure, the average energy value of the audio data of APP X within a preset duration can be determined by an instance of performing silent audio detection on APP X created in a pre-constructed unordered mapping container (mSilentAudioMap).

[0052] As described above, the electronic device may pre-create an unordered mapping container (mSilentAudioMap) in the AudioFlinger module responsible for managing and scheduling the transmission of audio data. For example, a SilentAudio submodule may be specifically constructed in the AudioFlinger module to store the created unordered mapping container. In this way, when APP X switches to background operation, and / or when APP X is active when the electronic device is on the screen, it is detected that there is an active audio track in APP X. The system application may issue an instruction by calling the setParameters interface, and the SecAudioParamFlinger submodule may parse the instruction and transmit the parsed instruction to the SilentAudio submodule. The SilentAudio submodule may create an instance of silent audio detection for APP X in the unordered mapping container based on the parsed instruction and according to the identification information of APP X (such as but not limited to the unique identifier (UID) of APP X, the package name, etc.).

[0053] The instance of performing silent audio detection on APP X may include a key value and a data value. The key value may represent the identification information of the application (the unique identifier (UID) of the application as described above, the package name, etc.). In this way, the system application can find the instance of performing silent audio detection on APP X according to the identification information of the application. The data value may include an energy superposition value (sum), an energy superposition count (count), and an instance creation time (time). The energy superposition count is used to record the number of times the energy value of the audio data of the application is obtained in the process of detecting silent audio once, the energy superposition value is used to accumulate the energy value of the audio data of the application obtained in the process of detecting silent audio once, and the instance creation time is used to indicate the moment when the instance is created.

[0054] Therefore, more specifically, in step S201, first, in response to the existence of an active audio track of APP X, an instance of silent audio detection for APP X may be created in a pre-constructed unordered mapping container (mSilentAudioMap), and then based on the information contained in the instance, the average energy value of the audio data of APP X within a preset duration may be determined. As described above, the instance may include the following information: identification information of the specific application, energy superposition value, energy superposition times, and instance creation time.

[0055] Here, it should be noted that an APP can have multiple Tracks simultaneously to perform different audio playback tasks. For example, an APP can have multiple Tracks for playing background music, playing sound effects, playing voice messages, etc. In addition, an APP may also frequently create / close Tracks. For example, when a music APP switches songs, it will close the Track that plays the song before the switch and recreate a new Track to play the song after the switch. Therefore, when obtaining the energy value of the audio data of an APP, it is necessary to add up the energy values ​​of all the Tracks of the APP.

[0056] On the other hand, the system application can detect the existence of an active audio track of a specific application through the following operations, wherein the specific application (i.e., APP X) can be an application switched to the background, and / or an application that is active after the electronic device turns off the screen (including but not limited to various applications that are in the foreground or background before the screen turns off). Specifically, the system application can monitor the switching between the foreground and background of APP X, the screen off state of the electronic device, and the active state of the application. When it is detected that APP X enters the background, and / or the electronic device enters the screen off state and APP X is still active, the system application can further determine whether APP X has an active Track, that is, determine whether APP X is playing audio. When APP X is playing audio, AudioFlinger will add the Track of APP X to the mActiveTracks list; and when APP X pauses / stops playing audio, AudioFlinger will remove the Track of APP X from the mActiveTracks list. Therefore, the system application can traverse all Tracks in the mActiveTracks list by using an interface (for example, isUsingAudio(packageName,uid)) to check whether there is a Track of APP X in the mActiveTracks list, that is, check the Track that carries the packageName / uid (package name / unique identifier) ​​of APP X. If it is checked that there is a Track of APP X in the mActiveTracks list, it means that APP X has an active Track.

[0057] When APP X plays audio, AudioTrack and Track can transmit audio data through shared memory. That is, APP X continuously writes the audio data to be played into the buffer (i.e., shared memory) of AudioTrack, while Track continuously obtains audio data from the shared memory of AudioTrack bound to it and calculates the energy value of the audio data in real time.

[0058] Figure 3 2 is a schematic diagram showing the transmission of audio data between AudioTrack and Track according to an embodiment of the present disclosure. Figure 3 As shown, Track can obtain audio data from the shared memory through the getNextBuffer instruction, calculate the energy value of the obtained audio data, and save the calculated energy value of the audio data in the energy value variable (mEnergy).

[0059] According to an embodiment of the present disclosure, after obtaining audio data from AudioTrack through shared memory, Track may select data of at least one sampling point from the obtained audio data, and calculate the energy value of the data of the at least one sampling point as the energy value of the obtained audio data. Here, the audio data obtained from the shared memory each time may correspond to multiple sampling points. In order to reduce the amount of calculation of the energy value of the audio data, thereby reducing the impact on the system performance of the electronic device, the data of at least one sampling point may be selected to calculate the energy value. For example, the at least one sampling point may be the first sampling point among multiple sampling points.

[0060] Assume that each shared memory can store 20ms of audio data, but the present disclosure is not limited to this. Generally speaking, the duration of the audio data stored in the shared memory each time is fluctuating to prevent malicious attacks. For each acquired audio data, the data of the first sampling point is taken to calculate its energy value. Figure 4 is a diagram illustrating an example of acquiring audio data for each shared memory according to an embodiment of the present disclosure. Figure 4 The upper part shows the waveform of the original audio data, whose amplitude changes with time. Figure 4 The lower part of shows the audio data of the first sampling point. Figure 4 As shown, audio data at 20ms, 40ms, 60ms, 80ms, etc. are taken. In this way, the sampling points can be evenly distributed in the entire audio data, which can basically reflect the characteristics of the entire audio data, and can quickly obtain the energy value of the audio data while ensuring the accuracy of detecting silent audio.

[0061] According to an embodiment of the present disclosure, while each Track selects data of at least one sampling point from the acquired audio data and calculates the energy value of the data of the at least one sampling point, each PlaybackThread may poll all Tracks associated with it (for example, Figure 1 , Track1, Track2, Track3, and Track4) shown in the figure to obtain the energy value variables of all Tracks. Subsequently, each PlaybackThread can provide the energy value variables of all Tracks it has obtained to the SilentAudio submodule. This is because PlaybackThread and Track are in a one-to-many relationship, that is, a PlaybackThread can have multiple associated Tracks. Generally speaking, the hardware device that outputs audio determines the corresponding PlaybackThread, and different APPs using the same hardware device that outputs audio can correspond to the same PlaybackThread, but the present disclosure is not limited to this. Therefore, when PlaybackThread polls all associated Tracks, these associated Tracks may not only be Tracks of APP X, but also include Tracks of other APPs. On the other hand, if an APP uses different hardware devices to output audio, multiple Tracks of the APP (for example, Track1, Track2, Track3, and Track4 of APP X) may also correspond to different PlaybackThreads.

[0062] According to an embodiment of the present disclosure, each Track may be pre-set with mEnergy for storing the calculated energy value of the audio data. In addition, each AudioTrack and Track may contain identification information of the application that created them (such as but not limited to a unique identifier (UID), package name, etc.). For example, Figure 1 The illustrated AudioTrack1, AudioTrack2, AudioTrack3, and AudioTrack4 may include identification information of APP X. Similarly, Track1, Track2, Track3, and Track4 may also include identification information of APP X.

[0063] Therefore, when obtaining the energy value of the audio data contained in each Track in the electronic device, first, in response to each Track, the audio data can be obtained from the shared memory corresponding to it, the energy value of the audio data obtained by each Track can be calculated respectively, and the calculated energy value can be saved in the mEnergy of each Track respectively; then, the energy value of its audio data can be obtained from the mEnergy of each Track. For example, each Track can obtain audio data from the shared memory corresponding to it, calculate the energy value of the obtained audio data, and save the calculated energy value in its energy value variable.

[0064] Then, PlaybackThread can obtain the energy value of the audio data from the energy value variable of each Track associated with it. At the same time, PlaybackThread can also obtain the identification information of the application that created the Track (for example, the identification information of APP X) from each Track associated with it. That is, PlaybackThread can obtain the identification information of the application contained in each Track in the electronic device and the energy value of the audio data. Then, PlaybackThread can pass the obtained data (that is, the identification information of the application and the energy value of the audio data) to the SilentAudio submodule, so that the instance of silent audio detection for APP X created in the unordered mapping container can update the energy superposition value (sum) and the number of energy superpositions (count) based on the identification information of the application contained in each Track. Finally, based on the updated energy superposition value (sum) and the number of energy superpositions (count) after the preset duration, the average energy value of the audio data of APP X within the preset duration can be determined. According to an embodiment of the present disclosure, after a preset duration has passed, the SilentAudio submodule may determine the average energy value of the audio data of APP X within the preset duration based on the energy superposition value (sum) and the number of energy superpositions (count) contained in the instance, and the SecAudioParamFlinger submodule may determine the average energy value of the audio data of APP X within the preset duration based on the energy superposition value (sum) and the number of energy superpositions (count) contained in the instance. However, the present disclosure is not limited thereto, and the average energy value of the audio data may be determined by any submodule in AudioFlinger. According to an embodiment of the present disclosure, the energy superposition value (sum) may be divided by the number of energy superpositions (count) to calculate the average energy value of the audio data of APP X. That is, the average energy value of the audio data of APP X is equal to sum / count.

[0065] More specifically, when the energy superposition value (sum) and the energy superposition number (count) are updated based on the identification information of the application included in each Track, the instance of performing silent audio detection on APP X may identify the energy value of the audio data of each Track containing the identification information of APP X based on the identification information of the application included in each Track. Whenever a Track containing the identification information of APP X is identified, the instance of performing silent audio detection on APP X may add the energy value of the audio data of the Track to the energy superposition value (sum) and increase the energy superposition number (count) by 1. For example, when Track 1 containing the identification information of APP X is identified, the energy value of the audio data of Track 1 may be added to the energy superposition value (sum), and the energy superposition number (count) may be increased by 1. Then, when Track 2 containing the identification information of APP X is identified, the energy value of the audio data of Track 2 may be added to the energy superposition value (sum), and the energy superposition number (count) may be increased by 1, and so on.

[0066] Return to reference Figure 1 In step S202, in response to the average energy value being less than the preset threshold, it can be determined that APP X plays silent audio. As described above, the SecAudioParamFlinger submodule or the SilentAudio submodule can determine the average energy value of the audio data of APP X, and when the average value of the audio data of APP X is less than the preset threshold, it can be determined that APP X plays silent audio.

[0067] According to an embodiment of the present disclosure, the preset threshold value may be in the range of 0.001 to 0.005, but the present disclosure is not limited thereto. On the other hand, when the average energy value of the audio data of APP X is greater than or equal to the preset threshold value, it can be determined that APP X does not play silent audio. On the other hand, the system application may issue an instruction through the getParameters interface to obtain the result of silent audio detection for APP X from the SecAudioParamFlinger submodule.

[0068] Since the application may be silent / silent for a period of time during the normal audio playback process, there may be a misjudgment in determining whether APP X plays silent audio at a single time. In order to solve this problem, the method for detecting silent audio according to an embodiment of the present disclosure can determine whether APP X plays silent audio multiple times to ensure the accuracy of detecting silent audio. Therefore, in step S203, in response to determining that APP X plays silent audio for a predetermined number of consecutive times, APP X can be preset. In theory, the system application can obtain the result of silent audio detection for APP X by calling the getParameters interface at any time after calling the setParameters interface. However, if the time interval between calling the setParameters interface and calling getParameters is too short (for example, but not limited to 1s), the detection result will be inaccurate due to too little sampled data; and if the time interval is too long, the system performance will be affected due to continuous sampling of data. Therefore, it is necessary to set a suitable interval time (that is, the time for determining whether APP X plays silent audio at a single time). On the other hand, if the number of times of determining whether APP X plays silent audio is too many, or the duration of each determination of whether APP X plays silent audio is too long, it will affect the system performance of the electronic device. Therefore, according to the embodiments of the present disclosure, the number of times of determining whether APP X plays silent audio and the duration of each determination of whether APP X plays silent audio can be reasonably set according to actual conditions to ensure the real-time and accuracy of silent audio detection and minimize the impact on system performance.

[0069] According to an embodiment of the present disclosure, the predetermined number of times may be set to, for example, 6 times, but the present disclosure is not limited thereto. For example, the preset number of times may be less than 6 times (for example, but not limited to 5 times), or more than 6 times (for example, but not limited to 7 times). However, as described above, in order to avoid affecting the system performance of the electronic device, the preset number of times should not be set too much (for example, but not limited to more than 10 times). Optionally, the duration of the first determination of whether APP X plays silent audio may be less than the duration of each other determination of whether APP X plays silent audio. In other words, the preset duration of the first determination of the average energy value may be less than the preset duration of each other determination of the average energy value. Here, the preset duration of the first determination of the average energy value is set relatively small in order to facilitate and quickly determine roughly whether APP X plays silent audio.

[0070] According to an embodiment of the present disclosure, performing a preset process on APP X may include marking APP X. In this way, the system application may freeze or close the marked APP X at any time as needed. However, the present disclosure is not limited thereto, for example, performing a preset process on APP X may also be directly closing or freezing APP X.

[0071] Optionally, after creating an instance of silent audio detection for APP X in the unordered mapping container (mSilentAudioMap), the created instance can be destroyed in the following circumstances to reduce energy consumption. First, in response to the time elapsed from the instance creation time (time) of the instance exceeding the preset time threshold, the instance can be destroyed. Specifically, setParameters and getParameters are two independent interfaces. When an exception occurs after the system application calls setParameters to create an instance of silent audio detection for APP X (such as an error in the system application, the system application does not call getParameters due to an unexpected logic vulnerability, etc.), it will appear that after the instance is created, it has not been possible to determine whether APP X plays silent audio. In this case, the instance of silent audio detection for APP X will continue, causing system freezes, increased power consumption and other problems. Therefore, the instance can be automatically destroyed when the time elapsed from the instance creation time (time) of the instance exceeds the preset time threshold to avoid the above situation.

[0072] Alternatively, in response to determining that APP X plays silent audio, the instance may be destroyed, or in response to APP X being closed (for example, the user actively closes APP X, APP X is accidentally closed due to a system failure, etc.), the instance may be destroyed. Here, it should be noted that since the instance is destroyed when it is determined that APP X plays silent audio, each time silent audio detection is performed on APP X, an instance for performing silent audio detection on APP X must be recreated.

[0073] The above specifically describes the method for detecting silent audio applied to an electronic device. The following briefly describes the device for detecting silent audio.

[0074] Figure 5 is a block diagram showing an apparatus for detecting silent audio according to an embodiment of the present disclosure. The apparatus for detecting silent audio may be provided in an electronic device and distributed in an application layer and an application framework layer of the electronic device.

[0075] Reference Figure 5 , the apparatus 500 for detecting silent audio may include an energy value determining unit 501 , a silent audio detecting unit 502 and a processing unit 503 .

[0076] The energy value determination unit 501 may determine the average energy value of the audio data of a specific application (e.g., APP X) within a preset time period when there is an active audio track of the specific application (e.g., APP X). APP X may be an application switched to the background, or may be an application that is active after the screen of the electronic device is turned off.

[0077] According to an embodiment of the present disclosure, the energy value determination unit 501 may create an instance of silent audio detection for APP X in a pre-constructed unordered mapping container (mSilentAudioMap) in response to the presence of an active audio track of APP X, and may determine the average energy value of the audio data of APP X within a preset duration based on the information contained in the instance. The instance may include the following information: identification information of APP X, energy superposition value (sum), number of energy superpositions (count), and instance creation time (time).

[0078] Further, the energy value determination unit 501 may obtain the identification information of the application program contained in each audio track in the electronic device and the energy value of the audio data, update the energy superposition value (sum) and the number of energy superpositions (count) of the instance based on the identification information of the application program contained in each audio track, and determine the average energy value of the audio data of APP X within the preset time length based on the updated energy superposition value (sum) and the number of energy superpositions (count) after the preset time length. Each audio track may be pre-set with an energy value variable (mEnergy). In this way, the energy value determination unit 501 may obtain audio data from the shared memory corresponding to each audio track in response to each audio track, calculate the energy value of the audio data obtained by each audio track, and save the calculated energy value in the energy value variable (mEnergy) of each audio track, and obtain the energy value of its audio data from the energy value variable (mEnergy) of each audio track.

[0079] When calculating the energy value of the audio data, the energy value determination unit 501 may select data of at least one sampling point from the acquired audio data, and calculate the energy value of the selected data of the at least one sampling point as the energy value of the acquired audio data.

[0080] According to an embodiment of the present disclosure, the energy value determination unit 501 may identify the energy value of the audio data of each audio track containing the identification information of APP X based on the identification information of the application program contained in each audio track. Whenever a audio track containing the identification information of APP X is identified, the energy value determination unit 501 may add the energy value of its audio data to the energy superposition value (sum) of the instance of performing silent audio detection on APP X, and increase the energy superposition count (count) of the instance by 1.

[0081] The silent audio detection unit 502 may determine that APP X plays silent audio in response to the average energy value determined by the energy value determination unit 501 being less than a preset threshold.

[0082] The processing unit 503 may perform a preset process on the specific application program of APP X in response to determining that APP X plays silent audio for a predetermined number of consecutive times.

[0083] According to an embodiment of the present disclosure, the processing unit 503 may destroy the instance in response to the time elapsed from the instance creation time (i.e., time) of the instance exceeding a preset time threshold; or, in response to determining that APP X plays silent audio, destroy the instance; or, in response to APP X being closed, destroy the instance.

[0084] According to an embodiment of the present disclosure, the duration of the first determination of whether APP X plays silent audio may be shorter than the duration of other determinations of whether APP X plays silent audio. In other words, the preset duration of the first determination of the average energy value may be shorter than the preset duration of other determinations of the average energy value.

[0085] The method for detecting silent audio according to an embodiment of the present disclosure may be written as a computer program and stored on a computer-readable storage medium. When the computer program is executed by a processor, the method for detecting silent audio as described above may be implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0086] Figure 6 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0087] Reference Figure 6 The electronic device 600 may include at least a memory 601 and a processor 602. The memory 601 stores a computer program, and when the computer program is executed by the processor 602, the method for detecting silent audio according to an embodiment of the present disclosure is implemented.

[0088] As an example, the electronic device 600 may be a smart phone, a tablet device, a personal digital assistant, or other mobile terminal capable of executing the above-mentioned computer program. In the electronic device 600, the processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not a limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc. The processor 602 may run instructions or codes stored in the memory 601, wherein the memory 601 may also store data. Instructions and data may also be sent and received through a network via a network interface device, wherein the network interface device may use any known transmission protocol. The memory 601 may be integrated with the processor 602, for example, a RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the memory 601 may include an independent device, such as an external disk drive, a storage array, or other storage devices that can be used by any database system. The memory 601 and the processor 602 may be operatively coupled or may be in communication with each other, such as through an I / O port, a network connection, etc., so that the processor 602 can read files stored in the memory.

[0089] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0090] According to the method and device for detecting silent audio in the embodiments of the present disclosure, it is possible to accurately detect an APP that plays silent audio in real time without substantially affecting the system performance of the electronic device, and by processing the detected APP that plays silent audio, it is possible to ensure that the electronic device enters sleep mode normally, thereby reducing device energy consumption and saving system resources.

[0091] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0092] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting silent audio, the method being applied to electronic equipment, characterized in that: The method comprises: In the case where there is an active audio track for a specific application, determining an average energy value of the audio data of the specific application within a preset duration; In response to the average energy value being less than a preset threshold, determining that the specific application is playing silent audio; In response to determining that the specific application plays silent audio for a predetermined number of consecutive times, a preset process is performed on the specific application.

2. The method according to claim 1, characterized in that The specific application is an application that is switched to run in the background, and / or the specific application is an application that is active after the screen of the electronic device is turned off.

3. The method according to claim 1, characterized in that The step of determining the average energy value of the audio data of the specific application within a preset duration when there is an active audio track of the specific application includes: In response to the existence of an active audio track of a specific application, an instance for performing silent audio detection on the specific application is created in a pre-constructed unordered mapping container, wherein the instance includes the following information: identification information of the specific application, an energy stacking value, a number of energy stacking times, and an instance creation time; Based on the information included in the instance, an average energy value of the audio data of the specific application within a preset time length is determined.

4. The method according to claim 3, characterized in that The step of determining the average energy value of the audio data of the specific application within a preset time period includes: Obtaining identification information of an application program included in each audio track in the electronic device and an energy value of the audio data; Based on the identification information of the application program included in each audio track, the energy stacking value and the energy stacking times of the instance are updated; Based on the updated energy superposition value and the number of energy superpositions after the preset time period, an average energy value of the audio data of the specific application within the preset time period is determined.

5. The method according to claim 4, characterized in that Each audio track is pre-set with an energy value variable; The step of obtaining the energy value of the audio data contained in each audio track in the electronic device includes: In response to each audio track acquiring audio data from the shared memory corresponding thereto, respectively calculating energy values ​​of the audio data acquired by each audio track, and respectively storing the calculated energy values ​​in energy value variables of each audio track; Get the energy value of the audio data from the energy value variable of each audio track.

6. The method according to claim 5, characterized in that The steps of calculating the energy value of the audio data obtained for each audio track include: Data of at least one sampling point is selected from the acquired audio data, and an energy value of the data of the at least one sampling point is calculated as the energy value of the acquired audio data.

7. The method according to claim 4, characterized in that Based on the identification information of the application program included in each audio track, the step of updating the energy superposition value and the energy superposition times of the instance includes: Based on the identification information of the application program included in each audio track, identifying the energy value of the audio data of each audio track including the identification information of the specific application program; Whenever an audio track containing identification information of the specific application is identified, the energy value of its audio data is added to the energy stacking value of the instance, and the energy stacking times of the instance is increased by 1.

8. The method according to claim 3, characterized in that The method further comprises: In response to the time elapsed from the instance creation time of the instance exceeding a preset time threshold, the instance is destroyed; or, in response to determining that the specific application plays silent audio, the instance is destroyed; or, in response to the specific application being closed, the instance is destroyed.

9. The method according to claim 1, characterized in that The preset time length for determining the average energy value for the first time is shorter than the preset time lengths for determining the average energy value for other times.

10. A device for detecting silent audio, the device being arranged in an electronic device, characterized in that: The device comprises: The energy value determination unit is configured to: determine an average energy value of the audio data of the specific application within a preset time length when there is an active audio track of the specific application; a silent audio detection unit, configured to: in response to the average energy value being less than a preset threshold, determine that the specific application is playing silent audio; The processing unit is configured to: in response to determining that the specific application plays silent audio for a predetermined number of consecutive times, perform preset processing on the specific application.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting silent audio according to any one of claims 1 to 9 is implemented.

12. An electronic device, characterized in that: The electronic device comprises: processor; A memory storing a computer program, which, when executed by the processor, implements the method for detecting silent audio as claimed in any one of claims 1 to 9.

Images