Audio playing control method and device, electronic equipment and storage medium

Abstract

The invention discloses an audio playing control method and device, electronic equipment and a storage medium. The embodiment of the invention relates to the technical field of cloud technology and the like. The method comprises the steps of collecting a first collection environment sound signal in an environment where a playing device is located in a process of playing a target audio by the playing device; determining the influence degree of the environmental noise on audio playing according to the playing equipment information of the playing equipment; determining a noise energy value according to the first collection environment sound signal; and if the noise energy value is higher than the energy threshold value corresponding to the influence degree, pausing playing of the target audio. In the application, the situation that the content in the target audio is lost due to the fact that the user cannot hear the content in the target audio clearly when the target audio is continuously played due to large noise influence in the environment is avoided, and the playing experience of the target audio is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly, to an audio playback control method, apparatus, electronic device, and storage medium. Background Art

[0002] Currently, users can obtain various rich entertainment experiences by listening to audio, and can also acquire knowledge by listening to audio.

[0003] However, during the process of playing audio on a terminal, there are various noises in the environment where the terminal is located, and it is difficult for users to clearly hear the content in the audio accurately, resulting in a poor audio playback experience. Therefore, there is an urgent need for a means to improve the audio playback experience. Summary of the Invention

[0004] In view of this, embodiments of this application propose an audio playback control method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, embodiments of this application provide an audio playback control method, the method including: during the process of playing a target audio on a playback device, collecting a first collected ambient sound signal in the environment where the playback device is located; determining the influence degree of ambient noise on audio playback according to the playback device information of the playback device; determining a noise energy value according to the first collected ambient sound signal; if the noise energy value is higher than the energy threshold corresponding to the influence degree, pausing the playback of the target audio, and the energy threshold is negatively correlated with the influence degree.

[0006] In a second aspect, embodiments of this application provide an audio playback control apparatus, the apparatus including: a collection module, configured to collect a first collected ambient sound signal in the environment where the playback device is located during the process of playing a target audio on the playback device; an influence degree determination module, configured to determine the influence degree of ambient noise on audio playback according to the playback device information of the playback device; an energy value determination module, configured to determine a noise energy value according to the first collected ambient sound signal; a pause module, configured to pause the playback of the target audio if the noise energy value is higher than the energy threshold corresponding to the influence degree, and the energy threshold is negatively correlated with the influence degree.

[0007] Optionally, the influence degree determination module is further configured to, if the playback device information indicates that the playback device is of an external speaker type, determine the influence degree as a first influence degree; if the playback device information indicates that the playback device is of a headphone type and the noise reduction function of the playback device is not enabled, determine the influence degree as a second influence degree, the first influence degree being higher than the second influence degree; if the playback device information indicates that the playback device is of a headphone type and the noise reduction function of the playback device is enabled, determine the influence degree according to the playback volume of the playback device in the playback device information.

[0008] Optionally, the influence degree determination module is further configured to: if the playback volume of the playback device is higher than the volume threshold, determine the influence degree as the third influence degree, where the second influence degree is higher than the third influence degree; if the playback volume of the playback device is not higher than the volume threshold, determine the influence degree as the fourth influence degree, where the third influence degree is higher than the fourth influence degree.

[0009] Optionally, the first ambient sound signal is collected at a target time point; the apparatus further includes a playback module, configured to cache the paused playback progress position of the target audio; obtain a second ambient sound signal in the environment where the playback device is located, which is collected after the target time point; if the noise energy value determined according to the second ambient sound signal is not higher than the energy threshold corresponding to the influence degree, continue to play the target audio based on the paused playback progress position.

[0010] Optionally, the playback module is further configured to determine a target progress position whose duration between the position before the paused playback progress position and the paused playback progress position is equal to a preset duration; continue to play the target audio from the target progress position of the target audio.

[0011] Optionally, the energy value determination module is further configured to: if the playback device information indicates that the playback device is of the external speaker type, obtain the original playback sound signal of the target audio; subtract the original playback sound signal from the first ambient sound signal to obtain the actual ambient sound signal in the environment where the playback device is located; determine the noise energy value according to the actual ambient sound signal.

[0012] Optionally, the energy value determination module is further configured to: if the playback device information indicates that the playback device is of the headphone type, use the first ambient sound signal as the actual ambient sound signal in the environment where the playback device is located; determine the noise energy value according to the actual ambient sound signal.

[0013] Optionally, the energy value determination module is further configured to filter out non-ambient noise signals in the actual ambient sound signal to obtain an ambient noise signal; determine the audio energy value of the ambient noise signal as the noise energy value.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the above method is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the above method is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program including computer instructions, and when the computer instructions are executed by a processor, the above method is implemented.

[0017] An audio playback control method, apparatus, electronic device, and storage medium provided by an embodiment of the present application. In the present application, first, during the process of playing a target audio on a playback device, a first ambient sound signal in the environment where the playback device is located is collected; then, according to the playback device information of the playback device, the influence degree of environmental noise on audio playback is determined, and according to the first ambient sound signal, the noise energy value is determined; if the noise energy value is higher than the energy threshold corresponding to the influence degree, it is determined that the noise in the environment has a greater impact and the user cannot clearly hear the content in the target audio. At this time, the playback of the target audio is paused, thereby avoiding the situation where the user cannot clearly hear the content in the target audio due to continued playback of the target audio, resulting in the loss of the content in the target audio, and improving the playback experience of the target audio; at the same time, it is not necessary for the user to manually pause the playback of the target audio when the noise in the environment has a greater impact, and it can also avoid the situation where part of the audio content in the target audio is missed due to untimely manual suspension of the playback of the target audio in a noisy environment and the need to manually return to the playback again. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 A schematic diagram of an application scenario applicable to the embodiments of the present application is shown;

[0020] Figure 2 A flowchart of an audio playback control method proposed in an embodiment of the present application is shown;

[0021] Figure 3 Shows Figure 2 A flowchart of the steps after step S140 in the corresponding embodiment in one embodiment is shown;

[0022] Figure 4 A schematic diagram of an audio playback control process in the embodiments of the present application is shown;

[0023] Figure 5 A block diagram of a noise detection module in the embodiments of the present application is shown;

[0024] Figure 6 A block diagram of a collection sub-module in the embodiments of the present application is shown;

[0025] Figure 7 A block diagram of an echo cancellation sub-module in the embodiments of the present application is shown;

[0026] Figure 8 The block diagram of an audio routing sub-module in an embodiment of the present application is shown;

[0027] Figure 9 The block diagram of a noise calculation sub-module in an embodiment of the present application is shown;

[0028] Figure 10 The block diagram of an audio playback control device proposed in an embodiment of the present application is shown;

[0029] Figure 11 The structural block diagram of an electronic device for executing the audio playback control method according to an embodiment of the present application is shown. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. According to the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0031] In the following description, the terms "first / second" involved are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first / second" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0033] It should be noted that: "a plurality" mentioned in this article refers to two or more. " / and" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. These three situations. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0034] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other relevant parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor or multiple processors or memories can be used to implement one or more modules or units. In addition, each module or unit can be a part of the whole module or unit that contains the function of the module or unit.

[0035] The present application discloses an audio playback control method, device, electronic device, and storage medium, which relate to cloud technology and the like.

[0036] Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that combines a large number of different types of storage devices (storage devices are also called storage nodes) in the network through functions such as cluster applications, grid technology, and distributed file systems, and works together through application software or application interfaces to provide data storage and service access functions externally.

[0037] Currently, the storage method of the storage system is as follows: Create a logical volume. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may be composed of a disk of a certain storage device or several storage devices. The client stores data on a certain logical volume, that is, stores the data on the file system. The file system divides the data into many parts, and each part is an object. The object not only contains data but also contains additional information such as a data identifier (ID, ID entity). The file system writes each object into the physical storage space of the logical volume respectively, and the file system will record the storage location information of each object. Thus, when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object.

[0038] The process of the storage system allocating physical storage space for a logical volume is specifically as follows: According to the capacity estimation of the objects stored in the logical volume (this estimation often has a large margin relative to the capacity of the objects to be actually stored) and the group of a redundant array of independent disks (RAID, Redundant Array of Independent Disk), the physical storage space is pre-divided into stripes, and a logical volume can be understood as a stripe, thereby allocating physical storage space for the logical volume.

[0039] Such as Figure 1As shown in the figure, the application scenarios applicable to the embodiments of the present application include a terminal 20 and a server 10, and the terminal 20 and the server 10 are communicatively connected through a wired network or a wireless network. The terminal 20 may include, but is not limited to, a mobile phone, a computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and other terminal devices that can execute an audio playback control method, or other applications that can call the audio playback control method (such as an instant messaging application, a shopping application, a search application, a game application, a forum application, a map traffic application, etc.).

[0040] The server 10 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 10 can be used to provide services for the applications running on the terminal 20.

[0041] Among them, the terminal 20 may send an audio acquisition request to the server 10. Based on the audio acquisition request, the server 10 returns the target audio corresponding to the audio acquisition request. The terminal 20 may receive the target audio and play the target audio. During the process of playing the target audio by the playback device, the terminal 20 collects the first collected environmental sound signal in the environment where the playback device is located; then, the terminal 20 determines the influence degree of the environmental noise on the audio playback according to the playback device information of the playback device; and determines the noise energy value according to the first collected environmental sound signal; if the noise energy value is higher than the energy threshold corresponding to the influence degree, the terminal pauses playing the target audio. In other embodiments, the terminal 20 may also perform audio playback control according to the method provided in the present application during the process of playing the audio or audio-visual stored by itself.

[0042] In the present application, the server 10 may send various audios to the distributed cloud storage system. The terminal 10 sends an audio acquisition request to the server 10, and the server 10 controls the distributed cloud storage system to return the target audio corresponding to the audio acquisition request to the terminal 20 according to the audio acquisition request.

[0043] Please refer to Figure 2 , Figure 2 shows a flowchart of an audio playback control method proposed in an embodiment of the present application. This method can be applied to an electronic device, and the electronic device can be Figure 1 the terminal 20 in, and this method includes:

[0044] S110. During the process of playing the target audio by the playback device, collect the first collected environmental sound signal in the environment where the playback device is located.

[0045] Among them, the electronic device can be an electronic device with an audio playback function, and the electronic device can have a component for playing audio (such as a speaker). The playback device can refer to the device that plays the target audio. When the target audio is played through the speaker of the electronic device itself, the playback device is the speaker in the electronic device. If the target audio is played through an external playback device (such as a stereo, headphones, speakers, etc.) connected to the electronic device, the external playback device is the playback device. For example, user a connects the mobile phone to a Bluetooth headset and listens to the music played by the mobile phone through the Bluetooth headset. At this time, the playback device is the Bluetooth headset.

[0046] The target audio played by the electronic device can be the audio stored in the electronic device or the audio sent by the audio server to the electronic device in real time. The target audio can be pure audio (such as music audio, blog audio, reading audio, etc.), or the audio in an audio-visual, such as film and television audio-visual, live audio-visual, conference audio-visual, recorded audio-visual, etc., which is not specifically limited here.

[0047] The environment where the playback device is located refers to the physical environment where the playback device itself is located, and the environment where the playback device is located may be different from the environment where the electronic device is located. For example, when the playback device is the speaker in the electronic device, the environment where the playback device is located is the environment where the electronic device is located. Another example is that when the playback device is a stereo connected to the electronic device, the environment where the stereo is located is the environment where the playback device is located.

[0048] An audio acquisition device can be installed or placed on the playback device. During the process of the playback device playing the target audio, the electronic device controls the audio acquisition device to acquire the sound signal in the environment where the playback device is located as the first acquired environmental sound signal. For example, when the playback device is the electronic device itself, the electronic device controls the built-in microphone of itself to acquire the first acquired environmental sound signal. Another example is that when the playback device is a stereo, a microphone can be installed or placed on the stereo, and then the electronic device is connected to the microphone and controls the microphone to acquire the first acquired environmental sound signal. After the audio acquisition device acquires the first acquired environmental sound signal, the electronic device directly obtains the first acquired environmental sound signal acquired by the audio acquisition device.

[0049] It is worth mentioning that during the process of the playback device playing the target audio, the environmental sound signal in the environment where the playback device is located is acquired in real time. Each acquired environmental sound signal is used as a first acquired environmental sound signal. Each first acquired environmental sound signal can be a sound signal with a fixed duration, and the duration of each first acquired environmental sound signal can be used as the environmental sound signal of an audio frame. For example, the first acquired environmental sound signal of 20s can form an audio frame, and the first acquired environmental sound signal acquired is also a sound signal of 20ms.

[0050] S120. Determine the influence degree of environmental noise on audio playback according to the playback device information of the playback device.

[0051] The playback device information of the playback device may include information indicating the playback device attributes of the playback device itself and information on the playback state when the playback device plays the target audio. Among them, the playback device attributes may include the playback device type of the playback device, whether the playback device enables a noise reduction function, etc., and the playback state may include the playback volume of the playback device. For example, the playback device information may include the playback device type of the playback device, whether the playback device enables a noise reduction function, and the playback volume of the playback device, etc.

[0052] The higher the influence degree, the greater the influence of the environmental noise in the environment where the playback device is located on the audio playback of the playback device, and the lower the influence degree, the smaller the influence of the environmental noise in the environment where the playback device is located on the audio playback of the playback device. Due to the different playback device attributes and playback state information of the playback device, the influence degree of environmental noise on the audio playback of the playback device is different; for example, in the case of an external speaker and a headphone for audio playback, the influence degree of environmental noise on audio playback is different; for another example, in the case of a high playback volume and a low playback volume, the influence degree of environmental noise on audio playback is also different; therefore, in this embodiment, based on the playback device information, the influence degree of environmental noise on audio playback is determined.

[0053] As an implementation manner, if the playback device information indicates that the playback device is of an external playback type, determine that the influence degree is the first influence degree; if the playback device information indicates that the playback device is of a headphone type and the playback device does not enable a noise reduction function, determine that the influence degree is the second influence degree, and the first influence degree is higher than the second influence degree; if the playback device information indicates that the playback device is of a headphone type and the playback device enables a noise reduction function, determine the influence degree according to the playback volume of the playback device in the playback device information.

[0054] The external playback type refers to the type to which the playback device that does not directly contact the human ear belongs. The playback devices of the external playback type may include speakers in speakers, sound boxes, electronic devices, etc. The headphone type may include various playback devices belonging to headphones, such as in-ear headphones, bone conduction headphones, and over-ear headphones, or wired headphones and wireless headphones classified according to the connection method.

[0055] When the playback device is of the external playback type, the playback volume can be ignored, but the influence of environmental noise cannot be ignored, and it is determined that the influence degree is the largest, that is, the influence degree is the first influence degree; in other words, in the case where the playback device is of the external playback type, the influence degree of environmental noise on audio playback caused by the playback volume of the playback device can be ignored, and environmental noise is mainly considered.

[0056] When the playback device is of the headphone type, it is impossible to directly evaluate the impact of ambient noise on audio playback. It is necessary to assist in the judgment by whether the noise reduction function is turned on and the playback volume of the playback device. If the playback device is of the headphone type and the noise reduction function is not turned on (the playback device not turning on the noise reduction function can refer to headphones without the noise reduction function or headphones with the noise reduction function but not turned on), it is determined that the degree of influence is lower than that of the playback device of the external speaker type, that is, the degree of influence is determined to be the second degree of influence; if the playback device is of the headphone type and the noise reduction function is turned on, it is necessary to combine the playback volume of the playback device to determine the degree of influence.

[0057] Optionally, determining the degree of influence according to the playback volume of the playback device in the playback device information includes: if the playback volume of the playback device is higher than the volume threshold, determining the degree of influence to be the third degree of influence, and the second degree of influence is higher than the third degree of influence; if the playback volume of the playback device is not higher than the volume threshold, determining the degree of influence to be the fourth degree of influence, and the third degree of influence is higher than the fourth degree of influence. Among them, the volume threshold can be a value set based on requirements, and this application does not make a limitation.

[0058] The playback volume of the headphones with the noise reduction function turned on is usually not very high. If the playback volume is high, the playback sound will be extremely loud, making the user's ears uncomfortable. Therefore, when the playback volume is not higher than the volume threshold, the playback device is less affected by ambient noise, and the degree of influence is determined to be the lowest - the fourth degree of influence. When the playback volume is higher than the volume threshold, it is considered that the influence of ambient noise on the playback device cannot be ignored and is higher than the lowest degree of influence. At this time, the degree of influence is determined to be the third degree of influence.

[0059] Based on the foregoing, in this application, the degree of influence of the playback device of the external speaker type is the largest, the degree of influence of the playback device of the headphone type without the noise reduction function turned on is the second largest, the degree of influence of the headphone playback type with the noise reduction function turned on and a relatively high playback volume (higher than the volume threshold) is the third largest, and the degree of influence of the headphone playback type with the noise reduction function turned on and a relatively low playback volume (not higher than the volume threshold) is the smallest.

[0060] S130. Determine the noise energy value according to the first collected ambient sound signal.

[0061] After collecting the first collected ambient sound signal, it is also necessary to perform noise energy analysis on the first collected ambient sound signal to determine the noise energy value corresponding to the first collected ambient sound signal. The higher the noise energy value, the stronger the ambient noise in the environment where the playback device is located, and the lower the noise energy value, the weaker the ambient noise in the environment where the playback device is located.

[0062] In one embodiment, S130 may include: if the playback device information indicates that the playback device is of the external speaker type, obtaining the original playback sound signal of the target audio; subtracting the original playback sound signal from the first collected ambient sound signal to obtain the actual ambient sound signal in the environment where the playback device is located; determining the noise energy value according to the actual ambient sound signal. The original playback sound signal refers to the sound signal output by the electronic device when playing the target audio.

[0063] The playback device information indicates that the playback device is of the external speaker type. The first collected ambient sound signal B consists of two parts: the actual ambient sound signal B1 in the environment where the playback device is located, and the playback audio signal B3 collected during the process of playing the target audio. The playback audio signal B2 can be approximated as the original playback sound signal B2 of the target audio. Therefore, in order to remove the playback audio signal B3 in the first collected ambient sound signal B, it is necessary to subtract the original playback sound signal B2 from the first collected ambient sound signal B to obtain the actual ambient sound signal B1 in the environment where the playback device is located. After obtaining the actual ambient sound signal B1, the noise energy value can be determined according to the actual ambient sound signal B1.

[0064] In some embodiments, the electronic device can obtain the audio file of the target audio, and then decode the audio file to obtain the playable audio data, and obtain the sound signal that drives the playback device when the playback device plays the audio data as the original playback sound signal.

[0065] In other embodiments, the target audio can be played through an audio application program, and the sound signal output by the audio playback program can be directly obtained as the original playback sound signal.

[0066] In this embodiment, by subtracting the first collected ambient sound signal from the original playback sound signal, the actual ambient sound signal is obtained. The actual ambient sound signal removes the influence of the original playback sound, making the determined noise value more accurate, and further enabling the noise energy value to more accurately indicate the strength of the ambient noise in the playback device.

[0067] In yet another embodiment, S130 may further include: if the playback device information indicates that the playback device is of the headphone type, using the first collected ambient sound signal as the actual ambient sound signal in the environment where the playback device is located; determining the noise energy value according to the actual ambient sound signal.

[0068] The playback device information indicates that the playback device is of the headphone type. The first collected ambient sound signal no longer includes the original playback sound signal. Therefore, the first collected ambient sound signal can be directly obtained as the actual ambient sound signal in the environment where the playback device is located.

[0069] Optionally, determining a noise energy value according to an actual environmental sound signal includes: filtering out non-environmental noise signals in the actual environmental sound signal to obtain an environmental noise signal; and determining the audio energy value of the environmental noise signal as the noise energy value. Among them, the non-environmental noise signal refers to a sound signal with a sound frequency within a non-noise frequency range, and the non-noise frequency range may refer to a frequency range less than 20 hz and greater than 20000 hz.

[0070] Non-environmental noise signals within the non-noise frequency range in the actual environmental sound signal can be filtered out to obtain a filtered sound signal as the environmental noise signal, and then the audio energy value of the environmental noise signal is determined as the noise energy value. Among them, the environmental noise signal can be converted into a 16-bit (bit) value, with one value corresponding to 1 bit, and then the maximum value among the 16 values involved in the 16-bit value is obtained as the audio energy value of the environmental noise signal. That is, after the environmental noise signal is converted into a 16-bit value, the maximum value among the 16 values involved in the 16-bit value is the noise energy value.

[0071] In this embodiment, filtering out non-environmental noise signals avoids the inaccurate situation of the noise energy value determined according to the signal including non-environmental noise signals, improves the accuracy of the determined noise energy value, and enables the noise energy value to more accurately reflect the intensity of the environmental noise in the environment where the playback device is located.

[0072] S140. If the noise energy value is higher than the energy threshold corresponding to the influence degree, pause playing the target audio.

[0073] Among them, different influence degrees correspond to different energy thresholds, and the energy threshold is negatively correlated with the influence degree. As can be seen from the foregoing, the first influence degree is higher than the second influence degree, the second influence degree is higher than the third influence degree, and the third influence degree is higher than the fourth influence degree. At this time, the energy threshold corresponding to the determined first influence degree is lower than the energy threshold corresponding to the second influence degree, the energy threshold corresponding to the second influence degree is lower than the energy threshold corresponding to the third influence degree, and the energy threshold corresponding to the third influence degree is lower than the energy threshold corresponding to the fourth influence degree.

[0074] When the noise energy value is higher than the energy threshold corresponding to the influence degree, it is determined that the influence of the environmental noise in the environment where the playback device is located is relatively large, and the user may have difficulty hearing the content in the target audio clearly. At this time, pause playing the target audio. When the noise energy value is not higher than the energy threshold corresponding to the influence degree, it is determined that the influence of the environmental noise in the environment where the playback device is located allows the user to hear the content in the target audio clearly. At this time, the target audio can be continuously played.

[0075] The greater the influence degree, the higher the influence of the ambient noise in the environment where the playback device is located on the audio playback, and the greater the influence of the ambient noise on the user's listening to the target audio. It is possible that noise with relatively small energy will also affect the user's listening experience. At this time, a relatively low energy threshold can be set. Similarly, the smaller the influence degree, the lower the influence of the ambient noise in the environment where the playback device is located on the audio playback, and the smaller the influence of the ambient noise on the user's listening to the target audio. It is possible that noise with relatively large energy will not affect the user's listening experience. At this time, a relatively high energy threshold can be set. Therefore, the energy threshold can be negatively correlated with the influence degree, so as to ensure the user's listening experience in different noise environments.

[0076] The method of this embodiment can be used in a pure audio scenario, that is, the target audio played by the electronic device is a pure audio, and the target audio is not the audio in the video played by the electronic device; it can also be applied to the playback control of the audio in the audio-video, which is not specifically limited here.

[0077] In this embodiment, first, during the process of the playback device playing the target audio, the first collected ambient sound signal in the environment where the playback device is located is collected; then, according to the playback device information of the playback device, the influence degree of the ambient noise on the audio playback is determined, and according to the first collected ambient sound signal, the noise energy value is determined; if the noise energy value is higher than the energy threshold corresponding to the influence degree, it is determined that the noise in the environment has a greater influence and the user cannot clearly hear the content in the target audio. At this time, the playback of the target audio is paused, so as to avoid the situation that the user cannot clearly hear the content in the target audio due to the continuous playback of the target audio, resulting in the loss of the content in the target audio, and improving the playback experience of the target audio; at the same time, it is not necessary for the user to manually pause the playback of the target audio when the noise in the environment has a greater influence, and it can also avoid the situation that due to the untimely manual pause of the playback of the target audio in the case of relatively large ambient noise, some audio content in the target audio is missed and needs to be manually returned to play again.

[0078] In addition, in this embodiment, for different playback devices, the strategies for determining the influence degree are different, so as to achieve targeted control when playing the target audio through different playback devices, and improve the accuracy of the playback control of the target audio.

[0079] In one embodiment, the first collected ambient sound signal is collected at a target time point. As Figure 3 shown, after S140, the method further includes:

[0080] S210. Cache the paused playback progress position of the target audio.

[0081] When the noise energy value is higher than the energy threshold corresponding to the influence degree, when pausing the playback of the target audio, the position where the playback progress of the target audio is located is the paused playback progress position of the target audio. For example, if the target audio is a song c with a duration of 4 minutes and 20 seconds, when it is detected that the noise energy value is higher than the energy threshold corresponding to the influence degree, the playback of song c is paused, and song c is paused at the 2 minute and 15 second mark, and the 2 minute and 15 second mark is the paused playback progress position.

[0082] After obtaining the paused playback progress position of the target audio, cache the paused playback progress position of the target audio.

[0083] S220. Obtain a second collected ambient sound signal in the environment where the playback device is located after the target time point.

[0084] The target time point refers to the moment when the first collected ambient sound signal is collected. After the target time point (that is, when pausing the playback of the target audio), continue to collect the ambient sound signal in the environment where the playback device is located as the second collected ambient sound signal.

[0085] S230. If the noise energy value determined based on the second collected ambient sound signal is not higher than the energy threshold corresponding to the influence degree, continue to play the target audio based on the paused playback progress position.

[0086] Regarding the second collected ambient sound signal, the noise energy value can be determined according to the steps of the aforementioned S130, which will not be elaborated here.

[0087] If the noise energy value determined based on the second collected ambient sound signal is not higher than the energy threshold corresponding to the influence degree, it is determined that the noise influence in the environment is small, and the user can hear the content in the target audio clearly, and the target audio can be continued to be played. At this time, the target audio can be continued to be played based on the paused playback progress position.

[0088] It should be noted that as described above, the duration of the ambient sound signal collected each time is fixed. After pausing the playback of the target audio, an ambient sound signal with a fixed duration can be collected as the second collected ambient sound signal. If the noise energy value determined based on the second collected ambient sound signal of this collection is higher than the energy threshold corresponding to the influence degree, continue to maintain the state of pausing the playback of the target audio, and continue to collect the ambient sound signal with a fixed duration as the new second collected ambient sound signal. This cycle continues until the noise energy value determined based on the second collected ambient sound signal is not higher than the energy threshold corresponding to the influence degree, and then continue to play the target audio based on the paused playback progress position.

[0089] In some embodiments, the target audio can be continued to be played from the paused playback progress position.

[0090] In some other embodiments, a progress position that is before the paused playback progress position and has a duration equal to a preset duration from the paused playback progress position may also be determined as the target progress position; and the target audio is continued to be played from the target progress position of the target audio. The preset duration may be a value set according to requirements. For example, the preset duration is 10 s.

[0091] That is to say, after the target audio is paused, if the target audio is continued to be played, it is played from the progress position that is the preset duration back from the paused playback progress position.

[0092] It is worth mentioning that if the duration between the start time of the target audio and the paused playback progress position of the target audio is less than the preset duration and there is no progress position that is before the paused playback progress position and has a duration equal to the preset duration from the paused playback progress position, the determined target progress position is the start time of the target audio; if the duration between the start time of the target audio and the paused playback progress position of the target audio is not less than the preset duration and there is a progress position that is before the paused playback progress position and has a duration equal to the preset duration from the paused playback progress position, the progress position that is before the paused playback progress position and has a duration equal to the preset duration from the paused playback progress position is determined as the target progress position.

[0093] In some scenarios, if the target audio is a live audio from a live broadcast (such as a conference live broadcast, etc.), after the target audio is paused, the live audio file during the period from pausing the live audio to resuming the live audio next time may be cached. In this way, it is convenient for the user to confirm the live audio content during the pause based on the cached live audio file after the live broadcast ends, and avoid missing some important content caused by pausing the live broadcast.

[0094] In some other scenarios, if the target audio is a live audio from a live broadcast, regardless of whether the noise energy value is detected to be higher than the energy threshold corresponding to the influence degree, the target audio is not paused, so as to ensure the continuity of the conference.

[0095] In some embodiments, the process of audio playback control is as Figure 4As shown, the target audio can be obtained from the local or audio server of the electronic device through the audio resource module of the electronic device, and then decoded by the decoding module to obtain the decoded data in PCM (Pulse Code Modulation, which is the audio information after digital conversion and the most primitive audio format. Essentially, it is a series of numerical values after digital conversion of audio signals) format. Then, the decoded data is processed by the sound effect processing module (where the sound effect processing can include enhancing bass, clarifying vocals, and replacing sound fields, etc.) to obtain the playback data, and then the playback data is cached in the playback cache so that the electronic device can obtain the playback data from the playback cache to play the target audio.

[0096] The playback device obtains the playback data of the target audio and plays the target audio through the playback data. During the playback of the target audio, the noise detection module is used to detect the noise energy value and influence degree, and the playback and pause of the target audio are controlled based on the detected noise energy value and influence degree.

[0097] Among them, the structure of the noise detection module is as Figure 5 shown. The noise detection module can include an acquisition sub-module, an acoustic echo cancellation (Acoustic Echo Cancellation, abbreviated as AEC, which can filter the sound played by the current application from the originally acquired sound to obtain the sound situation of the current environment) sub-module, a noise calculation sub-module, and an audio routing sub-module.

[0098] Among them, as Figure 6 shown, the acquisition sub-module can include an acquisition device unit, an acquisition gain unit, and an acquisition execution unit. Among them, the acquisition device unit refers to an acquisition device that can acquire environmental sound signals (such as the first acquired environmental sound signal and the second acquired environmental sound signal in the above text). The microphone in the electronic device can be preferentially selected to obtain the most real sound environment; the acquisition gain unit will adjust the acquired environmental sound signals. In order to obtain the most real environmental sound signals, the acquisition gain unit needs to be turned off to avoid misjudgment of noise; the acquisition control unit is used to control the acquisition device unit to acquire environmental sound signals and send the acquired environmental sound signals to the acoustic echo cancellation sub-module.

[0099] As Figure 7 shown, the acoustic echo cancellation sub-module obtains the acquired environmental sound signals and the original playback sound signals, cancels the echoes of the environmental sound signals and the original playback sound signals to obtain the actual environmental sound signals. Then, the acoustic echo cancellation sub-module sends the actual environmental sound signals to the audio routing sub-module.

[0100] As Figure 8As shown in the figure, the audio routing sub-module may include a routing device detection unit and a playback volume detection unit. The routing device detection unit is used to detect the playback device type of the playback device and whether the noise reduction function is enabled; the playback volume detection unit is used to detect the playback volume of the playback device.

[0101] As Figure 9 shown in the figure, the noise calculation sub-module is used to obtain the data detected by the audio routing sub-module and the actual environmental sound signal sent by the echo cancellation sub-module, perform non-environmental noise filtering on the actual environmental sound signal to obtain an environmental noise signal, and then determine the noise energy value according to the environmental noise signal. At the same time, the noise calculation sub-module can also determine the influence degree according to the playback device type of the playback device, whether the playback device enables the noise reduction function, and the playback volume of the playback device.

[0102] Finally, the noise calculation sub-module controls the target audio to pause or continue playing according to the influence degree and the noise energy value. When the noise energy value is higher than the energy threshold corresponding to the influence degree, it is determined that there is an influence, and the target audio is paused. When the noise energy value is not higher than the energy threshold corresponding to the influence degree, it is determined that there is no influence, and the target audio continues to be played. Among them, when continuing to play the target audio, it is played at the target progress position, and the target progress position is the progress position that is before the pause playback progress position and the duration between it and the pause playback progress position is equal to the preset duration.

[0103] In this embodiment, after pausing the playback of the target audio, the second collected environmental sound signal is continuously collected, and when the noise energy value determined according to the second collected environmental sound signal is not higher than the energy threshold corresponding to the influence degree, the target audio is automatically continued to be played based on the pause playback progress position, so that the user can continue to listen to the target audio without the user's manual operation to continue playing the target audio, thereby improving the user experience and playback efficiency.

[0104] In addition, when continuing to play the target audio, it is played at the target progress position. The target progress position refers to the progress position that is before the pause playback progress position and the duration between it and the pause playback progress position is equal to the preset duration, avoiding the situation of poor listening experience caused by the discontinuous information at the pause playback progress position when directly continuing to play the target audio from the pause playback progress position, improving the playback continuity of the target audio, and thus improving the user experience.

[0105] Please refer to Figure 10 , Figure 10 which shows a block diagram of an audio playback control device proposed in an embodiment of the present application. The device 900 includes:

[0106] The acquisition module 910 is configured to acquire a first acquired ambient sound signal in the environment where the playback device is located during the playback of the target audio by the playback device;

[0107] The influence degree determination module 920 is configured to determine the influence degree of environmental noise on audio playback according to the playback device information of the playback device;

[0108] The energy value determination module 930 is configured to determine a noise energy value according to the first acquired ambient sound signal;

[0109] The pause module 940 is configured to pause the playback of the target audio if the noise energy value is higher than the energy threshold corresponding to the influence degree, and the energy threshold is negatively correlated with the influence degree.

[0110] Optionally, the influence degree determination module 920 is further configured to determine the influence degree as the first influence degree if the playback device information indicates that the playback device is of the external speaker type; determine the influence degree as the second influence degree if the playback device information indicates that the playback device is of the headphone type and the playback device does not turn on the noise reduction function, and the first influence degree is higher than the second influence degree; determine the influence degree according to the playback volume of the playback device in the playback device information if the playback device information indicates that the playback device is of the headphone type and the playback device turns on the noise reduction function.

[0111] Optionally, the influence degree determination module 920 is further configured to determine the influence degree as the third influence degree if the playback volume of the playback device is higher than the volume threshold, and the second influence degree is higher than the third influence degree; determine the influence degree as the fourth influence degree if the playback volume of the playback device is not higher than the volume threshold, and the third influence degree is higher than the fourth influence degree.

[0112] Optionally, the first acquired ambient sound signal is acquired at a target time point; the device further includes a playback module, configured to cache the pause playback progress position of the target audio; acquire a second acquired ambient sound signal in the environment where the playback device is located after the target time point; and continue to play the target audio based on the pause playback progress position if the noise energy value determined according to the second acquired ambient sound signal is not higher than the energy threshold corresponding to the influence degree.

[0113] Optionally, the playback module is further configured to determine a target progress position whose duration between the progress position before the pause playback progress position and the pause playback progress position is equal to a preset duration; and continue to play the target audio from the target progress position of the target audio.

[0114] Optionally, the energy value determination module 930 is further configured to acquire the original playback sound signal of the target audio if the playback device information indicates that the playback device is of the external speaker type; subtract the original playback sound signal from the first acquired ambient sound signal to obtain the actual ambient sound signal in the environment where the playback device is located; and determine the noise energy value according to the actual ambient sound signal.

[0115] Optionally, the energy value determination module 930 is further configured to, if the playback device information indicates that the playback device is of the headphone type, use the first collected environmental sound signal as the actual environmental sound signal in the environment where the playback device is located; and determine the noise energy value according to the actual environmental sound signal.

[0116] Optionally, the energy value determination module 930 is further configured to filter out non-environmental noise signals in the actual environmental sound signal to obtain an environmental noise signal; and determine the audio energy value of the environmental noise signal as the noise energy value.

[0117] It should be noted that the device embodiments in this application correspond to the foregoing method embodiments. The specific principles in the device embodiments can be referred to the content in the foregoing method embodiments, and will not be elaborated here.

[0118] Figure 11 The block diagram of the electronic device for executing the audio playback control method according to the embodiment of the present application is shown. The electronic device may be Figure 1 the terminal in Figure 11 It should be noted that the computer system 1200 of the electronic device shown is only an example, and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0119] As Figure 11 shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage section 1208 into the random access memory (RAM) 1203, such as executing the method in the foregoing embodiments. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. The input / output (I / O) interface 1205 is also connected to the bus 1204.

[0120] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. The drive 1210 is also connected to the I / O interface 1205 as required. A removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 1210 as required so that a computer program read from it is installed into the storage section 1208 as required.

[0121] Specifically, according to an embodiment of the present application, the processes described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.

[0122] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0124] The units involved in the embodiments described in this application can be implemented in software or in hardware, and the described units can also be provided in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0125] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the methods in any of the above embodiments are implemented.

[0126] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the methods in any of the above embodiments.

[0127] It should be noted that although several modules or units of the devices for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0128] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable an electronic device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the methods according to the embodiments of the present application.

[0129] Other embodiments of the present application will be readily conceived by those skilled in the art after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common general knowledge or conventional technical means in the technical field not disclosed in the present application. It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. An audio playback control method, characterized in that The method includes: During the process of playing a target audio on a playback device, collecting a first ambient sound signal in the environment where the playback device is located; Determining the influence degree of ambient noise on audio playback according to the playback device information of the playback device; Determining a noise energy value according to the first ambient sound signal; If the noise energy value is higher than the energy threshold corresponding to the influence degree, pausing the playback of the target audio, where the energy threshold is negatively correlated with the influence degree.

2. The method according to claim 1, wherein The determining the influence degree of ambient noise on audio playback according to the playback device information of the playback device includes: If the playback device information indicates that the playback device is of an external speaker type, determining the influence degree as a first influence degree; If the playback device information indicates that the playback device is of a headphone type and the playback device does not turn on the noise reduction function, determining the influence degree as a second influence degree, where the first influence degree is higher than the second influence degree; If the playback device information indicates that the playback device is of a headphone type and the playback device turns on the noise reduction function, determining the influence degree according to the playback volume of the playback device in the playback device information.

3. The method according to claim 2, wherein The determining the influence degree according to the playback volume of the playback device in the playback device information includes: If the playback volume of the playback device is higher than a volume threshold, determining the influence degree as a third influence degree, where the second influence degree is higher than the third influence degree; If the playback volume of the playback device is not higher than the volume threshold, determining the influence degree as a fourth influence degree, where the third influence degree is higher than the fourth influence degree.

4. The method according to claim 1, wherein The first ambient sound signal is collected at a target time point; The method further includes: Caching the paused playback progress position of the target audio; Obtaining a second ambient sound signal in the environment where the playback device is located collected after the target time point; If the noise energy value determined according to the second ambient sound signal is not higher than the energy threshold corresponding to the influence degree, continuing to play the target audio based on the paused playback progress position.

5. The method according to claim 4, characterized in that, The continuing to play the target audio based on the paused playback progress position includes: Determining a target progress position whose duration between the progress position before the paused playback progress position and the paused playback progress position is equal to a preset duration; Continuing to play the target audio from the target progress position of the target audio.

6. The method according to claim 1, wherein The determining a noise energy value according to the first ambient sound signal includes: If the playback device information indicates that the playback device is of an external speaker type, obtaining the original playback sound signal of the target audio; Subtracting the first ambient sound signal from the original playback sound signal to obtain the actual ambient sound signal in the environment where the playback device is located; Determining a noise energy value according to the actual ambient sound signal.

7. The method according to claim 1, characterized in that, The determining a noise energy value according to the first ambient sound signal includes: If the playback device information indicates that the playback device is of a headphone type, using the first ambient sound signal as the actual ambient sound signal in the environment where the playback device is located; Determine a noise energy value according to the actual environmental sound signal.

8. The method according to claim 6 or 7, characterized in that, The determining of the noise energy value according to the actual environmental sound signal includes: Filter non-environmental noise signals in the actual environmental sound signal to obtain an environmental noise signal; Determine the audio energy value of the environmental noise signal as the noise energy value.

9. An audio playback control device, characterized in that, The device includes: An acquisition module, configured to acquire a first acquired environmental sound signal in the environment where the playback device is located during the playback of the target audio by the playback device; An influence degree determination module, configured to determine the influence degree of environmental noise on audio playback according to the playback device information of the playback device; An energy value determination module, configured to determine a noise energy value according to the first acquired environmental sound signal; A pause module, configured to pause the playback of the target audio if the noise energy value is higher than the energy threshold corresponding to the influence degree, and the energy threshold is negatively correlated with the influence degree.

10. An electronic device, characterized in that, including: A processor; A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method described in any one of claims 1-8 is implemented.

11. A computer-readable storage medium, characterized in that, On which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method described in any one of claims 1-8 is implemented.

12. A computer program product or a computer program, characterized in that, Including computer instructions, characterized in that when the computer instructions are executed by the processor, the method described in any one of claims 1-8 is implemented.

Images