Method, device, apparatus and medium for adjusting output audio according to ambient noise

Abstract

The embodiments of the present application disclose a method, apparatus, device and medium for adjusting output audio according to environmental noise. A specific implementation of the method includes: collecting environmental noise in the environment where the playback device is located; based on the collected environmental noise, detecting the noise frequency and noise volume of the environmental noise; generating audio enhancement information according to the noise frequency and noise volume; and superimposing the audio enhancement information on the original audio of the playback device to obtain updated audio. This embodiment can automatically adjust the output audio of the playback device according to the detected environmental noise, adjust the output audio to parameters suitable for the user, and largely eliminate the impact of environmental noise on the audio, thereby improving the user experience and the practicality of the playback device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, device and medium for adjusting output audio according to ambient noise. Background Art

[0002] At present, when smart speakers and other playback devices are initially turned on or restarted, they all use a fixed volume. Playback devices with a fixed volume are easily affected by environmental noise. For example, in a noisy environment, the startup volume is low and the user cannot hear the sound being played clearly. In this way, the playback device cannot be adjusted according to the user's usage environment, resulting in a low user experience. Summary of the invention

[0003] The embodiments of the present application provide a method and device for adjusting output audio according to ambient noise.

[0004] In a first aspect, an embodiment of the present application provides a method for adjusting output audio according to ambient noise, wherein the method comprises: collecting ambient noise in the environment in which a playback device is located; based on the collected ambient noise, detecting the noise frequency and noise volume of the ambient noise; generating audio enhancement information according to the noise frequency and noise volume; and superimposing the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0005] In some embodiments, the above-mentioned generating audio enhancement information according to noise frequency and noise volume includes: obtaining masked audio of original audio according to noise frequency and noise volume; generating audio enhancement information according to the volume of masked audio and the masking domain of ambient noise at the frequency of masked audio.

[0006] In some embodiments, the above-mentioned generating audio enhancement information based on the noise frequency and the noise volume includes: superimposing a preset volume threshold on the basis of the noise volume to obtain the expected volume of the original audio at the noise frequency; generating audio enhancement information based on the difference between the original volume and the expected volume of the original audio at the noise frequency.

[0007] In some embodiments, after the updated audio is obtained by superimposing the audio enhancement information on the original audio of the playback device, the method further includes: in response to detecting that the volume of the updated audio is greater than a preset threshold, stopping the playback of the updated audio.

[0008] In some embodiments, after the updated audio is obtained by superimposing the audio enhancement information on the original audio of the playback device, it also includes: determining the direction information and distance information between the playback device and the user based on the collected voice information; adjusting the volume of the updated audio based on the distance information, and directionally propagating the updated audio based on the direction information.

[0009] In some embodiments, the above-mentioned directionally propagating updated audio according to directional information includes: adjusting the playback device according to the directional information so that a super-directional speaker of the playback device faces the user; and playing the updated audio through the super-directional speaker.

[0010] In a second aspect, an embodiment of the present application provides a device for adjusting output audio according to ambient noise, the device comprising: a noise collection unit, configured to collect ambient noise in the environment in which a playback device is located; a noise detection unit, configured to detect the noise frequency and noise volume of the ambient noise based on the collected ambient noise; an audio enhancement unit, configured to generate audio enhancement information based on the noise frequency and noise volume; and an audio superposition unit, configured to superimpose the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0011] In a third aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0012] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0013] The method and device for adjusting the output audio according to the environmental noise provided in the embodiment of the present application first collect the environmental noise in the environment where the playback device is located; then, based on the collected environmental noise, detect the noise frequency and noise volume of the environmental noise; then, generate audio enhancement information according to the noise frequency and noise volume; finally, on the basis of the original audio of the playback device, superimpose the audio enhancement information to obtain updated audio. The present application can automatically adjust the output audio of the playback device according to the detected environmental noise, adjust the output audio to parameters suitable for the user, and largely eliminate the influence of environmental noise on the audio, thereby improving the user experience and the practicality of the playback device. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0015] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;

[0016] Figure 2 is a flow chart of an embodiment of a method for adjusting output audio according to ambient noise according to the present application;

[0017] Figure 3 is a schematic diagram of a masking curve of environmental noise in this application;

[0018] Figure 4 is a schematic diagram of an application scenario of the method for adjusting output audio according to environmental noise according to this embodiment;

[0019] Figure 5 is a flow chart of another embodiment of a method for adjusting output audio according to environmental noise according to the present application;

[0020] Figure 6 is a flow chart of another embodiment of a method for adjusting output audio according to environmental noise according to the present application;

[0021] Figure 7 is a structural diagram of an embodiment of a device for adjusting output audio according to ambient noise according to the present application;

[0022] Figure 8 It is a structural diagram of a computer system suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0023] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present disclosure, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] Figure 1 An exemplary architecture 100 is shown to which the method and apparatus of adjusting output audio according to environmental noise of the present application can be applied.

[0026] like Figure 1 As shown, the system architecture 100 may include playback devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the playback devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0027] The playback devices 101, 102, 103 may be hardware devices or software that support network connection for data interaction and data processing. When the playback devices 101, 102, 103 are hardware, they may be various electronic devices that support functions such as information interaction, network connection, audio and video playback, including but not limited to smart speakers, Bluetooth speakers, smart phones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the playback devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. It may be implemented as multiple software or software modules, for example, for providing distributed services, or it may be implemented as a single software or software module. No specific limitation is made here.

[0028] The server 105 may be a server that provides various services, such as providing data analysis and processing, data transmission and other functions to the playback devices 101, 102, 103. The server may store or process various received data and feed back the processing results to the playback device.

[0029] It should be noted that the method for adjusting the output audio according to the environmental noise provided in the embodiment of the present disclosure can be executed by the playback device 101, 102, 103, or by the server 105, or a part of it can be executed by the playback device 101, 102, 103, and the other part can be executed by the server 105. Accordingly, the device for adjusting the output audio according to the environmental noise can be set in the playback device 101, 102, 103, or in the server 105, or a part of it can be set in the playback device 101, 102, 103 and the other part can be set in the server 105. No specific limitation is made here.

[0030] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules for providing distributed services, or it can be implemented as a single software or software module. No specific limitation is made here.

[0031] It should be understood that Figure 1 The number of playback devices and servers in the embodiment is only for illustration. Any number of playback devices and servers may be provided according to implementation requirements.

[0032] Continue to refer Figure 2 , a process 200 of an embodiment of a method for adjusting output audio according to environmental noise is shown, comprising the following steps:

[0033] Step 201: collect the ambient noise in the environment where the playback device is located.

[0034] In this embodiment, the execution subject (for example Figure 1 The playback device in the system can collect ambient noise in the environment through a microphone array or noise collector provided inside the system.

[0035] Environmental noise refers to the sound generated in industrial production, construction, transportation and social life that interferes with the audio output of the playback device. For example, the sound of a car horn passing by the playback device, and the sound of people talking in the environment where the playback device is located.

[0036] In some optional implementations, the execution subject may collect the environmental noise in advance before the playback device plays the audio, and analyze and process the environmental noise to determine whether it is necessary to collect the environmental noise in the environment in real time while the playback device is playing the audio. For example, in the case where the noise frequency and noise volume of the environmental noise are constantly changing and unstable, the environmental noise in the environment may be collected in real time while the playback device is playing the audio; in the case where the noise frequency and noise volume of the environmental noise are relatively stable, the environmental noise in the environment may be collected only before the playback device plays the audio.

[0037] In this embodiment, the collected environmental noise can be stored by the playback device so that the environmental noise can be analyzed and processed by the playback device; the collected environmental noise can also be transmitted to an environmental noise processing device (for example, Figure 1 A server in the system) is used to analyze and process the environmental noise through an environmental noise processing device.

[0038] It should be noted that the audio played by the playback device in this embodiment may be pure audio data, such as voice data, music data, etc.; it may also be audio data in multimedia data.

[0039] Step 202: Detect the noise frequency and noise volume of the environmental noise based on the collected environmental noise.

[0040] In this embodiment, after collecting the environmental noise in the environment where the playback device is located, the execution subject may analyze the environmental noise to obtain the noise frequency and noise volume of the environmental noise.

[0041] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device has a noise analysis function, the execution subject of this step can be the playback device with the noise analysis function; otherwise, the execution subject of this step can be a server with the noise analysis function.

[0042] Step 203: Generate audio enhancement information according to the noise frequency and the noise volume.

[0043] In this embodiment, the execution entity may generate audio enhancement information according to the noise frequency and noise volume obtained in step 202 .

[0044] In this embodiment, the audio enhancement information is used to instruct the execution subject to enhance the output volume of the original audio, for example, it may be audio enhancement information instructing the execution subject to enhance the output volume of the original audio at a noise frequency.

[0045] In this embodiment, the execution subject may pre-set a volume adjustment coefficient for the original audio. For the original audio at the noise frequency, the volume of the original audio at the noise frequency is multiplied by the volume adjustment coefficient to obtain the expected volume of the original audio at the noise frequency; then, the audio enhancement information is generated according to the difference between the original volume and the expected volume of the original audio at the noise frequency. The volume adjustment coefficient is specifically set according to the actual situation and is not limited here.

[0046] For example, the execution subject detects that the noise frequency of the ambient noise is mainly between 35Hz and 40Hz, and the noise volume is 30dB. The volume of the original audio at a frequency of 35Hz-40Hz is 35dB, and the preset volume adjustment coefficient is 1.5. Then the expected volume of the original audio at this noise frequency is 35dB×1.5=52.5dB, and the audio enhancement information is 52.5dB-35dB=17.5dB, which is used to indicate that the volume of the original audio at a frequency of 35Hz-40Hz should be increased by 17.5dB. In this way, the volume of the original audio at a noise frequency of 35Hz-40Hz is 52.5dB, which is 22.5dB higher than the noise volume of 30dB, so that the original audio at the noise frequency can be clearly heard by the audience.

[0047] In some optional implementations of this embodiment, the execution entity may also superimpose a preset volume threshold on the basis of the noise volume to obtain the expected volume of the original audio at the noise frequency; and generate audio enhancement information based on the difference between the original volume and the expected volume of the original audio at the noise frequency.

[0048] For example, the execution subject detects that the noise frequency of the ambient noise is mainly 35Hz-40Hz, and its noise volume is 30dB. The volume of the original audio at the frequency of 35Hz-40Hz is 35dB, and the preset volume adjustment coefficient is 25dB. The expected volume of the original audio at this noise frequency is 30dB+25dB=55dB, and the audio enhancement information is 55dB-35dB=20dB, which is used to indicate that the volume of the original audio at the frequency of 35Hz-40Hz should be increased by 20dB. In this way, the volume of the original audio at the noise frequency of 35Hz-40Hz is 55dB, which is 25dB higher than the noise volume of 30dB, so that the original audio at the noise frequency can be heard clearly.

[0049] In some optional implementations of this embodiment, sound enhancement information may also be generated based on the masking effect of noise. The masking effect of noise refers to the phenomenon that the hearing threshold of a sound is increased due to the presence of one or more other sounds. The execution subject may obtain the masked audio of the original audio based on the noise frequency and the noise volume; and generate the audio enhancement information based on the volume of the masked audio and the masking domain of the ambient noise at the frequency of the masked audio.

[0050] Specifically, first, the execution subject can obtain the masking curve of the ambient noise according to the noise frequency and noise volume of the ambient noise. The horizontal axis of the masking curve is the frequency, which is used to represent the frequency range of the affected original audio; the vertical axis of the masking curve is the volume, which is used to represent the masking domain of the affected original audio.

[0051] refer to Figure 3 , showing the masking curves of environmental noises of 0.25KHz, 60dB, 1KHz, 58dB, 4KHz, and 56dB respectively. Among them, the dotted line is a curve representing the relationship between the frequency of the original audio and the hearing threshold. The hearing threshold refers to the minimum audible intensity, that is, the minimum sound intensity required for a person to distinguish the existence of a sound. Taking the environmental noise with a frequency of 0.25KHz and a volume of 60dB as an example, it mainly affects the original audio of 0.09KHz-1KHz. When the volume value of the original audio in the frequency range of 0.09KHz-1KHz is lower than the ordinate value of the corresponding masking curve, it will be masked by the environmental noise with a frequency of 0.25KHz and a volume of 60dB, and cannot be heard by the user.

[0052] Then, according to the masking curve of the ambient noise, the masked audio of the original audio is obtained. If the frequency of the original audio is within the influence range of the masking curve, and the volume of the original audio is lower than the masking domain of the masking curve corresponding to the frequency of the original audio, the original audio at this frequency will be masked. If the frequency of the original audio is within the influence range of the masking curve, the volume of the original audio is higher than the masking domain of the masking curve corresponding to the frequency of the original audio, but the difference between the volume of the original audio and the corresponding masking domain is less than a preset threshold, the original audio at this frequency is not masked, but is too affected by the ambient noise, which is reflected in the listener's feeling that the original audio at this frequency cannot be heard at all. In this implementation method, the above-mentioned completely masked audio and audio that is greatly affected by ambient noise can be used as masked audio.

[0053] Finally, audio enhancement information is generated according to the volume of the masked audio and the masking domain of the ambient noise at the frequency of the masked audio. For example, the audio enhancement information of the masked audio can be generated according to the difference between the minimum volume of the masked audio and the expected volume of the masked audio relative to the masking domain.

[0054] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device has the function of generating audio enhancement information, the execution subject of this step is the playback device with the function of generating audio enhancement information; otherwise, the execution subject of this step is the server with the function of generating audio enhancement information.

[0055] Step 204: based on the original audio of the playback device, the audio enhancement information is superimposed to obtain updated audio.

[0056] In this embodiment, the execution subject collects, transforms, filters, estimates, enhances, compresses, recognizes and other synthetic processes on the original audio and the determined audio enhancement information, thereby superimposing the audio enhancement information on the original audio to obtain updated audio.

[0057] In some optional implementations, a DSP (Digital Signal Processing) system may be provided in the execution subject, including the main components of a digital signal processor DSP, audio A / D (analog / digital) and D / A (digital / analog), RAM, ROM and peripheral processors. Through the DSP system, the execution subject can superimpose audio enhancement information on the original audio of the playback device to obtain updated audio.

[0058] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device has the audio enhancement information superposition function, the execution subject of this step can be the playback device with the audio enhancement information superposition function; otherwise, the execution subject of this step can be a server with the audio enhancement information superposition function.

[0059] In this embodiment, the execution entity automatically adjusts the output audio of the playback device according to the detected ambient noise, and adjusts the output audio to parameters suitable for the user, thereby eliminating the impact of ambient noise on the audio to a large extent, improving the user experience and the practicality of the playback device.

[0060] Figure 4An application scenario of the method for adjusting output audio according to environmental noise of the present embodiment is schematically shown. User 401 is listening to a radio program through playback device 402. During the playback process, playback device 402 collects the environmental noise in its environment, and transmits the collected environmental noise to server 403 through the network. Based on the collected environmental noise, server 403 detects that the noise frequency of the environmental noise is 270Hz-300Hz and the noise volume is 30dB. Then, server 403 generates audio enhancement information according to the noise frequency and noise volume of the environmental noise, and sends the audio enhancement information to playback device 402. For example, the audio enhancement information is used to characterize the volume of the audio in the frequency range of 270Hz-300Hz in the audio corresponding to the enhanced radio program. Finally, playback device 402 receives the audio enhancement information, superimposes the audio enhancement information on the original audio of the playback device to obtain updated audio, and plays the updated audio.

[0061] Continue to refer Figure 5 , shows a schematic process 500 of another embodiment of a method for adjusting output audio according to environmental noise according to the present application, comprising the following steps:

[0062] Step 501: collect the ambient noise in the environment where the playback device is located.

[0063] In this embodiment, step 501 is performed in a manner similar to step 201 and will not be described in detail herein.

[0064] Step 502: Detect the noise frequency and noise volume of the environmental noise based on the collected environmental noise.

[0065] In this embodiment, step 502 is performed in a manner similar to step 202 and will not be described in detail herein.

[0066] Step 503: Generate audio enhancement information according to the noise frequency and the noise volume.

[0067] In this embodiment, step 503 is performed in a manner similar to step 203 and will not be described in detail herein.

[0068] Step 504: based on the original audio of the playback device, the audio enhancement information is superimposed to obtain updated audio.

[0069] In this embodiment, step 504 is performed in a manner similar to step 204 and will not be described in detail herein.

[0070] Step 505: In response to detecting that the volume of the update audio is greater than a preset threshold, stop playing the update audio.

[0071] In this embodiment, the execution subject may detect the playing volume of the updating audio during the playing of the updating audio, and when it is detected that the volume of the updating audio is greater than a preset threshold, the playing of the updating audio is stopped.

[0072] Scientific research has found that continuous exposure to a noise environment of 85 dB can cause harm to the human body. In some optional implementations, the preset threshold can be set to 85 dB to protect the listener's hearing.

[0073] It should be noted that the preset threshold can be set specifically according to the actual environment and is not limited here.

[0074] It can be seen from the embodiment including steps 501-505 that Figure 2 Compared with the corresponding embodiments, process 500 of the method for adjusting output audio according to ambient noise in this embodiment specifically illustrates the adjustment limit on the original audio. In response to detecting that the volume of the updated audio is greater than a preset threshold, the execution body stops playing the updated audio to protect the listener's hearing.

[0075] Continue to refer Figure 6 , shows a schematic process 600 of another embodiment of a method for adjusting output audio according to environmental noise according to the present application, comprising the following steps:

[0076] Step 601: collect the ambient noise in the environment where the playback device is located.

[0077] In this embodiment, step 601 is performed in a manner similar to step 201 and will not be described in detail herein.

[0078] Step 602: Detect the noise frequency and noise volume of the environmental noise based on the collected environmental noise.

[0079] In this embodiment, step 602 is performed in a manner similar to step 202 and will not be described in detail herein.

[0080] Step 603: Generate audio enhancement information according to the noise frequency and the noise volume.

[0081] In this embodiment, step 603 is performed in a manner similar to step 203 and will not be described in detail herein.

[0082] Step 604: based on the original audio of the playback device, the audio enhancement information is superimposed to obtain updated audio.

[0083] In this embodiment, step 604 is performed in a manner similar to step 204 and will not be described in detail herein.

[0084] Step 605: Determine the direction information and distance information between the playback device and the user based on the collected voice information.

[0085] In this embodiment, the execution subject (for example Figure 1 The playback device in the video has a voice collection function and can determine the direction information and distance information between the playback device and the user based on the collected voice information. The voice information can be any voice information issued by the user, for example, it can be instruction information for instructing the playback device to perform a specific action.

[0086] Step 606, adjusting the volume of the updated audio according to the distance information, and directionally propagating the updated audio according to the direction information.

[0087] In this embodiment, after determining the direction information and distance information between the playback device and the user, the volume of the updated audio can be adjusted according to the distance value indicated by the distance information to meet the transmission distance requirement of the updated audio, and the updated audio can be directionally propagated according to the direction information.

[0088] In some optional implementations, the playback device rotation device is adjusted according to the direction information so that the super-directional speaker of the playback device disposed on the playback device rotation device faces the user; and the updated audio is played through the super-directional speaker. The sound wave energy emitted by the super-directional speaker is concentrated, and the sound is only propagated in a specified direction.

[0089] It can be seen from the embodiment including steps 601-606 that Figure 2 Compared with the corresponding embodiments, process 600 of the method for adjusting output audio according to ambient noise in this embodiment specifically illustrates adjusting and updating the volume of the audio according to the distance information between the playback device and the user, and directionally propagating the updated audio according to the direction information between the playback device and the user, so as to enhance the user experience.

[0090] Continue to refer Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for adjusting output audio according to environmental noise. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0091] like Figure 7 As shown, the device for adjusting output audio according to environmental noise includes: a noise collection unit 701, a noise detection unit 702, an audio enhancement unit 703 and an audio superposition unit 704.

[0092] The noise collection unit 701 is configured to collect the ambient noise in the environment where the playback device is located; the noise detection unit 702 is configured to detect the noise frequency and noise volume of the ambient noise based on the collected ambient noise; the audio enhancement unit 703 is configured to generate audio enhancement information according to the noise frequency and noise volume; the audio superposition unit 704 is configured to superimpose the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0093] In some embodiments, the audio enhancement unit 703 is further configured to: obtain the masked audio of the original audio according to the noise frequency and the noise volume; and generate audio enhancement information according to the volume of the masked audio and the masking domain of the ambient noise at the frequency of the masked audio.

[0094] In some embodiments, the audio enhancement unit 703 is further configured to: superimpose a preset volume threshold on the basis of the noise volume to obtain the expected volume of the original audio at the noise frequency; and generate audio enhancement information based on the difference between the original volume and the expected volume of the original audio at the noise frequency.

[0095] In some embodiments, the above-mentioned apparatus further includes: a stop playing unit (not shown in the figure), configured to stop playing the update audio in response to detecting that the volume of the update audio is greater than a preset threshold.

[0096] In some embodiments, the above-mentioned device also includes: an audio adjustment unit (not shown in the figure), which is configured to determine the direction information and distance information between the playback device and the user based on the collected voice information; adjust the volume of the updated audio according to the distance information, and directionally propagate the updated audio according to the direction information.

[0097] In some embodiments, the audio adjustment unit is further configured to: adjust the playback device rotation device according to the direction information so that the super-directional speaker of the playback device set on the playback device rotation device faces the user; and play the updated audio through the super-directional speaker.

[0098] Reference below Figure 8 , which shows a device suitable for implementing the embodiments of the present application (eg Figure 1 Schematic diagram of the structure of a computer system 800 of the devices 101, 102, 103, 105 shown. Figure 8 The device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0099] like Figure 8As shown, the computer system 800 includes a processor (e.g., CPU, central processing unit) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0100] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0101] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 809, and / or installed from a removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the method of the present application are executed.

[0102] It should be noted that the computer-readable medium of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0103] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the client computer, partially on the client computer, as a separate software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the client computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] According to one or more embodiments of the present disclosure, the present disclosure provides a method for adjusting output audio according to environmental noise, the method comprising: collecting environmental noise in the environment in which a playback device is located; based on the collected environmental noise, detecting the noise frequency and noise volume of the environmental noise; generating audio enhancement information according to the noise frequency and the noise volume; and superimposing the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0106] According to one or more embodiments of the present disclosure, in the method for adjusting output audio according to ambient noise provided by the present disclosure, audio enhancement information is generated according to noise frequency and noise volume, including: obtaining masked audio of original audio according to the noise frequency and noise volume; generating audio enhancement information according to the volume of the masked audio and the masking domain of the ambient noise at the frequency of the masked audio.

[0107] According to one or more embodiments of the present disclosure, in the method for adjusting output audio according to environmental noise provided by the present disclosure, audio enhancement information is generated according to the noise frequency and the noise volume, including: superimposing a preset volume threshold on the basis of the noise volume to obtain the expected volume of the original audio at the noise frequency; generating the audio enhancement information according to the difference between the original volume and the expected volume of the original audio at the noise frequency.

[0108] According to one or more embodiments of the present disclosure, in the method for adjusting output audio according to ambient noise provided by the present disclosure, after superimposing audio enhancement information on the original audio of the playback device to obtain updated audio, it also includes: in response to detecting that the volume of the updated audio is greater than a preset threshold, stopping playing the updated audio.

[0109] According to one or more embodiments of the present disclosure, in the method for adjusting output audio according to ambient noise provided by the present disclosure, after superimposing audio enhancement information on the original audio of the playback device to obtain updated audio, the method further includes: determining direction information and distance information between the playback device and the user based on the collected voice information; adjusting the volume of the updated audio based on the distance information, and directionally propagating the updated audio based on the direction information.

[0110] According to one or more embodiments of the present disclosure, in the method for adjusting output audio according to ambient noise provided by the present disclosure, directionally propagating updated audio according to directional information includes: adjusting a playback device according to the directional information so that a super-directional speaker of the playback device faces a user; and playing the updated audio through the super-directional speaker.

[0111] According to one or more embodiments of the present disclosure, the present disclosure provides a device for adjusting output audio according to environmental noise, including: a noise collection unit, configured to collect environmental noise in the environment where a playback device is located; a noise detection unit, configured to detect the noise frequency and noise volume of the environmental noise based on the collected environmental noise; an audio enhancement unit, configured to generate audio enhancement information based on the noise frequency and noise volume; and an audio superposition unit, configured to superimpose the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0112] According to one or more embodiments of the present disclosure, in the device for adjusting output audio according to ambient noise provided by the present disclosure, the audio enhancement unit is further configured to: obtain masked audio of the original audio according to the noise frequency and the noise volume; and generate audio enhancement information according to the volume of the masked audio and the masking domain of the ambient noise at the frequency of the masked audio.

[0113] According to one or more embodiments of the present disclosure, in the device for adjusting output audio according to ambient noise provided by the present disclosure, the audio enhancement unit is further configured to: superimpose a preset volume threshold on the basis of the noise volume to obtain the expected volume of the original audio at the noise frequency; and generate audio enhancement information according to the difference between the original volume and the expected volume of the original audio at the noise frequency.

[0114] According to one or more embodiments of the present disclosure, the device for adjusting output audio according to ambient noise provided by the present disclosure further includes: a stop playback unit, configured to stop playing the updated audio in response to detecting that the volume of the updated audio is greater than a preset threshold.

[0115] According to one or more embodiments of the present disclosure, the device for adjusting output audio according to ambient noise provided by the present disclosure also includes: an audio adjustment unit, configured to determine direction information and distance information between the playback device and the user based on the collected voice information; adjust the volume of the updated audio according to the distance information, and directionally propagate the updated audio according to the direction information.

[0116] According to one or more embodiments of the present disclosure, in the device for adjusting output audio according to ambient noise provided by the present disclosure, the audio adjustment unit is further configured to: adjust the playback device according to direction information so that the super-directional speaker of the playback device is facing the user; and play the updated audio through the super-directional speaker.

[0117] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising a noise collection unit, a noise detection unit, an audio enhancement unit, and an audio superposition unit. Among them, the names of these units do not constitute a limitation on the unit itself in certain cases, for example, the audio enhancement unit may also be described as a unit that "generates audio enhancement information according to noise frequency and noise volume".

[0118] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the computer device: collects the ambient noise in the environment where the playback device is located; based on the collected ambient noise, detects the noise frequency and noise volume of the ambient noise; generates audio enhancement information based on the noise frequency and noise volume; and superimposes the audio enhancement information on the original audio of the playback device to obtain updated audio.

[0119] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the present disclosure. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

Claims

1. A method for adjusting output audio according to ambient noise, the method comprising: Collect the ambient noise in the environment where the playback device is located; Based on the collected environmental noise, the noise frequency and noise volume of the environmental noise are detected; generating audio enhancement information according to the noise frequency and the noise volume; On the basis of the original audio of the playback device, the audio enhancement information is superimposed to obtain updated audio; The step of generating audio enhancement information according to the noise frequency and the noise volume includes: Obtaining a masking curve of the environmental noise according to the noise frequency and the noise volume; Acquire the masked audio of the original audio according to the masking curve of the environmental noise; The masked audio includes original audio whose frequency is within the influence range of the masking curve and whose volume is lower than the masking domain of the frequency corresponding to the original audio by the masking curve, and original audio whose frequency is within the influence range of the masking curve and whose volume is higher than the masking domain of the frequency corresponding to the original audio by the masking curve, and whose volume difference with the masking domain is less than a preset threshold; The audio enhancement information of the masked audio is generated according to the difference between the minimum volume of the masked audio and the expected volume of the masked audio relative to the masking domain.

2. The method according to claim 1, wherein: After the updated audio is obtained by superimposing the audio enhancement information on the original audio of the playback device, the method further includes: In response to detecting that the volume of the updated audio is greater than a preset threshold, stopping playing the updated audio.

3. The method according to claim 1, wherein: After the updated audio is obtained by superimposing the audio enhancement information on the original audio of the playback device, the method further includes: Determining direction information and distance information between the playback device and the user based on the collected voice information; The volume of the updated audio is adjusted according to the distance information, and the updated audio is directionally propagated according to the direction information.

4. The method according to claim 3, wherein: The directionally propagating the updated audio according to the direction information comprises: According to the direction information, adjusting the playback device so that the super-directional loudspeaker of the playback device faces the user; The update audio is played through the super-directional speaker.

5. A device for adjusting output audio according to ambient noise, the device comprising: A noise collection unit is configured to collect ambient noise in an environment where the playback device is located; A noise detection unit is configured to detect a noise frequency and a noise volume of the ambient noise based on the collected ambient noise; an audio enhancement unit configured to generate audio enhancement information according to the noise frequency and the noise volume; An audio superimposing unit, configured to superimpose the audio enhancement information on the original audio of the playback device to obtain updated audio; The audio enhancement unit is specifically used to obtain a masking curve of the ambient noise according to the noise frequency and the noise volume; obtain the masked audio of the original audio according to the masking curve of the ambient noise; the masked audio includes the original audio whose frequency is within the influence range of the masking curve and whose volume is lower than the masking domain of the masking curve corresponding to the frequency of the original audio, and the original audio whose frequency is within the influence range of the masking curve and whose volume is higher than the masking domain of the masking curve corresponding to the frequency of the original audio, and whose volume difference with the masking domain is less than a preset threshold; generate audio enhancement information of the masked audio according to the difference between the minimum volume of the masked audio and the expected volume of the masked audio relative to the masking domain.

6. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

Images