Audio signal processing method and device, computer program product, watch and earphone

Abstract

The invention belongs to the technical field of audio processing, and particularly relates to an audio signal processing method and device, a computer program product, a watch and an earphone. The method comprises the following steps: controlling the watch to collect a first audio signal under the condition that a voice interaction function is started; receiving a second audio signal collected by the earphone; the second audio signal and the first audio signal are collected synchronously; and performing noise reduction processing on the second audio signal by taking the first audio signal as an environment reference audio signal to obtain a third audio signal. According to the embodiment of the invention, the earphone and the watch can be used for synchronously collecting the audio signal, and the (first) audio signal collected by the watch can be used as the environment reference audio signal, so that the noise reduction effect of the (second) audio can be improved, a foundation is laid for subsequent efficient response, and the user experience can be effectively improved.

Description

Technical Field

[0001] The present application belongs to the field of audio processing technology, and in particular, relates to an audio signal processing method, device, computer program product, watch and earphone. Background Art

[0002] With the rapid development of science and technology, smart wearable devices such as True Wireless Stereo (TWS) headphones and smart watches have become an indispensable part of people's daily lives. In actual use, headphones can often be used in conjunction with watches and can provide users with a variety of voice interaction functions. Specifically, headphones can collect audio signals and send them to watches for corresponding processing. However, in noisy environments, watches often find it difficult to accurately capture and recognize user voice commands from the audio collected by the headphones, and are thus unable to respond efficiently and accurately, resulting in a decline in user experience. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an audio signal processing method, device, computer program product, watch and headphones to solve the problem that the audio signal processing method in the prior art cannot accurately capture and recognize the user's voice commands, resulting in a decrease in user experience.

[0004] A first aspect of an embodiment of the present application provides an audio signal processing method applied to a watch, which may include:

[0005] When the voice interaction function is turned on, controlling the watch to collect a first audio signal;

[0006] Receiving a second audio signal collected by the earphone; the second audio signal is collected synchronously with the first audio signal;

[0007] The first audio signal is used as an environmental reference audio signal, and noise reduction processing is performed on the second audio signal to obtain a third audio signal.

[0008] A second aspect of an embodiment of the present application provides an audio signal processing method applied to a headset, which may include:

[0009] When the voice interaction function is turned on, controlling the earphone to collect a second audio signal;

[0010] The second audio signal is sent to the watch; the second audio signal is used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal, and to perform noise reduction processing on the second audio signal, and the first audio signal and the second audio signal are collected synchronously.

[0011] A third aspect of an embodiment of the present application provides an audio signal processing device applied to a watch, which may include:

[0012] A first acquisition module, used to control the watch to collect a first audio signal when the voice interaction function is turned on;

[0013] An audio receiving module, used for receiving a second audio signal collected by the earphone; the second audio signal is collected synchronously with the first audio signal;

[0014] The noise reduction processing module is used to perform noise reduction processing on the second audio signal by taking the first audio signal as an environmental reference audio signal to obtain a third audio signal.

[0015] A fourth aspect of an embodiment of the present application provides an audio signal processing device applied to a headset, which may include:

[0016] A second acquisition module, used to control the headset to collect a second audio signal when the voice interaction function is turned on;

[0017] An audio sending module is used to send the second audio signal to the watch; the second audio signal is used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal, and to perform noise reduction processing on the second audio signal, and the first audio signal and the second audio signal are collected synchronously.

[0018] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned audio signal processing methods are implemented.

[0019] The sixth aspect of an embodiment of the present application provides a watch, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the watch implements the steps of any one of the above-mentioned audio signal processing methods.

[0020] A seventh aspect of an embodiment of the present application provides an earphone, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the earphone implements the steps of any one of the above-mentioned audio signal processing methods.

[0021] An eighth aspect of the embodiments of the present application provides a computer program product, including a computer program, and when the computer program is executed, any one of the above-mentioned audio signal processing methods is executed.

[0022] Compared with the prior art, the embodiments of the present application have the following beneficial effects: when the voice interaction function is turned on, the embodiments of the present application control the watch to collect a first audio signal; receive a second audio signal collected by the headset; collect the second audio signal synchronously with the first audio signal; use the first audio signal as an environmental reference audio signal, perform noise reduction processing on the second audio signal, and obtain a third audio signal. In the embodiments of the present application, the headset and the watch can be used to collect audio signals synchronously, and the (first) audio signal collected by the watch can be used as an environmental reference audio signal, thereby improving the effect of noise reduction on the (second) audio, laying the foundation for subsequent efficient response, and effectively improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of an application scenario of audio signal processing in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of another application scenario of audio signal processing in an embodiment of the present application;

[0026] Figure 3 This is a flow chart of an embodiment of an audio signal processing method applied to a watch in an embodiment of the present application;

[0027] Figure 4 This is a flow chart of an embodiment of an audio signal processing method applied to headphones in an embodiment of the present application;

[0028] Figure 5 This is a structural diagram of an embodiment of an audio signal processing device applied to a watch in an embodiment of the present application;

[0029] Figure 6 This is a structural diagram of an embodiment of an audio signal processing device applied to headphones in an embodiment of the present application;

[0030] Figure 7 A schematic block diagram of a watch in an embodiment of the present application;

[0031] Figure 8 This is a schematic block diagram of an earphone in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0034] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0037] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0038] With the rapid development of technology, smart wearable devices such as True Wireless Stereo (TWS) headphones and smart watches have become an indispensable part of people's daily lives.

[0039] In actual use, headphones can often be used in conjunction with watches and can provide users with a variety of voice interaction functions. For example, see Figure 1, the earphones can be placed in the corresponding charging compartment. When the user needs to use the earphones, the earphones can be taken out of the charging compartment. After that, the earphones can establish a communication connection with the watch through a preset communication link, so that the earphones and the watch can cooperate to realize the voice interaction function. For another example, please refer to Figure 2 The earphones can be placed in the watch (that is, the watch also serves as a charging compartment for the earphones). When the user needs to use the earphones, the earphones can be taken out of the watch. After that, the earphones can establish a communication connection with the watch through a preset communication link, so that the earphones and the watch can cooperate to realize the voice interaction function.

[0040] In the existing voice interaction solution, the earphones can collect audio signals and send them to the watch; the watch can send the audio signals collected by the earphones to a preset server for subsequent processing such as voice recognition and semantic understanding, and control the earphones to interact based on the information returned by the server.

[0041] However, in noisy environments (such as roadsides, shopping malls, etc.), it is often difficult to accurately capture and recognize the user's voice commands from the audio collected by the headphones, making it impossible to respond efficiently and accurately, resulting in a decrease in the user's interactive experience.

[0042] In view of this, the embodiments of the present application provide an audio signal processing method, device, computer program product, watch and headphones to solve the problem that the audio signal processing method in the prior art cannot accurately capture and recognize the user's voice commands, resulting in a decrease in user experience.

[0043] In an embodiment of the present application, the headset and the watch can establish a communication connection through a preset communication link, and can be used together to better realize the voice interaction function.

[0044] Among them, the communication link can be a communication link established based on at least one wireless communication solution such as Wireless Local Area Networks (WLAN) (for example, Wireless Fidelity (WiFi)), Bluetooth, Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), infrared technology (IR), Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA) and Long Term Evolution (LTE).

[0045] When the user needs to use the watch and headphones for voice interaction, the voice interaction function can be turned on through a preset command trigger method; here, the command trigger method may include but is not limited to voice wake-up, pressing physical buttons on the watch or headphones, gesture touch on the watch screen, etc. The embodiment of the present application does not make specific limitations on this, and can be specific and situational settings according to actual needs.

[0046] For example, users can turn on the voice interaction function by saying a specific wake-up word (such as "voice assistant"). For another example, users can press (specifically single-click, double-click, long press for 3 seconds, etc.) a specific physical button on the watch or headset to turn on the voice interaction function. For another example, users can operate the watch screen with specific touch gestures, such as long press, slide or double-click, to turn on the voice interaction function.

[0047] In an embodiment of the present application, after the voice interaction function is turned on, the audio signal can be collected by using a watch and headphones at the same time; here, the audio signal collected by the watch can be called a first audio signal, and the audio signal collected by the headphones can be called a second audio signal; since the watch is far away from the sound source (the user's mouth), it can be considered that the proportion of ambient noise in the first audio signal collected by the watch is relatively large, while the proportion of the user's voice is relatively small; on the contrary, since the headphones are relatively close to the sound source, it can be considered that the proportion of user voice in the second audio signal collected by the headphones is relatively large, while the proportion of ambient noise is relatively small.

[0048] In order to achieve a better noise reduction effect, the embodiment of the present application can fully consider the characteristics of the first audio signal and the second audio signal when performing voice interaction, and use the first audio signal as the environmental reference audio signal to perform more targeted noise reduction processing on the second audio signal to obtain a third audio signal after noise reduction processing; since the third audio signal after noise reduction processing will contain a purer user voice than the second audio signal, the third audio signal can be used for subsequent voice recognition and semantic understanding to obtain more accurate recognition and understanding results, so that a response that accurately meets user needs can be made, which helps to improve the user's voice interaction experience.

[0049] The voice interaction process of the embodiment of the present application involves the mutual cooperation between the watch and the earphone. For ease of understanding, the audio signal processing of the embodiment of the present application will be introduced in detail from the perspectives of the watch and the earphone respectively.

[0050] Specifically, see Figure 3 In the embodiment of the present application, an embodiment of an audio signal processing method applied to a watch may include steps S301 to S303:

[0051] Step S301: When the voice interaction function is turned on, control the watch to collect a first audio signal.

[0052] The watch may be provided with a preset sound pickup device, such as a microphone, and the watch may collect audio signals using the sound pickup device.

[0053] In a specific implementation method of an embodiment of the present application, if the user says a specific wake-up word, the headset can collect an audio signal containing the wake-up word and can send the audio signal to the watch. After receiving the audio signal, the watch can perform keyword recognition on the audio signal; if it is recognized that the audio signal contains a specific wake-up word, it can be considered that the user needs to turn on the voice interaction function; in addition, the watch can also send a voice interaction function activation instruction to the headset, so that the headset can know that the voice interaction function is turned on and can cooperate with the watch to realize voice interaction.

[0054] In another specific implementation method of the embodiment of the present application, if it is detected that the user presses a specific physical button of the watch, it can be considered that the user needs to turn on the voice interaction function; at this time, the watch can send the voice interaction function activation instruction to the headset, so that the headset can know in time that the voice interaction function has been turned on, and can cooperate with the watch to realize voice interaction.

[0055] In another specific implementation method of the embodiment of the present application, if it is detected that the user presses a specific physical button of the headset, it can be considered that the user needs to turn on the voice interaction function. At this time, the headset can send a voice interaction function activation instruction to the watch, so that the watch can know that the voice interaction function is turned on and can cooperate with the watch to realize voice interaction.

[0056] In another specific implementation method of the embodiment of the present application, if it is detected that the user performs a specific touch gesture on the watch screen, it can be considered that the user needs to turn on the voice interaction function; at this time, the watch can send the voice interaction function activation instruction to the headset, so that the headset can know that the voice interaction function has been turned on and can cooperate with the watch to realize voice interaction.

[0057] When the voice interaction function is turned on, the watch can be controlled to collect the first audio signal; specifically, after the user turns on the voice interaction function through a preset command trigger, the sound pickup device provided on the watch can be used to collect audio signals to obtain the first audio signal.

[0058] Step S302: Receive a second audio signal collected by the earphone.

[0059] Step S303: Use the first audio signal as an environmental reference audio signal and perform noise reduction processing on the second audio signal to obtain a third audio signal.

[0060] When the voice interaction function is turned on, the earphone can also synchronously collect the audio signal to obtain a second audio signal; then, the earphone can also send the second audio signal to the watch.

[0061] In a specific implementation of the embodiment of the present application, the watch can send the collected original second audio signal to the watch; after receiving the original second audio signal, the watch can perform signal processing on the collected original second audio signal to obtain a processed second audio signal, and then use the first audio signal as the environmental reference audio signal to perform noise reduction processing on the second audio signal to obtain a third audio signal. It should be understood that when the watch performs signal processing on the original second audio signal, unless otherwise specified, the second audio signal referred to below is the processed second audio signal.

[0062] In another specific implementation method of the embodiment of the present application, the earphones can perform signal processing on the collected original second audio signal to obtain a processed second audio signal; thereafter, the processed second audio signal is sent to the watch. In this case, unless otherwise specified, the second audio signals referred to below are all processed second audio signals.

[0063] The above-mentioned signal processing may include but is not limited to any one or more processes such as synchronization processing, gain adjustment, delay processing and mixing processing, and the embodiments of the present application are not limited to this.

[0064] The transmission process of the audio signal between the above-mentioned earphones and the watch can be implemented based on a preset audio transmission protocol, which is not limited in the embodiments of the present application; for example, the transmission process of the audio signal between the earphones and the watch can be implemented based on the Hands-Free Profile (HFP).

[0065] Since the computing power of the watch is relatively sufficient, after receiving the second audio signal collected by the headset, the watch can perform noise reduction processing on the second audio signal to better identify the user voice contained in the second audio signal. Specifically, since the distance between the watch and the headset is relatively close, and the proportion of ambient noise in the first audio signal is relatively large, the first audio signal can be regarded as an environmental reference audio signal, so that the ambient noise in the second audio signal can be more accurately removed to obtain a third audio signal.

[0066] The above-mentioned noise reduction process may refer to any audio noise reduction method, which is not limited in the embodiment of the present application.

[0067] In a specific implementation of the embodiment of the present application, the watch can identify the environmental noise in the second audio signal based on the first audio signal using the Recursive Least Squares (RLS) algorithm. Specifically, the initial RLS filter can be initialized; then, the RLS filter can learn the environmental reference audio signal (i.e., the first audio signal) to learn the environmental noise, and then, based on the learned environmental noise, a signal segment similar to the environmental noise can be identified from the second audio signal (hereinafter referred to as the fourth audio signal), and the fourth audio signal can be filtered out from the second audio signal to obtain a third audio signal.

[0068] Compared with the second audio signal, the third audio signal has a higher signal-to-noise ratio, so the third audio signal can contain relatively pure user voice. Afterwards, subsequent speech recognition and semantic understanding can be performed based on the third audio signal to improve the accuracy of speech recognition and semantic understanding.

[0069] In the embodiment of the present application, the server can be used to perform voice recognition and semantic understanding on the third audio signal to meet the real-time requirements of the voice interaction process.

[0070] It should be understood that the servers that perform speech recognition and semantic understanding on the third audio signal can be different servers or the same server; for example, a certain server can be used to perform speech recognition on the third audio signal to obtain a speech recognition result; then, another server can be used to perform semantic understanding on the speech recognition result to obtain a semantic understanding result. For another example, a server can be used to first perform speech recognition on the third audio signal to obtain a speech recognition result; then, the server can perform semantic understanding on the speech recognition result to obtain a semantic understanding result.

[0071] The following will take the servers that perform speech recognition and semantic understanding on the third audio signal as different servers as an example to introduce the process of performing speech recognition and semantic understanding on the third audio signal in an embodiment of the present application.

[0072] In an embodiment of the present application, a preset communication module (such as a fourth generation mobile communication technology (4th Generation, 4G) data communication module) may be provided in the watch, and based on the communication module, the watch can establish a communication connection with the server.

[0073] After obtaining the third audio signal, the watch can send the third audio signal to a server (hereinafter referred to as the first server) providing automatic speech recognition (ASR) service through the communication module. The third audio signal can instruct the first server to perform speech recognition based on the third audio signal.

[0074] After receiving the third audio signal, the first server may perform voice recognition on the third audio signal to obtain text content corresponding to the third audio signal; then, the first server may send the text content to the watch.

[0075] After receiving the text content sent by the first server, the watch can send the text content to a server (hereinafter referred to as the second server) that provides natural language processing (Natural Language Processing, NLP) services. The text content can instruct the second server to perform semantic understanding based on the text content.

[0076] After receiving the text content sent by the watch, the second server can perform semantic understanding based on the text content; wherein, the method of semantic understanding can be any semantic understanding method in the prior art, and the embodiment of the present application is not limited to this; thereafter, the second server can also determine the interactive voice corresponding to the text content based on the result of semantic understanding.

[0077] For example, if the text content is a text corresponding to "Please help me play the next song", the interactive voice corresponding to the text content determined may be a voice corresponding to "OK, play the next song for you".

[0078] For another example, if the text content is a text corresponding to "play music", the interactive voice corresponding to the text content determined may be a voice corresponding to "OK, I will play music for you right away".

[0079] For another example, if the text content is the text corresponding to "Exit noise reduction mode", the determined interactive voice corresponding to the text content may be the voice corresponding to "OK, no problem".

[0080] For another example, if the text content is text corresponding to "enter noise reduction mode", the interactive voice corresponding to the text content determined may be a voice corresponding to "OK, noise reduction mode will be turned on for you right away".

[0081] For another example, after semantic understanding of the text content, if no logical semantic understanding result can be obtained, it can be determined that the interactive voice corresponding to the text content is the voice corresponding to "Sorry, I didn't hear what you said clearly."

[0082] Afterwards, the second server may determine the interactive voice corresponding to the text content as interactive information, and may send the interactive information to the watch.

[0083] In a specific implementation of an embodiment of the present application, the second server may also determine an interactive instruction corresponding to the text content. For example, if the text content is the text corresponding to "Please help me play the next song", the interactive instruction corresponding to the text content may be a song switching control instruction to instruct the headset and the watch to perform a song switching operation. For another example, if the text content is the text corresponding to "Play music", the interactive instruction corresponding to the text content may be a music playing instruction to instruct the headset to play music. For another example, if the text content is the text corresponding to "Exit noise reduction mode", the interactive instruction corresponding to the text content may be a noise reduction mode exit instruction to instruct the headset to exit the noise reduction mode. For another example, if the text content is the text corresponding to "Enter noise reduction mode", the interactive instruction corresponding to the text content may be a noise reduction mode entry instruction to instruct the headset to enter noise reduction mode.

[0084] If the second server determines the interactive voice and interactive instructions corresponding to the text content, the interactive voice and interactive instructions can be determined as interactive information, and the interactive information can be sent to the watch.

[0085] After receiving the interaction information, the watch can send a control instruction to the headset through a preset communication link based on the interaction information to instruct the headset to interact.

[0086] In the embodiment of the present application, if the interactive information includes interactive voice, the headset can be controlled to play the interactive voice through the control instruction. If the interactive information includes interactive instructions, the headset can be controlled to perform corresponding interactive actions through the control instruction.

[0087] For example, if the interactive information includes interactive voice, the watch can generate corresponding control instructions based on the interactive voice, and can send the control instructions to the earphone to instruct the earphone to play the interactive voice; after receiving the control instructions, the earphone can play the interactive voice.

[0088] For another example, the interactive information includes interactive voice and interactive instructions, and the interactive instruction is specifically a song switching control instruction. The watch can determine the audio corresponding to the song after the song switching based on the song switching control instruction (recorded as audio 1); then, it can generate a corresponding control instruction (recorded as play instruction 1) based on the interactive voice and audio 1, and can send play instruction 1 to the headset; after receiving play instruction 1, the headset can play the interactive voice, and then, can play audio 1 to achieve song switching.

[0089] For another example, the interactive information includes interactive voice and interactive instructions, and the interactive instruction is specifically a music playback instruction. The watch can determine the audio corresponding to the song being played based on the music playback instruction (recorded as audio 2); then, it can generate a corresponding control instruction (recorded as playback instruction 2) based on the interactive voice and audio 2, and can send playback instruction 2 to the headset; after receiving playback instruction 2, the headset can play the interactive voice, and then, can play audio 2 to achieve music playback.

[0090] For another example, the interactive information includes interactive voice and interactive instructions, and the interactive instruction is specifically an instruction to exit the noise reduction mode. The watch can generate corresponding control instructions based on the interactive voice and the instruction to exit the noise reduction mode, and can send the control instructions to the headphones. After receiving the control instructions, the headphones can play the interactive voice and then exit the noise reduction mode of the headphones.

[0091] For another example, the interactive information includes interactive voice and interactive instructions, and the interactive instruction is specifically an instruction to enter the noise reduction mode. The watch can generate corresponding control instructions based on the interactive voice and the instruction to enter the noise reduction mode, and can send the control instructions to the headphones. After receiving the control instructions, the headphones can play the interactive voice and then enter the noise reduction mode of the headphones.

[0092] The audio signal processing method of the embodiment of the present application will be described in detail below from the perspective of headphones.

[0093] Specifically, see Figure 4 In an embodiment of the present application, an audio signal processing method applied to a headset may include steps S401 to S402:

[0094] Step S401: When the voice interaction function is turned on, control the earphone to collect a second audio signal.

[0095] Step S402: Send a second audio signal to the watch.

[0096] When the user needs to perform voice interaction, the user can turn on the voice interaction function through a preset command trigger method. If the headset detects the preset command trigger method, it can send a voice interaction function turn-on command to the watch, so that the watch can know that the voice interaction function is turned on and can cooperate with the headset to realize voice interaction; or, if the headset receives the voice interaction function turn-on command sent by the watch, it can turn on the voice interaction function to cooperate with the watch to realize voice interaction.

[0097] In the embodiment of the present application, the earphone may be provided with a preset sound pickup device, such as a microphone. When the voice interaction function is turned on, the earphone may collect audio signals using the sound pickup device provided therein to obtain a second audio signal.

[0098] As an example, the headset may be provided with three microphones for collecting audio signals, and the three microphones may be set at different positions of the headset as needed, so as to realize active noise cancellation (ANC) of the hybrid topology structure, whereby the collected second audio signal will achieve preliminary noise reduction through ANC. In addition, the number of microphones included in the headset and the specific position of the microphone on the headset can be specific and situationally set according to actual needs, and this application does not limit this.

[0099] After collecting the second audio signal, the headset can send the second audio signal to the watch, and the second audio signal can be used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal and perform noise reduction processing on the second audio signal.

[0100] After receiving the second audio signal, the watch can perform noise reduction processing on the second audio signal based on the first audio signal. The specific processing process can refer to the relevant description above and will not be repeated here.

[0101] In a specific implementation of the embodiment of the present application, after collecting the second audio signal, the earphone can also perform signal processing on the second audio signal to obtain a processed second audio signal, and can send the processed second audio signal to the watch so that the watch performs noise reduction processing on the processed second audio signal. The second audio signal can specifically include an audio signal collected by the left earphone (hereinafter referred to as the left channel audio signal) and an audio signal collected by the right earphone (hereinafter referred to as the right channel audio signal); the signal processing can specifically include but is not limited to any one or more processes such as synchronization processing, gain adjustment, delay processing, and mixing processing.

[0102] For example, the left channel audio signal and the right channel audio signal can be processed synchronously. Specifically, the left channel audio signal and the right channel audio signal can be aligned in time according to the acquisition time of the left channel audio signal and the acquisition time of the right channel audio signal to avoid sound distortion or poor cancellation effect caused by phase deviation.

[0103] For another example, the gain of the left channel audio signal and the right channel audio signal may be adjusted. Specifically, the volume of the left channel audio signal and the right channel audio signal may be compared. Thereafter, the volume of the left channel audio signal and / or the right channel audio signal may be adjusted to make the volume of the two audio signals equivalent, thereby better balancing the two channels.

[0104] For another example, the left channel audio signal and the right channel audio signal may be delayed. Specifically, the left channel audio signal and / or the right channel audio signal may be delayed and adjusted to simulate different recording environments.

[0105] For another example, the left channel audio signal and the right channel audio signal may be mixed to obtain a stereo audio signal; the specific mixing method may refer to any audio signal mixing method in the prior art, and the present application embodiment does not limit this. For example, the left channel audio signal and the right channel audio signal may be directly added in proportion to obtain a stereo audio signal.

[0106] In a specific implementation method of an embodiment of the present application, in order to improve the robustness of voice interaction, it can also be determined whether the audio signal collected after the voice interaction function is turned on contains user voice; if the audio signal collected after the voice interaction function is turned on contains user voice, voice interaction can be performed based on the audio signal; otherwise, it can be considered that a false trigger has occurred, and the voice interaction function can be turned off; specifically, the audio signal can be collected using headphones within a preset time period after the voice interaction function is turned on, and it can be detected whether the audio signal collected by the headphones contains user voice; the specific value of the above-mentioned preset time period can be set according to actual needs, and the embodiment of the present application is not limited to this. For example, the preset time period can be 3 seconds, 5 seconds, 10 seconds, etc.

[0107] As an example, the preset time period can be 5 seconds, then the headset can keep collecting audio signals within 5 seconds after the voice interaction function is turned on, and can perform energy-based voice activity detection (VAD) on the audio signal; usually, if the collected audio signal contains user voice, the energy of the audio signal will be large; if the collected audio signal does not contain user voice, the energy of the audio signal will be small; therefore, it can be determined based on the energy of the audio signal whether the audio signal contains user voice; if the audio signal does not contain user voice, it can be considered that a false trigger has occurred, and the voice interaction function can be turned off; if the audio signal contains user voice, voice interaction can be performed based on the audio signal. Here, the headset can perform energy-based VAD on the collected audio signal, and if it is determined that the audio signal does not contain user voice, a voice interaction function turn-off instruction can be sent to the watch to instruct the watch to turn off the voice interaction function.

[0108] As another example, the preset time period can be 5 seconds, then the headset can keep collecting audio signals within 5 seconds after the voice interaction function is turned on, and can send the audio signal collected within 5 seconds to the watch. After receiving the audio signal, the watch can perform energy-based VAD on the audio signal, and can determine whether the audio signal contains user voice based on the energy of the audio signal; if the audio signal does not contain user voice, it can be considered that a false trigger has occurred, and then a voice interaction function shutdown instruction can be sent to the headset to instruct the headset to turn off the voice interaction function; if the audio signal contains user voice, voice interaction can be performed based on the audio signal.

[0109] After receiving the second audio signal, the watch can perform noise reduction processing on the second audio signal to obtain a third audio signal, and can perform voice recognition and semantic understanding on the third audio signal through a preset server to obtain interaction information corresponding to the third audio signal; thereafter, the watch can send control instructions to the earphones based on the interaction information to instruct the earphones to make corresponding interactions.

[0110] After receiving the control instruction sent by the watch, the control instruction can be parsed to obtain parsed information, and then the headset can be controlled to interact based on the parsed information.

[0111] For example, if the parsed information includes interactive voice, the headset can play the interactive voice.

[0112] For another example, if the parsed information includes interactive voice and audio 1, the headset can play the interactive voice, and then play audio 1 to achieve interaction of song switching or music playback.

[0113] For another example, if the parsed information includes interactive voice and noise reduction mode exit instructions, the headset can play the interactive voice, and then exit the noise reduction mode of the headset.

[0114] For another example, if the parsed information includes interactive voice and noise reduction mode entry instructions, the headset can play the interactive voice and then enter the noise reduction mode of the headset.

[0115] The noise reduction processing performed by the watch can significantly highlight the user's voice in the second audio signal, improve the accuracy of subsequent voice recognition, and reduce misunderstanding of user commands, thereby improving the user's voice interaction experience in noisy environments.

[0116] In summary, in the embodiment of the present application, when the voice interaction function is turned on, the watch is controlled to collect a first audio signal; a second audio signal collected by the earphone is received; the second audio signal is collected synchronously with the first audio signal; the first audio signal is used as an environmental reference audio signal, and the second audio signal is subjected to noise reduction processing to obtain a third audio signal. In the embodiment of the present application, the earphone and the watch can be used to collect audio signals synchronously, and the (first) audio signal collected by the watch can be used as an environmental reference audio signal, thereby improving the effect of noise reduction on the (second) audio, laying the foundation for subsequent efficient response, and effectively improving the user experience.

[0117] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0118] Corresponding to the audio signal processing method applied to a watch described in the above embodiment, Figure 5 A structural diagram of an embodiment of an audio signal processing device applied to a watch provided in an embodiment of the present application is shown.

[0119] In an embodiment of the present application, an audio signal processing device applied to a watch may include:

[0120] The first acquisition module 501 is used to control the watch to collect the first audio signal when the voice interaction function is turned on;

[0121] The audio receiving module 502 is used to receive a second audio signal collected by the earphone; the second audio signal is collected synchronously with the first audio signal;

[0122] The noise reduction processing module 503 is configured to use the first audio signal as an environmental reference audio signal and perform noise reduction processing on the second audio signal to obtain a third audio signal.

[0123] In a specific implementation of the embodiment of the present application, the device further includes:

[0124] a signal processing module, configured to perform signal processing on the second audio signal to obtain a processed second audio signal;

[0125] Furthermore, the noise reduction processing module is further configured to use the first audio signal as an environmental reference audio signal and perform noise reduction processing on the processed second audio signal to obtain a third audio signal.

[0126] In a specific implementation of the embodiment of the present application, the noise reduction processing module includes:

[0127] an audio signal determination submodule, configured to determine a fourth audio signal from the second audio signal based on the first audio signal; the fourth audio signal being a signal segment in the second audio signal corresponding to the first audio signal;

[0128] The audio signal filtering submodule is used to filter the fourth audio signal from the second audio signal to obtain the third audio signal.

[0129] In a specific implementation of the embodiment of the present application, the device further includes:

[0130] an audio sending module, configured to send the third audio signal to a server; the third audio signal is used to instruct the server to generate interaction information based on the third audio signal;

[0131] The instruction sending module is used to send a control instruction to the headset based on the interaction information when the interaction information is received.

[0132] Corresponding to the audio signal processing method applied to headphones described in the above embodiment, Figure 6 A structural diagram of an embodiment of an audio signal processing device applied to headphones provided in an embodiment of the present application is shown.

[0133] In an embodiment of the present application, an audio signal processing device applied to a headset may include:

[0134] The second acquisition module 601 is used to control the earphone to collect the second audio signal when the voice interaction function is turned on;

[0135] The audio sending module 602 is used to send the second audio signal to the watch; the second audio signal is used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal, and perform noise reduction processing on the second audio signal. The first audio signal and the second audio signal are collected synchronously.

[0136] In a specific implementation of the embodiment of the present application, the device further includes:

[0137] a signal processing module, configured to perform signal processing on the second audio signal to obtain a processed second audio signal;

[0138] Furthermore, the audio sending module is also used to send the processed second audio signal to the watch.

[0139] In a specific implementation of the embodiment of the present application, the device further includes:

[0140] The interaction module is used to control the headset to interact in response to the control instruction.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] Figure 7 A schematic block diagram of a watch provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0144] like Figure 7As shown, the watch 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned audio signal processing method embodiments are implemented, for example Figure 3 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 The functions of modules 501 to 503 are shown.

[0145] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the watch 7.

[0146] Those skilled in the art will understand that Figure 7 It is only an example of watch 7 and does not constitute a limitation of watch 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the watch 7 may also include input and output devices, network access devices, buses, etc.

[0147] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0148] The memory 71 may be an internal storage unit of the watch 7, such as a hard disk or memory of the watch 7. The memory 71 may also be an external storage device of the watch 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the watch 7. Furthermore, the memory 71 may include both an internal storage unit and an external storage device of the watch 7. The memory 71 is used to store the computer program and other programs and data required by the watch 7. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0149] Figure 8 A schematic block diagram of a watch provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0150] like Figure 8 As shown, the watch 8 of this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, the steps in the above-mentioned audio signal processing method embodiments are implemented, for example Figure 4 Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of modules 601 to 602 are shown.

[0151] Exemplarily, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 82 in the watch 8.

[0152] Those skilled in the art will understand that Figure 8 It is only an example of the watch 8 and does not constitute a limitation of the watch 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the watch 8 may also include input and output devices, network access devices, buses, etc.

[0153] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0154] The memory 81 may be an internal storage unit of the watch 8, such as a hard disk or memory of the watch 8. The memory 81 may also be an external storage device of the watch 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the watch 8. Furthermore, the memory 81 may include both an internal storage unit and an external storage device of the watch 8. The memory 81 is used to store the computer program and other programs and data required by the watch 8. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0155] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0156] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0157] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0158] In the embodiments provided in the present application, it should be understood that the disclosed devices / watches / headphones and methods can be implemented in other ways. For example, the device / watch embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0162] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An audio signal processing method, characterized in that: Applied to a watch, the method comprises: When the voice interaction function is turned on, controlling the watch to collect a first audio signal; Receiving a second audio signal collected by the earphone; the second audio signal is collected synchronously with the first audio signal; The first audio signal is used as an environmental reference audio signal, and noise reduction processing is performed on the second audio signal to obtain a third audio signal.

2. The audio signal processing method according to claim 1, characterized in that: After receiving the second audio signal collected by the earphone, the method further includes: performing signal processing on the second audio signal to obtain a processed second audio signal; The step of taking the first audio signal as an environmental reference audio signal and performing noise reduction processing on the second audio signal to obtain a third audio signal includes: The first audio signal is used as an environmental reference audio signal, and noise reduction processing is performed on the processed second audio signal to obtain a third audio signal.

3. The audio signal processing method according to claim 1, characterized in that: The step of taking the first audio signal as an environmental reference audio signal and performing noise reduction processing on the second audio signal to obtain a third audio signal includes: Based on the first audio signal, determining a fourth audio signal from the second audio signal; the fourth audio signal is a signal segment in the second audio signal corresponding to the first audio signal; The fourth audio signal is filtered out from the second audio signal to obtain the third audio signal.

4. The audio signal processing method according to any one of claims 1 to 3, characterized in that: After taking the first audio signal as the environmental reference audio signal and performing noise reduction processing on the second audio signal to obtain a third audio signal, the method further includes: Sending the third audio signal to the server; the third audio signal is used to instruct the server to generate interaction information based on the third audio signal; When the interaction information is received, a control instruction is sent to the headset based on the interaction information.

5. An audio signal processing method, characterized in that: Applied to headphones, the method comprises: When the voice interaction function is turned on, controlling the earphone to collect a second audio signal; The second audio signal is sent to the watch; the second audio signal is used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal, and to perform noise reduction processing on the second audio signal, and the first audio signal and the second audio signal are collected synchronously.

6. The audio signal processing method according to claim 5, characterized in that: After controlling the earphone to collect the second audio signal when the voice interaction function is turned on, the method further includes: performing signal processing on the second audio signal to obtain a processed second audio signal; The sending the second audio signal to the watch comprises: The processed second audio signal is sent to the watch.

7. The audio signal processing method according to claim 5 or 6, characterized in that: Also includes: In response to the control instruction, the earphone is controlled to interact.

8. An audio signal processing device, characterized in that: Applied to a watch, the device comprises: A first acquisition module, used to control the watch to collect a first audio signal when the voice interaction function is turned on; An audio receiving module, used for receiving a second audio signal collected by the earphone; the second audio signal is collected synchronously with the first audio signal; The noise reduction processing module is used to perform noise reduction processing on the second audio signal by taking the first audio signal as an environmental reference audio signal to obtain a third audio signal.

9. An audio signal processing device, characterized in that: Applied to headphones, the device comprises: A second acquisition module, used to control the headset to collect a second audio signal when the voice interaction function is turned on; An audio sending module is used to send the second audio signal to the watch; the second audio signal is used to instruct the watch to use the first audio signal collected by the watch as the environmental reference audio signal, and to perform noise reduction processing on the second audio signal, and the first audio signal and the second audio signal are collected synchronously.

10. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables the audio signal processing method according to any one of claims 1 to 4 to be executed, or enables the audio signal processing method according to any one of claims 5 to 7 to be executed.

11. A watch comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the watch implements the steps of the audio signal processing method according to any one of claims 1 to 4.

12. A headset comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the headset implements the steps of the audio signal processing method according to any one of claims 5 to 7.

Images