Two-way communication method and device, electronic device and computer-readable storage medium

Abstract

The embodiments of the present invention disclose a two-way communication method and device, an electronic device and a computer-readable storage medium. The method includes: using a microphone array to collect user voice and environmental noise in real time; calculating the position of the user's mouth relative to the microphone array and the environmental noise sound pressure level according to the difference in the microphone collection signals at different positions in the microphone array, and calculating the distance of the user's ear relative to the speaker; when sending a call, determining the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array, and controlling the microphone array to collect audio signals within the collection protection angle; when receiving a call, determining the optimal gain value of the speaker according to the size of the sound of the other end received by the speaker, the distance of the user's ear relative to the speaker, the environmental noise sound pressure level and the speaker sensitivity, and controlling the speaker to use the optimal gain value to play the sound signal of the other end. The solution of the embodiment of the present invention can improve the two-way call quality of the smart watch.

Description

Technical Field

[0001] The present invention relates to the field of voice processing technology, and in particular to a two-way conversation method and device, electronic equipment and a computer-readable storage medium. Background Art

[0002] At present, electronic devices with two-way call functions, such as handheld devices, conference phones, and smart wearable devices, are becoming more and more popular. Common ones include mobile phones, tablets, telephone conference terminals, smart watches, smart bracelets, etc. When a user is making a two-way call, the position of the electronic device or the position of the user's head is not fixed. Different users or the same user at different times may have different positions of the electronic device relative to the user's head. When the user's head is far away from the electronic device, the ambient sound is noisy, or the other party's voice is too soft and the transmitted signal is too weak, the call quality will be affected.

[0003] How the microphone can better pick up the user's voice and transmit it to the other party so that the other party can hear clear voice; how the speaker's playback signal can reach the user's ear with an appropriate signal-to-noise ratio so that the user can hear clear voice from the other party are both issues that need to be solved to improve the quality of two-way calls. Summary of the invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a two-way communication method and device, an electronic device and a computer-readable storage medium, which can effectively improve the quality of two-way communication.

[0005] According to a first aspect of the present invention, a two-way communication method is provided, which is applied to an electronic device with a microphone array and a speaker, comprising:

[0006] Use microphone array to collect user voice and environmental noise in real time;

[0007] The position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise are calculated based on the difference in the microphone signals collected by different positions in the microphone array, and the distance of the user's ear relative to the speaker is calculated based on the fact that the relative positions of the user's ear and the user's mouth are fixed and the relative positions of the microphone array and the speaker are fixed;

[0008] When speaking, the microphone array is controlled to collect audio signals within the collection protection angle, according to the position of the user's mouth relative to the microphone array, and the sound emitted by the user's mouth is within the collection protection angle;

[0009] When receiving a call, the optimal gain value of the speaker is determined based on the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the ambient noise sound pressure level, and the speaker sensitivity, and the speaker is controlled to use the optimal gain value to play the sound signal of the other party.

[0010] The step of determining the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array includes:

[0011] The line connecting the multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline;

[0012] Determining the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call;

[0013] The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

[0014] As an improvement to the above solution, the method of the embodiment of the present invention further includes:

[0015] The gain value of each microphone in the microphone array is adjusted according to the sound pressure level of the audio signal collected by the microphone array.

[0016] As an improvement to the above solution, the method of the embodiment of the present invention further includes:

[0017] Set the minimum speaker gain adjustment step;

[0018] When the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is smaller than the minimum speaker gain adjustment step, the speaker is controlled to still use the current gain value to play the sound signal of the other end.

[0019] As an improvement to the above solution, the method of the embodiment of the present invention further includes:

[0020] When the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker, the speaker is controlled to use the maximum gain value to play the sound signal of the other end, and at the same time reminds the user that the current call status is not good.

[0021] According to a second aspect of the present invention, there is provided a two-way communication device, which is applied to an electronic device with a microphone array and a speaker, comprising:

[0022] A collection module is used to collect user voice and environmental noise in real time using a microphone array;

[0023] A calculation module, used to calculate the position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise according to the difference in the microphones collected at different positions in the microphone array, and calculate the distance of the user's ear relative to the speaker based on the fact that the relative position of the user's ear and the user's mouth is fixed and the relative position of the microphone array and the speaker is fixed;

[0024] A speech processing module, used for determining the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array when speaking, and controlling the microphone array to collect audio signals within the collection protection angle if the sound emitted by the user's mouth is within the collection protection angle;

[0025] The call processing module is used to determine the optimal gain value of the speaker when receiving a call based on the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the ambient noise sound pressure level, and the speaker sensitivity, and control the speaker to use the optimal gain value to play the sound signal of the other party.

[0026] As an improvement of the above solution, the speech transmission processing module includes a collection protection angle determination unit, and the collection protection angle determination unit is used to:

[0027] The line connecting the multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline;

[0028] Determining the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call;

[0029] The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

[0030] As an improvement of the above solution, the speech processing module also includes a microphone gain control unit, which is used to control the gain value of each microphone in the microphone array according to the sound pressure level of the audio signal collected by the microphone array.

[0031] As an improvement of the above solution, the call receiving processing module includes a gain step setting unit and a speaker gain control unit.

[0032] The gain step setting unit is used to set the minimum speaker gain adjustment step;

[0033] The speaker gain control unit is used to control the speaker to still use the current gain value to play the sound signal of the other end when the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is less than the minimum speaker gain adjustment step.

[0034] As an improvement of the above solution, the speaker gain control unit is further used to control the speaker to use the maximum gain value to play the sound signal of the other end when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker;

[0035] The call processing module also includes a call status reminder unit.

[0036] The call status reminder unit is used to remind the user that the current call status is not good when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker.

[0037] According to a third aspect of the present invention, there is provided an electronic device, comprising a microphone array, a speaker, a memory and a processor.

[0038] The memory stores a computer program, which is loaded and executed by the processor to implement the aforementioned two-way communication method.

[0039] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more computer programs, and when the one or more computer programs are executed by a processor, the two-way communication method described above is implemented.

[0040] The embodiments of the present invention can achieve the following beneficial effects by adopting at least one of the above technical solutions:

[0041] The two-way communication method and device, electronic device and computer-readable storage medium provided by the embodiments of the present invention use a microphone array to collect user voice and environmental noise in real time, calculate the position of the user's mouth relative to the microphone array and the sound pressure level of the environmental noise according to the difference of the microphone collection signals at different positions in the microphone array, and calculate the distance of the user's ear relative to the speaker; when sending a call, the collection protection angle of the microphone array is determined according to the position of the user's mouth relative to the microphone array, and the microphone array is controlled to collect audio signals only within the collection protection angle. Since the sound emitted by the user's mouth is within the collection protection angle, and the microphone array does not process the audio signal outside the collection protection angle, the noise in the environment can be partially removed, and the signal-to-noise ratio of the collected user voice can be improved, so that the other end can hear clear voice; when receiving a call, the optimal gain value of the speaker is determined according to the size of the sound of the other end received by the speaker, the distance of the user's ear relative to the speaker, the sound pressure level of the environmental noise and the sensitivity of the speaker, and the speaker is controlled to use the optimal gain value to play the sound signal of the other end, ensuring that the sound signal of the other end can also reach the user's ear with a suitable signal-to-noise ratio, so that the user can hear the clear voice of the other end. It can be seen that the solution of the embodiment of the present invention can improve the quality of two-way calls, and can also effectively solve the impact on call quality caused by the non-fixed relative position of electronic devices such as smart watches and the user's head during two-way calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings. In the drawings:

[0043] Figure 1 It is a flowchart of a two-way communication method provided by an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a collection protection angle of a microphone array provided by an embodiment of the present invention;

[0045] Figure 3 It is a structural schematic diagram of a two-way communication device provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. These embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0048] Figure 1 is a flow chart of a two-way communication method provided by an embodiment of the present invention. The method of the embodiment of the present invention is applied to an electronic device with a microphone array and a speaker, see Figure 1 As shown, including:

[0049] Step S110: using a microphone array to collect user voice and environmental noise in real time.

[0050] A microphone array is composed of two or more microphones located at different positions in space. In practical applications, considering that electronic devices are generally small in size and it is difficult to have multiple microphones in space, the microphone array of common electronic devices with two-way call functions, especially smart wearable devices such as smart watches and smart bracelets, is generally composed of two microphones. In the following description, a microphone array composed of two microphones will be used as an example to illustrate the solution of the embodiment of the present invention. It can be understood that the solution of the embodiment of the present invention is also applicable to a microphone array composed of multiple microphones.

[0051] When a user is on a call, the microphone array is used to collect surrounding sound signals in real time. The collected sound signals include user voice and environmental noise.

[0052] Step S120, based on the differences in signals collected by microphones at different positions in the microphone array, the position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise are calculated, and based on the fact that the relative positions of the user's ears and the user's mouth are fixed, and the relative positions of the microphone array and the speaker are fixed, the distance of the user's ears relative to the speaker is calculated.

[0053] Each microphone in the microphone array has a different spatial position relative to the position of the user's head, so the audio signals collected will also be different. The audio signals collected by each microphone are processed, and the processing methods include but are not limited to: amplification, noise elimination, channel merging, characteristic audio signal screening, Fourier transform, etc., to obtain the signal energy, frequency domain data and / or time domain data of the audio signal, and then calculate the signal energy difference, phase difference, delay information, etc. of the audio signals received by different microphones.

[0054] Since the relative positions of different microphones in the microphone array are known, the position of the user's mouth as the sound source relative to the microphone array can be calculated based on the differences in audio signals collected by microphones at different positions in the microphone array, and the sound pressure level of the ambient noise can also be calculated.

[0055] In addition, based on the fact that the relative positions of the user's ears and mouth are fixed for the same user, and the relative positions of the microphone array and the speaker in the electronic device are fixed, after obtaining the position of the user's mouth relative to the microphone array, the distance of the user's ears relative to the speaker can be further calculated.

[0056] Step S130, when speaking, determine the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array, the sound emitted by the user's mouth is within the collection protection angle, and control the microphone array to collect audio signals within the collection protection angle.

[0057] After the position of the user's mouth as the sound source relative to the microphone array is calculated in the above step S120, the collection protection angle of the microphone array is determined according to the position of the user's mouth relative to the microphone array, so that the sound emitted by the user is within the collection protection angle, and the microphone array is controlled to collect audio signals only within the collection protection angle, and audio signals outside the collection protection angle are not processed. This can partially remove noise in the environment and improve the signal-to-noise ratio of the collected signal.

[0058] Figure 2 It is a schematic diagram of a collection protection angle of a microphone array provided in an embodiment of the present invention. Figure 2 In the figure, the microphone array consists of two microphones Mic1 and Mic2, and the position of the user's mouth is identified as the sound source, where L2 is the line connecting the two microphones Mic1 and Mic2, L3 is the line connecting Mic2 and the sound source, L1 and L2 constitute the collection protection angle A1, L4 and L2 constitute the collection protection angle A2, and A1 = A2. The microphone array only processes audio signals within the collection protection angles A1 and A2, and does not process audio signals outside the collection protection angles A1 and A2.

[0059] How to determine the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array? The embodiment of the present invention provides the following determination method:

[0060] The line connecting multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline;

[0061] Determine the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call;

[0062] The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

[0063] Combine the following Figure 2 The above-mentioned determination method will be described.

[0064] Figure 2 In the figure, the microphone array is composed of two microphones Mic1 and Mic2, the line connecting the two microphones Mic1 and Mic2 is used as the baseline L2, the user's mouth is identified as the sound source, and the line connecting the sound source and the microphone Mic2 which is the farthest from the microphone array is used as the sound source line L3. The sound source collection angle A0 is determined by the acute angle formed by the intersection of the sound source line L3 and the baseline L2.

[0065] During a call, the position of the user's mouth relative to the microphone array may change due to the movement of the electronic device or the user's head. By statistically analyzing the position changes of the user's mouth relative to the microphone array, the fluctuation range of the sound source line L3 is determined, and the fluctuation angle A3 of the sound source line L3 is determined based on the maximum fluctuation range of the sound source line L3.

[0066] The collection protection angle A2 of the microphone array on the sound source side is determined by the sum of the sound source collection angle A0 and the fluctuation angle A3, where A2=A0+A3; and then with the baseline L2 as the symmetry axis, the collection protection angle A1 of the microphone array on the non-sound source side is determined, where A1=A2.

[0067] From the above description, it can be seen that when determining the collection protection angle of the microphone array, the embodiment of the present invention determines the collection protection angle of the microphone array by summing the sound source collection angle A0 and the fluctuation angle A3 of the sound source line L3. This takes into account the position change of the user's mouth relative to the microphone array during the user's call. Even if the sound source line L3 fluctuates within a certain range during the user's call, the sound emitted by the user's mouth is still within the collection protection angle.

[0068] Step S140, when receiving a call, the optimal gain value of the speaker is determined according to the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the ambient noise sound pressure level and the speaker sensitivity, and the speaker is controlled to use the optimal gain value to play the sound signal of the other party.

[0069] First, it should be noted that step S140 and step S130 are two directions or two aspects of a two-way call. In an actual call, these two steps may be processed in parallel or may be processed alternately.

[0070] The speaker sensitivity is recorded in the speaker parameter table based on the test results when the speaker leaves the factory. The distance between the user's ear and the speaker and the ambient noise sound pressure level can be calculated in real time based on the signal collected by the microphone array, and these two values ​​have been calculated in the above step S120. The volume of the sound received by the speaker is related to the strength of the electrical signal input to the speaker, which can be obtained based on the electrical signal received by the speaker during the user's call.

[0071] The sound pressure level of the speaker playback signal reaching the user's ear is related to the volume of the opposite end sound received by the speaker, the distance of the user's ear relative to the speaker, the ambient noise sound pressure level, the speaker gain value, and the speaker sensitivity. This step S140 determines the optimal gain value of the speaker according to the volume of the opposite end sound received by the speaker, the distance of the user's ear relative to the speaker, the ambient noise sound pressure level, and the speaker sensitivity, and controls the speaker to use the optimal gain value to play the opposite end sound signal when receiving a call, thereby ensuring that the opposite end sound signal can reach the user's ear with a suitable signal-to-noise ratio, so that the user can hear clearly and obtain a better listening effect.

[0072] The process of determining the optimal gain value of the speaker is described below.

[0073] The speaker will undergo a sensitivity test in the laboratory before leaving the factory. The test method is: when the distance between the speaker and the user's ear is a predetermined distance D_Golden, the preset gain value of the speaker is Gain_RA, and the rated input value of the speaker is IN_RA, the sound pressure level of the speaker playback signal reaching the user's ear is measured to be SPLSPK(IN_RA, D_Golden, Gain_RA). The sound pressure level SPLSPK, the preset gain value Gain_RA of the speaker, the predetermined distance D_Golden and the rated input value IN_RA are recorded together in the speaker sensitivity parameters.

[0074] During a two-way call, the signal-to-noise ratio of the speaker playback signal reaching the user's ear is SNR(i), SNR(i) = SPLSPK(IN_i, D_i, Gain_i) - SPL(noise_i). SPLSPK(IN_i, D_i, Gain_i) is the sound pressure level of the speaker playback signal reaching the user's ear at time i, which is affected by the speaker input signal IN_i, the distance D_i between the speaker and the user's ear, and the speaker gain value Gain_i. SPL(noise_i) is the ambient noise sound pressure level at time i, which can be calculated in real time based on the audio signal collected by the microphone array.

[0075] The speaker parameter table generally also records the minimum signal-to-noise ratio SNR_MIN parameter. If the signal-to-noise ratio SNR(i) of the speaker playback signal reaching the user's ear is to be no less than the minimum signal-to-noise ratio SNR_MIN, the minimum value of the sound pressure level SPLSPK of the speaker playback signal reaching the user's ear must be SNR_MIN+SPL(noise_i).

[0076] Combined with the known speaker sensitivity parameters SPLSPK (IN_RA, D_Golden, Gain_RA), the speaker gain value Gain_i can be calculated as Gain_i = SNR_MIN + SPL (noise_i) - SPLSPK (IN_RA, D_Golden, Gain_RA) + (IN_RA - IN_i) + 20*log10 (D_i / D_Golden) + Gain_RA,

[0077] Among them, IN_i is the signal size input to the speaker at time i, and this data is affected by the sound volume of the other end; D_i is the distance between the user's ear and the speaker at time i, and this data can be calculated based on the audio signal collected by the microphone array; Gain_RA is the preset gain value of the speaker before leaving the factory.

[0078] It can be seen from the above steps that the two-way call method of the embodiment of the present invention, when sending a call, determines the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array, and controls the microphone array to collect audio signals only within the collection protection angle. Since the sound emitted by the user's mouth is within the collection protection angle, and the microphone array does not process the audio signal outside the collection protection angle, it is possible to partially remove the noise in the environment, improve the signal-to-noise ratio of the collected user sound, so that the other party can hear clear voice; when receiving a call, the optimal gain value of the speaker is determined according to the volume of the sound of the other party received by the speaker, the distance of the user's ear relative to the speaker, the sound pressure level of the ambient noise, and the sensitivity of the speaker, and the speaker is controlled to use the optimal gain value to play the sound signal of the other party, thereby ensuring that the sound signal of the other party can also reach the user's ear with a suitable signal-to-noise ratio, so that the user can hear clear voice from the other party.

[0079] In some method embodiments, the two-way communication method of the embodiment of the present invention further includes: adjusting the gain value of each microphone in the microphone array according to the sound pressure level of the audio signal collected by the microphone array.

[0080] Microphone gain refers to the excitation and level compensation of the microphone input signal to make the microphone perform better. Microphone gain is essentially the adjustment of the audio level to affect the amplitude, showing the height of the external sound. If the microphone gain is too large, it is easy to be distorted. If it is too small, the noise will be large and the effect will be poor. Therefore, it is necessary to set a suitable gain value for each microphone in the microphone array according to the sound pressure level of the audio signal collected by the microphone array, so that the audio signal collected by the microphone remains clean and energetic, and at the same time does not cause distortion or "puff" sound, and does not damage the sound quality and clarity.

[0081] In some method embodiments, the two-way call method of the embodiment of the present invention also includes: setting a minimum speaker gain adjustment step; when the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is less than the minimum speaker gain adjustment step, controlling the speaker to still use the current gain value to play the sound signal of the other end.

[0082] During a two-way call, the volume of the sound received by the speaker, the distance between the user's ear and the speaker, and the sound pressure level of the ambient noise will all change, and changes in these factors will cause changes in the optimal gain value of the speaker. To avoid frequent adjustments to the speaker gain value, you can set a minimum speaker gain adjustment step, such as 3dB, and compare the determined optimal gain value of the speaker with the current gain value of the speaker. If the difference between the two is within the 3dB range, the control speaker will still use the current gain value to play the sound signal of the other end.

[0083] In some method embodiments, the two-way call method of the embodiment of the present invention also includes: when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker, controlling the speaker to use the maximum gain value to play the sound signal of the other end, and at the same time reminding the user that the current call status is not good.

[0084] When the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker, it means that the speaker is not in the best call state. Either the sound received by the speaker is too small, or the distance between the user's ear and the speaker is too large, or the surrounding environment is too noisy. In this case, even if the speaker is controlled to use the maximum gain value to play the sound signal of the other end, it cannot guarantee that the user can hear the clear voice of the other end. At this time, the user can be reminded that the call is not in the best state. The content of the reminder can be: reduce the distance between the user's ear and the speaker, or choose a quieter environment. The reminder can be a voice reminder, a vibration reminder, or a text wake-up call through the display screen. Of course, multiple methods can be combined to give reminders at the same time.

[0085] In summary, the two-way conversation method of the embodiment of the present invention can improve the two-way conversation quality in many aspects, and can also effectively solve the impact on the conversation quality caused by the non-fixed relative position of the electronic device and the user's head during the two-way conversation.

[0086] The two-way communication method described above is based on the same technical concept. The embodiment of the present invention further provides a two-way communication device, which is applied to an electronic device with a microphone array and a speaker. Figure 3 is a schematic diagram of the structure of a two-way communication device provided by an embodiment of the present invention, see Figure 3 As shown, the two-way communication device 300 of the embodiment of the present invention includes:

[0087] The collection module 310 is used to collect user voice and environmental noise in real time using a microphone array;

[0088] A calculation module 320 is used to calculate the position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise according to the difference in the microphones collected at different positions in the microphone array, and calculate the distance of the user's ear relative to the speaker based on the fact that the relative position of the user's ear and the user's mouth is fixed and the relative position of the microphone array and the speaker is fixed;

[0089] The speech processing module 330 is used to determine the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array when speaking, and the sound emitted by the user's mouth is within the collection protection angle, and control the microphone array to collect audio signals within the collection protection angle;

[0090] The call processing module 340 is used to determine the optimal gain value of the speaker when receiving a call based on the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the ambient noise sound pressure level, and the speaker sensitivity, and control the speaker to use the optimal gain value to play the sound signal of the other party.

[0091] In some device embodiments, the speech processing module 330 includes a collection protection angle determination unit 331, and the collection protection angle determination unit 331 is used to:

[0092] The line connecting the multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline;

[0093] Determining the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call;

[0094] The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

[0095] In some device embodiments, the speech processing module 330 further includes a microphone gain control unit 332, wherein the microphone gain control unit 332 is configured to control the gain value of each microphone in the microphone array according to the sound pressure level of the audio signal collected by the microphone array.

[0096] In some device embodiments, the call processing module 340 includes a gain step setting unit 341 and a speaker gain control unit 342.

[0097] The gain step setting unit 341 is used to set the minimum speaker gain adjustment step;

[0098] The speaker gain control unit 342 is used to control the speaker to still use the current gain value to play the sound signal of the other end when the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is less than the minimum speaker gain adjustment step.

[0099] In some device embodiments, the speaker gain control unit 342 is further configured to control the speaker to use the maximum gain value to play the sound signal of the other end when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker;

[0100] The call processing module further includes a call status reminder unit 343, which is used to remind the user that the current call status is not good when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker.

[0101] The implementation process of each module or unit in the two-way communication device of the embodiment of the present invention can refer to the above method embodiment, which will not be described in detail here.

[0102] The embodiment of the present invention also provides an electronic device, which is of the same technical concept as the aforementioned two-way communication method. Figure 4 The electronic device 400 provided in the embodiment of the present invention includes: a microphone array 401, a speaker 402, a memory 410 and a processor 420, wherein the memory 410 may be a memory, such as a high-speed random access memory (RAM), or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 410 stores a computer program, which is loaded and executed by the processor 420 to implement the aforementioned two-way communication method.

[0103] At the hardware level, the electronic device 400 may also selectively include hardware such as a display panel 430, an interface module 440, and a communication module 450. The memory 410, the processor 420, the display panel 430, the interface module 440, the communication module 450, the microphone array 401, the speaker 402, etc. may be interconnected via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0104] The embodiment of the present invention further provides a computer-readable storage medium, which stores one or more computer programs. When the one or more computer programs are executed by a processor, the two-way communication method described above is implemented, and is specifically used to execute:

[0105] Use microphone array to collect user voice and environmental noise in real time;

[0106] The position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise are calculated based on the difference in the microphone signals collected by different positions in the microphone array, and the distance of the user's ear relative to the speaker is calculated based on the fact that the relative positions of the user's ear and the user's mouth are fixed and the relative positions of the microphone array and the speaker are fixed;

[0107] When speaking, the microphone array is controlled to collect audio signals within the collection protection angle, according to the position of the user's mouth relative to the microphone array, and the sound emitted by the user's mouth is within the collection protection angle;

[0108] When receiving a call, the optimal gain value of the speaker is determined based on the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the ambient noise sound pressure level, and the speaker sensitivity, and the speaker is controlled to use the optimal gain value to play the sound signal of the other party.

[0109] It should be understood by those skilled in the art that the various embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware embodiments. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing a computer program.

[0110] These computer programs can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] In some typical configurations, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable storage medium.

[0113] Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A two-way communication method, applied to an electronic device with a microphone array and a speaker, comprising: Use microphone array to collect user voice and environmental noise in real time; The position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise are calculated based on the difference in the microphone signals collected by different positions in the microphone array, and the distance of the user's ear relative to the speaker is calculated based on the fact that the relative positions of the user's ear and the user's mouth are fixed and the relative positions of the microphone array and the speaker are fixed; When speaking, the microphone array is controlled to collect audio signals within the collection protection angle, according to the position of the user's mouth relative to the microphone array, and the sound emitted by the user's mouth is within the collection protection angle; When receiving a call, the optimal gain value of the speaker is determined according to the volume of the sound received by the speaker, the distance between the user's ear and the speaker, the sound pressure level of the ambient noise, and the sensitivity of the speaker, and the speaker is controlled to use the optimal gain value to play the sound signal of the other party; Determining the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array includes: The line connecting the multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline; Determining the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call; The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

2. The method according to claim 1, characterized in that: Also includes: The gain value of each microphone in the microphone array is adjusted according to the sound pressure level of the audio signal collected by the microphone array.

3. The method according to claim 1, characterized in that: Also includes: Set the minimum speaker gain adjustment step; When the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is smaller than the minimum speaker gain adjustment step, the speaker is controlled to still use the current gain value to play the sound signal of the other end.

4. The method according to claim 1 or 3, characterized in that: Also includes: When the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker, the speaker is controlled to use the maximum gain value to play the sound signal of the other end, and at the same time reminds the user that the current call status is not good.

5. A two-way communication device, applied to an electronic device with a microphone array and a speaker, comprising: A collection module is used to collect user voice and environmental noise in real time using a microphone array; A calculation module, used to calculate the position of the user's mouth relative to the microphone array and the sound pressure level of the ambient noise according to the difference in the microphones collected at different positions in the microphone array, and calculate the distance of the user's ear relative to the speaker based on the fact that the relative position of the user's ear and the user's mouth is fixed and the relative position of the microphone array and the speaker is fixed; A speech processing module, used for determining the collection protection angle of the microphone array according to the position of the user's mouth relative to the microphone array when speaking, and controlling the microphone array to collect audio signals within the collection protection angle if the sound emitted by the user's mouth is within the collection protection angle; The call processing module is used to determine the optimal gain value of the speaker according to the volume of the sound of the other end received by the speaker, the distance between the user's ear and the speaker, the sound pressure level of the ambient noise, and the sensitivity of the speaker when receiving a call, and control the speaker to use the optimal gain value to play the sound signal of the other end; The speech transmission processing module includes a collection protection angle determination unit, and the collection protection angle determination unit is used to: The line connecting the multiple microphones in the microphone array is used as the baseline, the line connecting the user's mouth and the microphone farthest from the microphone array is used as the sound source line, and the sound source collection angle is determined by the acute angle formed by the intersection of the sound source line and the baseline; Determining the fluctuation angle of the sound source line according to the position change of the user's mouth relative to the microphone array during the call; The collection protection angle of the microphone array on the sound source side is determined by the sum of the sound source collection angle and the fluctuation angle, and then the collection protection angle of the microphone array on the non-sound source side is determined with the baseline as the symmetry axis. The collection protection angles on both sides of the baseline are the same.

6. The device according to claim 5, characterized in that: The speech processing module further includes a microphone gain control unit, which is used to control the gain value of each microphone in the microphone array according to the sound pressure level of the audio signal collected by the microphone array.

7. The device according to claim 5, characterized in that: The call receiving processing module includes a gain step setting unit and a speaker gain control unit. The gain step setting unit is used to set the minimum speaker gain adjustment step; The speaker gain control unit is used to control the speaker to still use the current gain value to play the sound signal of the other end when the difference between the determined optimal gain value of the speaker and the current gain value of the speaker is less than the minimum speaker gain adjustment step.

8. The device according to claim 5 or 7, characterized in that: The speaker gain control unit is further used to control the speaker to use the maximum gain value to play the sound signal of the other end when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker; The call processing module also includes a call status reminder unit. The call status reminder unit is used to remind the user that the current call status is not good when the determined optimal gain value of the speaker exceeds the maximum gain value of the speaker.

9. An electronic device comprising a microphone array, a speaker, a memory and a processor, The memory stores a computer program, which is loaded and executed by the processor to implement the two-way communication method according to any one of claims 1 to 4.

10. A computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs, when executed by a processor, implement the two-way communication method according to any one of claims 1 to 4.

Images