Control method of microphone on intelligent glasses, electronic equipment and storage medium

Abstract

The invention relates to a control method of a microphone on intelligent glasses, electronic equipment and a storage medium. The control method comprises the steps that a pickup direction and a target microphone are determined according to an application scene of the intelligent glasses, a microphone array of the intelligent glasses comprises a plurality of microphones with different distribution positions and adjustable directions, and the target microphone is a microphone in the microphone array; a sound signal in the application scene is collected through the target microphone, and the target microphone is adjusted to point to the pickup direction. Through the application, the technical problem that the intelligent glasses microphone lacks flexibility in multi-scene application in the prior art is solved, and the technical effect of improving the flexibility of the intelligent glasses microphone is achieved.

Description

Technical Field

[0001] The present application relates to the field of speech recognition, and in particular to a control method, electronic device, and storage medium for a microphone on smart glasses. Background Art

[0002] With the continuous advancement of technology, smart glasses have become an indispensable part of people's daily lives. By adding a microphone array to smart glasses, smart glasses can be equipped with a sound pickup function. Therefore, smart glasses support a variety of applications in terms of voice, such as voice assistants, conversation translation, meeting records, and telephone translation.

[0003] However, in the related technology, smart glasses microphones lack flexibility in multi-scenario applications, and no effective solution has been proposed for this technical problem. Summary of the invention

[0004] The embodiments of the present application provide a control method, an electronic device, and a storage medium for a microphone on smart glasses, so as to at least solve the technical problem in the related art that the microphone of smart glasses lacks flexibility in multi-scenario applications.

[0005] According to one aspect of an embodiment of the present application, a method for controlling a microphone on smart glasses is provided, including: determining a sound pickup direction and a target microphone according to an application scenario of the smart glasses, the microphone array of the smart glasses including a plurality of microphones with different distribution positions and adjustable directions, the target microphone being a microphone in the microphone array; collecting sound signals in the application scenario through the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction.

[0006] Optionally, determining the sound pickup direction and the target microphone according to an application scenario of the smart glasses includes: in response to the target sound source of the application scenario being the wearer, determining the sound pickup direction to be the direction of the wearer; obtaining the ambient noise intensity; determining the target microphone according to the ambient noise intensity, the target microphone including a first microphone, the first microphone being the microphone that is closest to the wearer in the distribution position of the microphone array.

[0007] Optionally, determining the target microphone according to the ambient noise intensity includes: in response to the ambient noise intensity exceeding a noise intensity threshold, determining that the target microphone also includes a second microphone in the microphone array that is farthest from the first microphone.

[0008] Optionally, determining the sound pickup direction and the target microphone according to an application scenario of the smart glasses includes: in response to the target sound source of the application scenario being a non-wearer, determining the sound pickup direction as a non-wearer direction; obtaining the possibility of wearer voice input in the application scenario; determining the target microphone according to the possibility of wearer voice input in the application scenario, the target microphone including at least one third microphone, the third microphone being the microphone closest to the non-wearer in the non-wearer direction.

[0009] Optionally, determining the target microphone according to the possibility of voice input by the wearer in the application scenario includes: in response to the possibility of no voice input by the wearer in the application scenario, determining that the target microphone also includes a first microphone, wherein the first microphone is the microphone that is closest to the wearer in the distribution position of the microphone array.

[0010] Optionally, before determining the pickup direction and the target microphone according to the application scenario of the smart glasses, it also includes: initializing the microphones in the microphone array using a preconfigured gain coefficient so that the sound signals of the initialization sound source collected by the microphones in the microphone array have the same amplitude, and the initialization sound source is a non-wearer sound source or a wearer sound source.

[0011] Optionally, the initialization sound source is the wearer sound source. After determining the pickup direction and the target microphone according to the application scenario of the smart glasses, it also includes: in response to the target sound source of the application scenario including a non-wearer sound source, adjusting the gain of the microphone in the target microphone for collecting the non-wearer sound source so that the sound signals of the non-wearer sound source collected by the microphone for collecting the non-wearer sound source have the same amplitude.

[0012] Optionally, the target microphone is adjusted to point to the sound pickup direction, including: obtaining the phase difference of different target microphones in the sound pickup direction; adjusting the delay of the sound signal collected by the target microphone according to the phase difference, so that the sound signals collected by different target microphones have the same phase in the sound pickup direction.

[0013] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a processor, and a memory for storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method in any of the above embodiments.

[0014] According to another aspect of the embodiments of the present application, a non-transitory machine-readable medium storing computer instructions is also provided, where the computer instructions are used to enable a computer to execute the method in any of the above embodiments.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which, when executed by a processor of a computer, causes the computer to execute the method in any of the above embodiments.

[0016] In an embodiment of the present application, the sound pickup direction and the target microphone are determined according to the application scenario of the smart glasses. The microphone array of the smart glasses includes a plurality of microphones with different distribution positions and adjustable directions. The target microphone is a microphone in the microphone array. The sound signal in the application scenario is collected by the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction. In other words, the embodiment of the present application can dynamically adjust the direction of the target microphone in the microphone array according to the application scenario of the smart glasses, so as to collect the sound signal in the corresponding application scenario through the target microphone, instead of setting the direction of the microphone to a fixed state as in the related art, which results in the lack of flexibility of the microphone on the smart glasses in multi-scenario applications, thereby achieving the technical effect of improving the flexibility of the microphone on the smart glasses.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0019] Figure 1 is a flow chart of a method for controlling a microphone on smart glasses according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of the distribution positions of microphones in smart glasses according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a microphone frequency response curve according to an embodiment of the present application;

[0022] Figure 4 is a structural block diagram of a control device for a microphone on smart glasses according to an embodiment of the present application;

[0023] Figure 5 Schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0024] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0025] A microphone array composed of multiple microphones can enhance audio in a specific direction. However, in related technologies, microphone arrays are usually pre-designed, that is, the direction of the microphone is fixed. For example, two microphones form a microphone array, where one microphone collects the sound signal of the wearer of the smart glasses and one microphone collects the sound signal of the non-wearer of the smart glasses. However, in the application scenario of voice assistant, only the sound signal of the wearer of the smart glasses needs to be collected. In application scenarios such as conversation translation and meeting recording, the sound signals of the wearer of the smart glasses and the non-wearer of the smart glasses need to be collected at the same time, and the sound signal of the non-wearer of the smart glasses is more important. Since the non-wearer is farther away from the microphone than the wearer, the signal-to-noise ratio (that is, the relative intensity difference between the target sound signal and the background noise) is lower. Therefore, the above-mentioned fixed microphone direction obviously lacks flexibility in multi-scenario applications.

[0026] In order to solve the above technical problems, the present application provides relevant solutions in the embodiments, which are described in detail below. Figure 1 The flowchart of the control method of the microphone on the smart glasses shown in the figure comprises the following steps:

[0027] Step S102, determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, the microphone array of the smart glasses includes a plurality of microphones with different distribution positions and adjustable directions, and the target microphone is a microphone in the microphone array.

[0028] The application scenarios of smart glasses include but are not limited to: voice assistant, simultaneous interpretation, conversational translation, lecture mode, hearing-impaired mode, etc. Among them, the voice assistant generally only speaks to the wearer. Simultaneous interpretation is generally a face-to-face conversation between the wearer and the non-wearer. The speech content is recognized as text, and the recognized text is recorded, but the speaker is not distinguished. Conversational translation is generally a face-to-face conversation between the wearer and the non-wearer. The speech content is recognized as text, and the recognized text is recorded to distinguish the speaker. The lecture mode is generally that the wearer acts as an audience, listens to the teacher's lecture or speech, and translates the speech content into the desired language and displays it on the screen. The hearing-impaired mode is generally for people with hearing impairments, and they need to recognize the other party's speech content and display it on the glasses screen. In the embodiments of the present application, the above application scenarios correspond to the applications that the smart glasses are running.

[0029] The above-mentioned sound pickup direction can be determined according to the target sound source of the smart glasses application scenario. In the application scenario of voice assistant, the target sound source is the wearer, and the sound pickup direction is determined to be the wearer's direction; in the application scenario of simultaneous interpretation or dialogue translation, the target sound source is the wearer and the non-wearer, and the sound pickup direction is determined to be the wearer's direction and the non-wearer's direction. In the application scenarios of lecture mode and hearing impairment mode, the target sound source is the non-wearer, and the sound pickup direction is determined to be the non-wearer's direction.

[0030] Exemplarily, the wearer direction and the non-wearer direction can be two directions preset by the smart glasses, wherein the wearer direction can represent the direction of an ordinary wearer relative to the smart glasses when wearing the smart glasses, and the non-wearer direction can mark the direction of a non-wearer whose voice usually needs to be recorded relative to the smart glasses. Generally speaking, the non-wearer direction is directly in front of the glasses.

[0031] It is understandable that the smart glasses can also determine the direction of the wearer and the non-wearer through the sound source localization algorithm, which will not be described in detail.

[0032] The microphones in the above-mentioned microphone array are usually placed in the frame of the glasses, the temples of the glasses, etc. The position design of these microphones takes into account the user's wearing habits, the direction of the sound source, and the interference of external environmental noise. Microphones in different positions can collect sound signals from multiple directions. The adjustable microphone directionality enables each microphone to detect and focus on sound signals in different directions. For example, when a user uses smart glasses in a noisy environment, the directivity of the microphone array can be dynamically adjusted to reduce the interference of background noise and enhance the clarity of the target sound.

[0033] Optionally, the target microphone may be one or more microphones in a microphone array, which may be specifically determined according to an application scenario of the smart glasses.

[0034] Step S104: collecting the sound signal in the application scenario through the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction.

[0035] In one possible implementation, the target microphone on the smart glasses can be rotated by a mechanical device so that the target microphone points in the sound pickup direction, or a guiding reflective surface (such as an acoustic mirror) can be used to guide the sound waves into the target microphone so that the target microphone points in the sound pickup direction.

[0036] In another possible implementation, the virtual direction of the microphone can be adjusted using signal processing technology. Optional adjustment methods include: S11, obtaining the phase difference between different target microphones in the pickup direction; S12, adjusting the delay of the sound signal collected by the target microphone according to the phase difference, so that the sound signals collected by different target microphones have the same phase in the pickup direction. Specifically, in S11, the distance d between different target microphones can be obtained. When the sound wave approaches different target microphones from a specific angle θ, the time difference Δt can be calculated based on the path difference of the sound wave from the angle θ to the different target microphones, and then combined with the frequency f of the sound wave, the phase difference Δφ=2πfΔt is calculated. In S12, the time difference Δt between the reference microphone r (one of multiple target microphones can be set) and each other target microphone i can be calculated. i,r , as shown in Formula 1, where v is the speed of sound waves in air, which is approximately 343 m / s:

[0037]

[0038] Then the collected signal S of the i-th target microphone i (t) applies a delay Δt i,r , as shown in Formula 2:

[0039] s′ i (r) = s i (r-Δt i,r ) (Formula 2)

[0040] That is, after obtaining the phase difference, the phase difference is converted into a time delay, and then the time delay is added to the sound signal collected by the target microphone. This can make the signals of different microphones in the pickup direction reach the same phase, thereby achieving phase alignment of the signal in that direction, enhancing the sound in a specific direction while reducing interference in other directions, thereby achieving dynamic adjustment of the direction of the microphone.

[0041] Exemplarily, assume that there are three target microphones M1, M2 and M3, the distance between the three target microphones is the same and equal to 0.1 meter, θ = 30°, the frequency f is 1kHz, and M1 is set as the reference microphone. Then the time delay differences between the three target microphones M1, M2 and M3 and the reference microphone are 0, 146μs, and 292μs respectively, and the phase differences Δφ2,1≈0.916 radians and Δφ2,1≈1.832 radians. Finally, the signals of microphones M2 and M3 are adjusted to s′2(t)=s2(t-146μs) and s′3(t)=s3(t-292μs), respectively.

[0042] Through the above steps S102 to S104, the sound pickup direction and the target microphone are determined according to the application scenario of the smart glasses. The microphone array of the smart glasses includes multiple microphones with different distribution positions and adjustable directions. The target microphone is a microphone in the microphone array; the sound signal in the application scenario is collected by the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction. In other words, the embodiment of the present application can dynamically adjust the direction of the target microphone in the microphone array according to the application scenario of the smart glasses, so as to collect the sound signal in the corresponding application scenario through the target microphone, instead of setting the direction of the microphone to a fixed state in the related art, which leads to the lack of flexibility of the microphone on the smart glasses in multi-scenario applications, thereby achieving the technical effect of improving the flexibility of the microphone on the smart glasses.

[0043] In a possible implementation, the sound pickup direction and the target microphone are determined according to the application scenario of the smart glasses, including: S11, in response to the target sound source of the application scenario being the wearer, the sound pickup direction is determined to be the direction of the wearer; S12, the ambient noise intensity is obtained; S13, according to the ambient noise intensity, the target microphone is determined, the target microphone includes a first microphone, and the first microphone is the microphone closest to the wearer in the distribution position of the microphone array. In order to effectively enhance the sound signal of the collected target sound source, suppress background noise, and thus improve the performance of speech recognition, the embodiment of the present application also proposes that the above S13 may include: S131, in response to the ambient noise intensity exceeding the noise intensity threshold, it is determined that the target microphone also includes the second microphone farthest from the first microphone in the microphone array. That is, when the ambient noise intensity exceeds the noise intensity threshold, the first microphone closest to the wearer and the second microphone farthest from the first microphone in the distribution position of the microphone array are determined as target microphones.

[0044] When the target sound source is the wearer, the above-mentioned environmental noise may be non-wearer sounds, traffic noise, natural noise, etc. When the intensity of the environmental noise is low (for example, a quiet environment), the microphone closest to the wearer in the microphone array may be determined as the target microphone. When the intensity of the environmental noise is high (for example, playing music), in order to enhance the wearer's voice, the first microphone closest to the wearer in the microphone array may be combined with other microphones in the microphone array to be determined as the target microphone. The microphone combination may include one or more, which are specifically determined according to the signal-to-noise ratio requirements. For example, Figure 2As shown, microphone 1 is close to the wearer, microphone 2 and microphone 3 are facing the front of the wearer, microphone 3 is farthest from microphone 1, and microphone 2 is close to microphone 1. The application scenario of smart glasses is voice assistant. Then, when the ambient noise intensity is low, only microphone 1 can be determined as the target microphone. When the ambient noise is high, microphone 1 and microphone 3 can be determined as target microphones. Compared with determining microphone 1 and microphone 2 as target microphones, determining microphone 1 and microphone 3 as target microphones can obtain better beamforming effect, so the sound pickup effect is better. When the ambient noise is very high, microphone 1, microphone 2, and microphone 3 can all be determined as target microphones to maximize the sound pickup effect of the wearer.

[0045] Optionally, in an embodiment of the present application, while ensuring the quality of the sound signal, a smaller number of target microphones can be determined to further save power of the smart glasses.

[0046] The sound pickup direction includes at least the wearer direction and the non-wearer direction. The above description is that the sound pickup direction is the wearer direction. The embodiment of the present application also proposes a method for determining the target microphone when the sound pickup direction is the non-wearer direction. Optionally, the sound pickup direction and the target microphone are determined according to the application scenario of the smart glasses, including:

[0047] S21, in response to the target sound source of the application scenario being a non-wearer, determining the sound pickup direction to be a non-wearer direction.

[0048] S22, obtaining the possibility of voice input by the wearer in the application scenario.

[0049] In the embodiments of the present application, in some application scenarios, the wearer needs to speak, and in some application scenarios, the wearer does not need to speak, but only needs to listen to the non-wearer speaking. Therefore, the wearer's voice input is possible in different application scenarios. For example, in the application scenario of lecture mode, the sound pickup direction is the non-wearer direction, but it is possible that the wearer will ask questions to the non-wearer, so the wearer may have voice input. In the application scenario of hearing impairment mode and the wearer of the smart glasses cannot speak, the sound pickup direction is the non-wearer direction, and the wearer has no possibility of voice input.

[0050] S23, determining the target microphone according to the possibility of voice input by the wearer in the application scenario, where the target microphone includes at least one third microphone, and the third microphone is the microphone closest to the non-wearer in the direction of the non-wearer.

[0051] Optionally, in the embodiment of the present application, in the case where the non-wearer is assumed to be in front of the wearer, the third microphone is a microphone with a pre-designed sound pickup direction facing the front, that is, a microphone with a sound pickup direction facing the non-wearer direction. Figure 2As shown, three microphones are included. When it is determined that a non-wearer is directly in front of the smart glasses wearer, microphone 3 can be determined as the third microphone.

[0052] In the embodiment of the present application, when determining that the sound pickup direction is the non-wearer direction, it is also necessary to flexibly determine the target microphone according to whether the wearer has the possibility of voice input in the application scenario. For example, in the application scenario of simultaneous interpretation or dialogue translation, the wearer has the possibility of voice input, and the target microphone includes the microphone closest to the non-wearer in the non-wearer direction, and the target microphone does not include the microphone closest to the wearer. In other words, the sound pickup direction of the microphone closest to the wearer is still the wearer, so that the wearer's voice can be collected when the wearer speaks.

[0053] In one possible implementation, the target microphone is determined based on the possibility of voice input by the wearer in the application scenario, and also includes: S31, in response to the possibility of no voice input by the wearer in the application scenario, determining that the target microphone also includes a first microphone, and the first microphone is the microphone that is closest to the wearer in the distribution position in the microphone array.

[0054] Optionally, when the sound pickup direction is in the direction of the non-wearer and the wearer has no possibility of voice input, the microphone in the microphone array that is closest to the wearer and the microphone that is closest to the non-wearer in the non-wearer direction can be determined as the target microphone. Of course, in order to maximize the sound pickup effect for the non-wearer, other microphones in the microphone array can also be determined as target microphones. Exemplarily, in the application scenario of the speech impairment mode, the wearer has no possibility of voice input, and the target microphones include not only the microphone closest to the non-wearer in the non-wearer direction, but also the microphone closest to the wearer. In other words, the sound pickup direction of the microphone closest to the wearer is adjusted from the wearer direction to the non-wearer direction, thereby improving the sound pickup effect for the non-wearer. Figure 2 As shown, assuming that in the application scenario of the speech impairment mode, microphone 3 is closest to the non-wearer, then microphone 3 and microphone 1 can be determined as target microphones. In order to maximize the sound pickup effect for the non-wearer, microphone 1, microphone 2, and microphone 3 can all be determined as target microphones.

[0055] Different microphones have different characteristics. There are analog microphones, digital microphones, microphones from different manufacturers, or different models from the same manufacturer, and even microphones from different batches of the same model from the same manufacturer may have inconsistent characteristics. In addition, exactly the same microphones are placed in different positions on the smart glasses. For example, they are sunk into the frame at different depths, positions, and directions, that is, different sound cavities, which will also result in different collected sound signals. In addition, due to problems with the design of intelligent hardware, such as the selection and iteration of new and old product components, the types of microphones on smart glasses are different. The microphone array composed of such heterogeneous microphones has a poor array algorithm effect. In this regard, the embodiment of the present application proposes that before determining the pickup direction and the target microphone according to the application scenario of the smart glasses, it also includes:

[0056] S41, initializing microphones in the microphone array using preconfigured gain coefficients so that sound signals of an initialization sound source collected by the microphones in the microphone array have the same amplitude, and the initialization sound source is a non-wearer sound source or a wearer sound source.

[0057] Optionally, during the initialization phase of the smart glasses, the preconfigured gain coefficient can be configured in the microphone circuit so that the sound signals of the initialization sound source collected by the microphones in the microphone array have the same amplitude. The microphone circuit generally includes a sound pressure sensor, an amplifier, and an analog-to-digital converter (ADC), and the gain coefficient can be set in the amplifier. In addition, when preconfiguring the gain coefficient, the microphone acoustic overload point (AOP) needs to be considered, that is, the maximum sound level that the microphone can accept.

[0058] In the above step S41, one or more initial gains can be preset to the microphones in the microphone array from the hardware, so as to achieve the same or similar amplitude values ​​of the sound signals of the initialization sound source collected by multiple microphones of the same sound.

[0059] In the present application, the initialization sound source can be one of the wearer sound source and the non-wearer sound source. After initialization, the sound signals collected by each microphone in the direction of the initialization sound source can have the same amplitude value. However, the microphone in the present application may collect sound from the wearer direction as well as sound from the non-wearer direction. When collecting sound from the non-initialization sound source direction, the amplitude values ​​between different microphones may differ, which is not conducive to signal enhancement. Therefore, when collecting sound from the non-initialization sound source direction, the gain of the microphone needs to be further adjusted (sensitivity adjustment).

[0060] Optionally, taking the initialization sound source as the wearer's sound source as an example, after determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, the method further includes:

[0061] S51, in response to the target sound source of the application scenario including a non-wearer sound source, adjusting the gain of the microphone for collecting the non-wearer sound source in the target microphone so that the sound signals of the non-wearer sound source collected by the microphone for collecting the non-wearer sound source have the same amplitude.

[0062] Optionally, when the target sound source switches from the wearer to the non-wearer, the above-mentioned initial gain also needs to be adjusted accordingly, that is, a corresponding compensation operation needs to be performed on the above-mentioned initial gain. Optionally, the updated AOP can be determined based on the microphone's own AOP, and then the microphone gain is adjusted accordingly according to the updated AOP. Among them, determining the updated AOP includes: taking the minimum value of the target microphone AOP to ensure that the target microphones can operate within the determined range. The corresponding adjustment of the microphone gain according to the updated AOP can be similar to an equalizer in a manner so that the sound signals of the non-wearer sound sources collected by the microphone for collecting non-wearer sound sources have the same amplitude.

[0063] Exemplarily, assume that there are two microphones. The two microphones initially collect the wearer's sound signals, and now switch to collecting non-wearer sound signals. Through the above-mentioned preconfigured gain coefficient, under the maximum possible speech (empirical value), the sound signal amplitude value output by microphone 1 is 10, and the sound signal amplitude value output by microphone 2 is 5. The sound signal amplitude values ​​output by microphone 1 and microphone 2 are different. Considering that the AOP of microphone 1 itself is 8 and the AOP of microphone 2 itself is 10, the updated AOP is determined to be the minimum value between the two, that is, the latest AOP value is 8, then the gain of microphone 1 can be reduced, and then the gain of microphone 2 can be increased, so that the sound signal amplitude values ​​output by microphone 1 and microphone 2 for the maximum possible speech do not exceed the AOP, and the sound signals of the non-wearer sound sources collected have the same amplitude.

[0064] Optionally, in the embodiment of the present application, there is a certain relationship between the gain and sensitivity of the microphone, wherein the sensitivity calculation formula of the analog microphone is as follows:

[0065]

[0066] Among them, V out is the output voltage (related to gain) of the microphone at a specific sound pressure level (e.g. 1Pa or 94dB SPL), V ref is the parameter voltage, usually 1V.

[0067] The sensitivity of a digital microphone is calculated as follows:

[0068]

[0069] Among them, A out A is the digital output amplitude (related to gain) of the microphone at a specific sound pressure level (usually 1Pa) and a specific frequency. FS is the full-scale output of the digital microphone. For example, the full-scale output value of a 16-bit microphone may be 32767. Therefore, after adjusting the gain of the microphone, the sensitivity of the microphone will also change, that is, there is a considerable relationship between adjusting the gain of the microphone and adjusting the sensitivity of the microphone.

[0070] After the gain adjustment is performed in the time domain, the gain adjustment is also required in the frequency domain. The embodiment of the present application realizes the adjustment of each frequency point in the frequency domain by adjusting the frequency response curve. Optionally, a bandpass filter for the entire frequency point can be used, and a scaling factor can be added for adjustment. For example, some microphones are more sensitive to high frequencies of 3000Hz, and the collected sound signal value is relatively large, so a peak will be found on the frequency response curve. At this time, this peak must be suppressed. A bandpass filter for the entire frequency point can be used, and a scaling factor can be added to adjust the response of this frequency point. If there are multiple such peaks and troughs, multiple such filters can be used, and multiple filters can be cascaded, similar to an equalizer, and finally the desired frequency response curve is obtained, such as Figure 3 As shown, the adjusted frequency response curve sound signal is in the frequency range of 20Hz to 2000Hz, the size of the received signal is almost the same, and the curve is very flat. It can be understood that the embodiment of the present application can also adjust the frequency response curves of different target microphones so that different target microphones have the same or similar frequency response curves.

[0071] In the above step S51, the above initial gain can be adjusted at the software level, which shortens the adjustment time and reduces the adjustment cost compared to adjusting the microphone gain at the hardware level.

[0072] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a control device for a microphone on smart glasses. Figure 4 The figure shows a structural block diagram of a control device for a microphone on smart glasses according to an embodiment of the present application, including:

[0073] A determination module 42 determines a sound pickup direction and a target microphone according to an application scenario of the smart glasses, wherein the microphone array of the smart glasses includes a plurality of microphones with different distribution positions and adjustable directions, and the target microphone is a microphone in the microphone array;

[0074] The collection module 44 collects the sound signal in the application scenario through the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction.

[0075] In one possible implementation, the determination module 42 includes: a first determination unit, used to determine the pickup direction as the wearer's direction in response to the target sound source of the application scenario being the wearer; a first acquisition unit, used to acquire the ambient noise intensity; and a second determination unit, used to determine the target microphone according to the ambient noise intensity, the target microphone including a first microphone, which is the microphone that is closest to the wearer in the distribution position in the microphone array.

[0076] The second determination unit is further configured to determine, in response to the ambient noise intensity exceeding a noise intensity threshold, that the target microphone further includes a second microphone in the microphone array that is farthest from the first microphone.

[0077] The determination module 42 also includes: a third determination unit, which is used to determine the pickup direction as the non-wearer direction in response to the target sound source of the application scenario being a non-wearer; a second acquisition unit, which is used to acquire the possibility of wearer voice input in the application scenario; and a fourth determination unit, which is used to determine the target microphone according to the possibility of wearer voice input in the application scenario, and the target microphone includes at least one third microphone, which is the microphone closest to the non-wearer in the non-wearer direction.

[0078] The fourth determination unit is further configured to determine, in response to the wearer having no possibility of voice input in the application scenario, that the target microphone further includes a first microphone, where the first microphone is the microphone that is closest to the wearer in the distribution position in the microphone array.

[0079] Optionally, the above-mentioned device also includes: a processing module, which is used to initialize the microphones in the microphone array using a preconfigured gain coefficient before determining the pickup direction and the target microphone according to the application scenario of the smart glasses, so that the sound signals of the initialized sound source collected by the microphones in the microphone array have the same amplitude, and the initialized sound source is a non-wearer sound source or a wearer sound source.

[0080] The above-mentioned device also includes: a second adjustment module, which is used to initialize the sound source as the wearer's sound source, determine the pickup direction and the target microphone according to the application scenario of the smart glasses, and in response to the target sound source of the application scenario including the non-wearer sound source, adjust the gain of the microphone in the target microphone for collecting the non-wearer sound source so that the sound signals of the non-wearer sound source collected by the microphone for collecting the non-wearer sound source have the same amplitude.

[0081] The above device is also used to obtain the phase difference of different target microphones in the sound pickup direction; adjust the delay of the sound signal collected by the target microphone according to the phase difference, so that the sound signals collected by different target microphones have the same phase in the sound pickup direction.

[0082] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0083] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the embodiment of the present application when executed by the at least one processor.

[0084] In a possible implementation, the electronic device may be a smart glasses. The smart glasses are provided with a microphone array, which includes a plurality of microphones distributed at different positions and pointing to the subject.

[0085] An embodiment of the present application also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0086] The embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0087] refer to Figure 5 , the structural block diagram of the electronic device that can be used as the server or client of the embodiment of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0088] like Figure 5As shown, the electronic device includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 505. An input / output (I / O) interface 505 is also connected to the bus 505.

[0089] Multiple components in the electronic device are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 505. The input unit 506 can be any type of device that can input information to the electronic device, and the input unit 506 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 507 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include but is not limited to a disk, an optical disk. The communication unit 505 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0090] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present application may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 502 and / or a communication unit 505. In some embodiments, the computing unit 501 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0091] The computer program for implementing the method of the embodiment of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of the present application embodiment, machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable signal medium can include but is not limited to electronic, magnetic, optical, electromagnetic or infrared systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0093] It should be noted that the term "including" and its variations used in the embodiments of the present application are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0094] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0095] The various steps described in the method implementation methods provided in the embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present application is not limited in this respect.

[0096] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0097] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for controlling a microphone on smart glasses, comprising: Determine the sound pickup direction and the target microphone according to the application scenario of the smart glasses, wherein the microphone array of the smart glasses includes a plurality of microphones with different distribution positions and adjustable directions, and the target microphone is a microphone in the microphone array; The sound signal in the application scenario is collected by the target microphone, wherein the target microphone is adjusted to point to the sound pickup direction.

2. The method according to claim 1, determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, comprising: In response to the target sound source of the application scenario being the wearer, determining the sound pickup direction to be the direction of the wearer; Get the ambient noise intensity; The target microphone is determined according to the ambient noise intensity, where the target microphone includes a first microphone, and the first microphone is a microphone that is closest to the wearer in the microphone array.

3. The method according to claim 2, determining the target microphone according to the ambient noise intensity, comprising: In response to the environmental noise intensity exceeding a noise intensity threshold, determining that the target microphone further includes a second microphone in the microphone array that is farthest from the first microphone.

4. The method according to claim 1, determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, comprising: In response to the target sound source of the application scenario being a non-wearer, determining the sound pickup direction to be a non-wearer direction; Obtaining the possibility of voice input by the wearer in the application scenario; The target microphone is determined according to the possibility of voice input by the wearer in the application scenario, and the target microphone includes at least one third microphone, and the third microphone is the microphone closest to the non-wearer in the direction of the non-wearer.

5. The method according to claim 4, determining the target microphone according to the possibility of voice input by the wearer in the application scenario, comprises: In response to the wearer having no possibility of voice input in the application scenario, it is determined that the target microphone further includes a first microphone, where the first microphone is a microphone that is closest to the wearer in a distribution position in the microphone array.

6. The method according to any one of claims 1 to 5, before determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, further comprising: The microphones in the microphone array are initialized using preconfigured gain coefficients so that the sound signals of the initialization sound sources collected by the microphones in the microphone array have the same amplitude, and the initialization sound source is a non-wearer sound source or a wearer sound source.

7. The method according to claim 6, wherein the initialization sound source is the wearer's sound source, and after determining the sound pickup direction and the target microphone according to the application scenario of the smart glasses, the method further comprises: In response to the target sound source of the application scenario including a non-wearer sound source, the gain of the microphone for collecting the non-wearer sound source in the target microphone is adjusted so that the sound signals of the non-wearer sound source collected by the microphone for collecting the non-wearer sound source have the same amplitude.

8. The method according to claim 1, wherein the target microphone is adjusted to point toward the sound pickup direction, comprising: Acquire phase differences of different target microphones in the sound pickup direction; The delay of the sound signal collected by the target microphone is adjusted according to the phase difference, so that the sound signals collected by different target microphones have the same phase in the sound pickup direction.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images