Audio acquisition method, electronic equipment and storage medium

Abstract

The invention provides an audio acquisition method, electronic equipment and a storage medium. The first electronic equipment comprises a microphone array, the microphone array comprises at least two microphones, and the method comprises the steps that the first electronic equipment collects a first audio signal output by a target sounding object through the microphone array; the first electronic equipment determines a first distance between the first electronic equipment and the target sounding object based on the first audio signal; when the first distance is smaller than a first preset distance, the first electronic equipment obtains a second audio signal output by the target sounding object through the microphone array; the first electronic device saves the second audio signal within the first application. The electronic equipment can automatically pick up the audio, user operation is reduced, the audio collection efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an audio acquisition method, an electronic device and a storage medium. Background Art

[0002] With the development of terminal technology, electronic devices can support the functions of audio output and audio collection. For example, electronic devices can play audio through speakers (speakers), and electronic devices can also collect audio through microphones (mics).

[0003] Electronic devices can save the audio collected by the microphone locally, and electronic devices can also send the audio collected by the microphone to other devices. However, users are required to find the audio collection entrance in the application that supports the voice recording function and control the electronic device to collect audio, which is cumbersome for users to operate. Alternatively, users can collect audio by using voice to wake up the application that supports the voice recording function on the electronic device, but voice wake-up is prone to accidental touches. How to provide a convenient, fast and accurate method for collecting audio needs further study. Summary of the invention

[0004] The present application provides an audio acquisition method, an electronic device, and a storage medium, which improve the audio acquisition efficiency, reduce user operations, and enhance the user experience.

[0005] In a first aspect, the present application provides an audio collection method, wherein a first electronic device includes a microphone array, the microphone array includes at least two microphones, and the method includes: the first electronic device collects a first audio signal output by a target sound-emitting object through the microphone array; the first electronic device determines a first distance between the first electronic device and the target sound-emitting object based on the first audio signal; when the first distance is less than a first preset distance, the first electronic device obtains a second audio signal output by the target sound-emitting object through the microphone array; the first electronic device saves the second audio signal in a first application.

[0006] Optionally, the electronic device may store the first audio signal or may not store the first audio signal.

[0007] Optionally, the first electronic device stores the second audio signal in the first application. The first electronic device may store the second audio signal in the first application directly, or convert the second audio signal into text information and then store the text information in the first application.

[0008] The electronic device can determine the first distance between the electronic device and the target sound-emitting object based on the collected audio signal. When the first distance between the electronic device and the target sound-emitting object is greater than the first preset distance, or the first distance between the electronic device and the target sound-emitting object is greater than the first preset distance for a certain period of time, the electronic device can determine that near-field speech is recognized. The electronic device can automatically start collecting audio and automatically save the audio in the first application. The electronic device automatically picks up audio, reduces user operations, improves audio collection efficiency, and enhances user experience.

[0009] In combination with the first aspect, in a possible implementation manner, the first electronic device saves the second audio signal in the first application, specifically including:

[0010] In response to the first distance being less than the first preset distance, the first electronic device starts the first application and saves the second audio signal in the first application, or the first electronic device converts the second audio signal into a first text message and saves the first text message in the first application.

[0011] In this way, after recognizing the near-field voice, the electronic device can automatically save the collected audio signal or the text information corresponding to the collected audio signal in the first application.

[0012] In combination with the first aspect, in a possible implementation method, the first electronic device saves the second audio signal in the first application, specifically including: in response to the first distance being less than the first preset distance, the first electronic device opens the first application, and sends the second audio signal to the second electronic device through the first application, or the first electronic device converts the second audio signal into a first text message, and sends the first text message to the second electronic device through the first application.

[0013] In this way, after recognizing the near-field voice, the electronic device can automatically send the collected audio signal to the second electronic device with which the communication connection is established, or send the text information corresponding to the collected audio signal to the second electronic device with which the communication connection is established.

[0014] In combination with the first aspect, in a possible implementation, a first electronic device is connected to a third electronic device via Bluetooth; the first electronic device saves the second audio signal in a first application, specifically including: the first electronic device sends the second audio signal to the third electronic device via the Bluetooth connection; the third electronic device saves the second audio signal in the first application.

[0015] Optionally, the first electronic device may be a Bluetooth headset. After recognizing the near-field voice, the Bluetooth headset may automatically send the collected audio signal to a third electronic device that has established a Bluetooth connection, or send text information corresponding to the collected audio signal to the third electronic device that has established a Bluetooth connection.

[0016] In combination with the first aspect, in a possible implementation, while the first electronic device obtains the second audio signal through the microphone array, the first electronic device also obtains the third audio signal output by other sound-emitting objects through the microphone array; the electronic device saves the second audio signal in the first application, specifically including: when the first electronic device determines that the distance between the first electronic device and the other sound-emitting objects exceeds a first preset distance, the electronic device saves the second audio signal in the first application.

[0017] Optionally, the second audio signal and the third audio signal may be audio signals in the same audio file. The electronic device 100 may extract the second audio signal or the third audio signal from the same audio file.

[0018] Optionally, the second audio signal and the third audio signal may also be audio signals in two audio files respectively.

[0019] Near-field voice may refer to the audio outputted by a target sound-emitting object within a first preset distance from the electronic device. Far-field voice may refer to the audio outputted by a target sound-emitting object beyond the first preset distance from the electronic device.

[0020] In this way, after recognizing near-field speech and starting to collect audio signals, the electronic device can determine whether the collected audio signal is a near-field audio signal or a far-field audio signal, and only save the near-field audio signal without saving the far-field audio signal, thereby avoiding interference from the far-field audio signal.

[0021] In combination with the first aspect, in a possible implementation method, when the first distance is less than a first preset distance, the first electronic device obtains a second audio signal output by the target sound-emitting object through a microphone array, specifically including: the first electronic device determines the probability of a voice signal contained in the first audio signal; when the first distance is less than the first preset distance, and when the probability of the voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0022] In this way, before starting to collect audio, the electronic device can determine the probability of the voice signal contained in the audio signal. When the probability of the voice signal contained in the audio signal is greater than the first threshold, that is, the probability of recognizing that the user is speaking is high, the electronic device can collect and save the second audio signal.

[0023] In combination with the first aspect, in a possible implementation, a first electronic device is connected to a third electronic device via Bluetooth; the first electronic device obtains a second audio signal output by a target sound-emitting object through a microphone array, specifically including: when the first electronic device determines that the first electronic device is in a handheld state, the first distance is less than a first preset distance, and the probability of a voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0024] Optionally, a sensor is pre-installed on the first electronic device, and whether it is in a handheld state can be confirmed through a sensor signal collected by the sensor.

[0025] In this way, when the first electronic device is a Bluetooth headset, when the Bluetooth headset is in a handheld state, it can be considered that the current user has the intention to speak to the Bluetooth headset. When the Bluetooth headset is in a handheld state, the distance between the Bluetooth headset and the target sound-emitting object is within a first preset distance, and the probability of the voice signal contained in the first audio signal is greater than a first threshold, the Bluetooth headset can enter the near-field voice mode and start to automatically collect and save the audio signal, which can improve the accuracy of the Bluetooth headset entering the near-field voice mode.

[0026] In combination with the first aspect, in a possible implementation, the first electronic device includes a speaker; before the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array, the method also includes: the first electronic device emits a first ultrasonic signal through the speaker; the first electronic device receives the reflected second ultrasonic signal; the first electronic device determines the second distance between the first electronic device and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the first ultrasonic signal and the second ultrasonic signal; when the second distance is less than the second preset distance and the vibration frequency of the target sound-emitting part of the target sound-emitting object is within the first range, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0027] Exemplarily, the first range may be between 20 Hz and 40 Hz.

[0028] Optionally, the second preset distance may be smaller than the first preset distance. Optionally, the second preset distance may also be equal to the first preset distance.

[0029] Optionally, the target sound-emitting object may emit sound at a certain vibration frequency when emitting sound. The first electronic device may obtain the vibration frequency of the target sound-emitting part of the target sound-emitting object through the first ultrasonic signal and the second ultrasonic signal, and determine whether the target sound-emitting object is emitting sound based on the obtained vibration frequency of the target sound-emitting object. When the vibration frequency of the target sound-emitting object is within the first range, it can be considered that the target sound-emitting object is emitting sound.

[0030] In this way, when the first electronic device enters the near-field voice mode, it can be confirmed whether the first electronic device and the target sound-emitting object are close enough and whether the target sound-emitting part of the target sound-emitting object is emitting sound, thereby improving the accuracy of the first electronic device entering the near-field voice mode.

[0031] In combination with the first aspect, in a possible implementation, after the first electronic device saves the second audio signal in the first application, the method also includes: when it is determined based on the first ultrasonic signal and the second ultrasonic signal for a continuous first preset time period that the second distance is greater than the second preset distance and / or the vibration frequency of the target sound-emitting part of the target sound-emitting object is not within the first range, the first electronic device stops acquiring the audio signal through the microphone array.

[0032] After the first electronic device enters the near-field voice mode, the first electronic device needs to continuously monitor whether the first electronic device and the target sound-emitting object are close enough, and whether the target sound-emitting part of the target sound-emitting object is making a sound. When it is monitored that the distance between the first electronic device and the target sound-emitting object exceeds the second preset distance, and / or the target sound-emitting part of the target sound-emitting object is not making a sound, the first electronic device can automatically exit the near-field voice mode and stop collecting audio.

[0033] In combination with the first aspect, in a possible implementation, the microphone array includes a first microphone and a second microphone, and the positions of the first microphone and the second microphone on the first electronic device are different; the first electronic device collects a first audio signal output by a target sound-emitting object through the microphone array, specifically including: the first electronic device collects the first audio signal output by the target sound-emitting object through the first microphone and the second microphone respectively; the first electronic device determines a first distance between the first electronic device and the target sound-emitting object based on the first audio signal, specifically including: the first electronic device obtains a first energy value of the first audio signal collected by the first microphone and a second energy value of the first audio signal collected by the second microphone; the first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first energy value and the second energy value; and / or, the first electronic device obtains a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; the first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first moment and the second moment.

[0034] In this way, the electronic device can determine the distance between the first electronic device and the target sound-emitting object through the difference in energy values ​​of the same audio signal collected by multiple microphones in the microphone array and / or the difference between the moments of receiving the same audio signal.

[0035] In combination with the first aspect, in a possible implementation, the microphone array includes a first microphone and a second microphone, the first microphone and the second microphone are located at different positions on the first electronic device, the speaker includes a first speaker and a second speaker, the first speaker is located near the first microphone, and the second speaker is located near the second microphone; the first electronic device sends a first ultrasonic signal through the speaker, specifically including: when the first electronic device determines that the distance between the first microphone and the target sound-emitting object is less than the distance between the second microphone and the target sound-emitting object, the first electronic device sends the first ultrasonic signal through the first speaker, wherein the first speaker is the speaker closest to the target sound-emitting object.

[0036] In combination with the first aspect, in a possible implementation manner, the first electronic device receives the reflected second ultrasonic signal, which specifically includes: the first electronic device receives the reflected second ultrasonic signal through a first microphone.

[0037] In this way, the first electronic device can send out an ultrasonic signal through a speaker closest to the target sound-emitting object and receive a reflected ultrasonic signal through a microphone closest to the target sound-emitting object, thereby improving the accuracy of identifying the target sound-emitting object.

[0038] In combination with the first aspect, in a possible implementation, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object, specifically including: the first electronic device obtains a first energy value of the first audio signal collected by the first microphone and a second energy value of the first audio signal collected by the second microphone; when the first energy value is greater than the second energy value, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object; and / or the first electronic device obtains a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; when the first moment is less than the second moment, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object.

[0039] In a second aspect, the present application provides an electronic device, comprising a microphone array, a memory, and a processor; wherein the microphone array, the memory, and the processor are coupled, the memory is used to store a computer program, and when the processor executes and calls the computer program, the electronic device executes an audio acquisition method provided in any possible implementation of the first aspect.

[0040] In a third aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enables the electronic device to execute an audio acquisition method provided in any possible implementation of the first aspect.

[0041] In a fourth aspect, the present application provides a computer program product comprising instructions, and when the above-mentioned computer program product is run on an electronic device, the electronic device executes an audio acquisition method provided in any possible implementation manner of the above-mentioned first aspect.

[0042] In a fifth aspect, the present application provides a chip system, which includes one or more processors, and the processor is used to call computer instructions to enable an electronic device to execute an audio acquisition method provided in any possible implementation of the first aspect above.

[0043] For the description of the beneficial effects of the second to fifth aspects, reference may be made to the description of the beneficial effects in the first aspect, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1A-1H A UI diagram showing an application in the electronic device 100 collecting audio;

[0045] Figure 2 A schematic structural diagram of an electronic device 100 is shown;

[0046] Figure 3 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention;

[0047] Figure 4A-Figure 4N A schematic diagram showing a group of electronic devices 100 automatically collecting audio when recognizing near-field speech;

[0048] Figure 5 A schematic diagram of the functional modules of an audio acquisition method provided in this application;

[0049] Figure 6 A schematic diagram of a method flow of an audio acquisition method provided in this application;

[0050] Figure 7A-7C A schematic diagram of the arrangement positions of microphones and speakers on a set of electronic devices 100;

[0051] Figure 7D-7F A schematic diagram showing another group of electronic devices 100 automatically collecting audio when recognizing near-field speech;

[0052] Figure 8 A schematic diagram of the functional modules of an audio acquisition method provided in this application;

[0053] Fig. 9 A schematic diagram of another method flow of an audio acquisition method provided by the present application;

[0054] Fig.10A flowchart of another audio acquisition method provided in this application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0056] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0057] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of a wearable device.

[0058] The user can operate in an application that supports audio recording function on the electronic device to control the application to collect audio. In some embodiments, the application can collect audio and save the audio or convert the audio into text. In other embodiments, the application can collect audio and send the audio to other devices.

[0059] Figure 1A-Figure 1D A UI diagram of an application in an electronic device 100 collecting audio is shown.

[0060] Exemplarily, the application may be a memo application.

[0061] Figure 1AThe desktop of the electronic device 100 is shown. The desktop of the electronic device 100 displays application icons of multiple applications, for example, a weather application icon, a stock application icon, a calculator application icon, a settings application icon, an email application icon, a music application icon, a video application icon, a browser application icon, a map application icon, a gallery application icon, a memo application icon, a voice assistant application icon, a beauty application icon, etc. A page indicator is also displayed below the multiple application icons to indicate the total number of pages on the desktop and the positional relationship between the currently displayed page and other pages. For example, the desktop may include three pages, and a white dot in the page indicator may indicate that the currently displayed page is the rightmost page of the three pages. Further optionally, there are multiple tray icons (such as a dialing application icon, a message application icon, a contact application icon, and a camera application icon) below the page indicator, and the tray icon remains displayed when the page is switched. Optionally, a status bar is displayed in the upper part of the desktop. The status bar may include: one or more signal strength indicators of a mobile communication signal (also referred to as a cellular signal), a battery status indicator, a time indicator, etc.

[0062] For example, Figure 1A As shown, the electronic device 100 can receive an input operation (such as a click) from a user on a memo application icon on the desktop, and in response to the user's input operation, the electronic device 100 can display Figure 1B The user interface 1100 shown is the main interface of the memo application provided in the embodiment of the present application.

[0063] like Figure 1B As shown, the user interface 1100 may include historical note information, and the current user has not created any notes. The user interface 1100 also includes a to-do list option and an add note option. Among them, the user can view one or more items that the user needs to deal with within a period of time through the to-do list option. The user can also create a new note through the add note option.

[0064] For example, Figure 1B As shown, the electronic device 100 may receive a user input operation (eg, click) for adding a note option, and in response to the user's input operation, the electronic device 100 may display Figure 1C User interface 1200 is shown.

[0065] like Figure 1CAs shown, the user can edit text in the user interface 1200, and the user interface 1200 includes multiple editing options, such as a view list option, a set style option, an insert picture option, a recording option, and a handwriting option. The user can also use the recording option to allow the memo application to collect the user's audio. In one possible implementation, the memo application can directly save the user's audio. In other possible implementations, the memo application can convert the user's audio into text and save the text information.

[0066] For example, Figure 1C As shown, the electronic device 100 may receive an input operation (eg, a single click) from a user for a recording option in the user interface 1200. In response to the input operation from the user, the memo application may display a Figure 1D The user interface 1300 is shown and the audio output by the user is collected.

[0067] However, if the user needs to use the recording function in the memo application, the user needs to follow the steps in sequence. Figure 1A-Figure 1C The user needs to use the steps to open the recording function in the memo application and collect the audio output by the user through the memo application, which is cumbersome for the user to operate. If the user does not frequently use the memo application, the user may not be able to find the location of the recording option in time, and the user experience is not good.

[0068] Figure 1E-1H Another UI diagram of an application in the electronic device 100 collecting audio is shown.

[0069] Exemplarily, the application may be an instant messaging application.

[0070] For example, Figure 1E As shown, the electronic device 100 can receive an input operation (such as a single click) from a user on an instant messaging application icon on the desktop, and in response to the user's input operation, the electronic device 100 can display Figure 1F The user interface 1400 shown. The user interface 1400 is the main interface of the instant messaging application provided in the embodiment of the present application. The user interface 1400 includes dialog boxes of multiple contacts. For example, the dialog box of the contact "Lisa", the dialog box of the contact "Alan", the dialog box of the contact "Henry", the dialog box of the contact "Lucy", etc. The electronic device 100 can receive a user operation to display a real-time chat interface of a certain contact.

[0071] For example, Figure 1F As shown, the electronic device 100 may receive an input operation (eg, a click) from a user on a dialog box for the contact "Lisa" in the user interface 1400. In response to the input operation from the user, the electronic device 100 may display Figure 1GUser interface 1500 is shown. User interface 1500 is a real-time chat interface for contact "Lisa".

[0072] like Figure 1G As shown, the user interface 1500 includes multiple chat records. The user interface 1500 also includes a voice input option 1501, an emoticon input option 1502, a multi-function option 1503, and a keyboard switch option 1504. Among them, the user can record the audio content input by the user through the voice input option 1501 and send the audio content input by the user to the contact "Lisa". The user can select one or more emoticons to send to the contact "Lisa" through the emoticon input option 1502. The user can send pictures, location information, initiate voice / video calls, send files, etc. to the contact "Lisa" through the multi-function option 1503. The user can display the keyboard input box on the user interface 1500 by switching the keyboard option 1504.

[0073] For example, Figure 1G As shown, the electronic device 100 can receive a user input operation (such as long press) for the voice input option 1501 in the user interface 1500, and in response to the user's input operation, the electronic device 100 can display Figure 1H Option 4205, option 4206, prompt information 4207 and voice collection area 4208 are shown, and prompt information 4207 includes the text "Release to send" to prompt the user how to send audio. Among them, the user needs to long press the voice collection area 4208 before the electronic device 100 collects the voice. When the user does not long press the voice collection area 4208, the electronic device 100 stops collecting voice and sends the previously collected voice to the electronic device 200 (not shown in the figure). The user can move his finger to option 4206 so that the electronic device 100 converts the previously collected voice into text and sends it to the electronic device 200. The user can move his finger to option 4205 so that the electronic device 100 stops collecting and stops sending voice messages to the electronic device 200.

[0074] Similarly, if the user needs to use the audio collection function in the instant messaging application, the user needs to follow the steps in sequence. Figure 1E-1H The steps of opening the audio collection function in the instant messaging application and collecting the audio output by the user through the instant messaging application are cumbersome for the user to operate. If the user does not frequently use the instant messaging application, the user may not be able to find the position of the voice input option 1501 in time, and the user experience is not good.

[0075] In other embodiments, the user may also wake up the memo application to collect audio or wake up the instant messaging application to collect audio by voice command. However, there are many cases of accidental touches in the voice wake-up method.

[0076] In other embodiments, the user can also use a distance sensor to detect whether the user is close. When the user is close to the device, the electronic device 100 automatically controls the memo application to collect audio or controls the instant messaging application to collect audio. However, distance detection requires an additional proximity device and can only detect whether the user is close, but cannot determine whether the user is speaking, and there are many cases of false touches.

[0077] Based on the above analysis, the present application provides an audio acquisition method, which includes the following steps:

[0078] Step 1: The electronic device 100 confirms whether there is near-field speech. In the case of near-field speech, the electronic device 100 can confirm that the user needs to use the audio collection function of the electronic device 100.

[0079] Among them, near-field voice can refer to an audio signal emitted by a target sound-emitting object within a first preset distance (for example, 1 meter) from the electronic device 100.

[0080] Optionally, the electronic device 100 is pre-installed with multiple microphones. If the same audio signal emitted by the same target sound-emitting object reaches the multiple microphones at different times and energy values, it can be determined whether the audio signal is near-field speech based on the difference in the times and energy values ​​of the audio signal reaching the multiple microphones. For an introduction to near-field speech, please refer to Figure 6 The detailed description in the embodiments is not repeated here in this application.

[0081] In some embodiments, the electronic device 100 may further confirm whether the target sound source is identified. After identifying the target sound source, the electronic device 100 determines that the user needs to use the audio collection function of the electronic device 100. This can improve the accuracy of confirming that the user uses the audio collection function of the electronic device 100.

[0082] In a possible implementation, the electronic device 100 may send an ultrasonic signal and confirm whether the target sound source is identified through the reflected ultrasonic signal. If the user is speaking, the ultrasonic signal hits the user's mouth, and the mouth movement causes the frequency and / or amplitude of the ultrasonic signal to change. The electronic device 100 can identify the target sound source based on the characteristics of the reflected ultrasonic signal.

[0083] Not limited to ultrasonic signals, the electronic device 100 can also determine the target sound source through other methods, which is not limited in this application.

[0084] Step 2: The electronic device 100 collects audio.

[0085] The electronic device 100 may save the collected audio locally, or the electronic device 100 may convert the collected audio into text and save the text, or the electronic device 100 may send the collected audio to other electronic devices, or the electronic device 100 may convert the collected audio into text and the electronic device 100 may send the text to other electronic devices.

[0086] By using this method, the electronic device 100 can automatically collect audio and process the audio when near-field audio is recognized. The application can be controlled to collect audio without the user having to manually start the audio collection function of the application, thereby improving audio collection efficiency, reducing user operations, and improving user experience.

[0087] The following introduces the hardware structure of an electronic device 100 provided in an embodiment of the present application.

[0088] Figure 2 A schematic structural diagram of an electronic device 100 is shown.

[0089] The following is a detailed description of the embodiment using the electronic device 100 as an example. It should be understood that: Figure 2 The electronic device 100 shown is only an example, and the electronic device 100 may have more Figure 2 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0090] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0091] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0092] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0093] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0094] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0095] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0096] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0097] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0098] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0099] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0100] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0101] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0102] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0103] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0104] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0105] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0106] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0107] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0108] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0109] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0110] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0111] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0112] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0113] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microled, micro-oled, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0114] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0115] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0116] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0117] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0118] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0119] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0120] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0121] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0122] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0123] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0124] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0125] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0126] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0127] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0128] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0129] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0130] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0131] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0132] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0133] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0134] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0135] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0136] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0137] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0138] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0139] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0140] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0141] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0142] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0143] The SIM card interface 195 is used to connect a SIM card.

[0144] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0145] Figure 3 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0146] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0147] The application layer can include a series of application packages.

[0148] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0149] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0150] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0151] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0152] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0153] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0154] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0155] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0156] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0157] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0158] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0159] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0160] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0161] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0162] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0163] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0164] A 2D graphics engine is a drawing engine for 2D drawings.

[0165] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0166] Next, an application scenario of an audio acquisition method provided by this application is introduced in conjunction with UI.

[0167] The electronic device 100 can automatically determine whether there is near-field speech at present. After determining that there is near-field speech, the electronic device 100 can automatically start the audio collection function of the first application and collect audio. In one possible implementation, the electronic device 100 can save the audio collected by the first application in the first application, or convert the audio collected by the first application into text and save the text in the first application. In other possible implementations, the electronic device 100 can send the audio collected by the first application to the electronic device 200 that establishes a communication connection with it.

[0168] In some embodiments, when the electronic device 100 is not connected to a Bluetooth headset, the electronic device 100 can collect audio through a microphone on the electronic device 100. The electronic device 100 can also play audio through a speaker on the electronic device 100.

[0169] Figure 4A-Figure 4N A schematic diagram showing a group of electronic devices 100 recognizing near-field speech and automatically collecting audio.

[0170] in, Figure 4A-4J A schematic diagram is shown in which an electronic device 100 recognizes near-field speech, automatically collects audio, and saves the audio.

[0171] In some embodiments, the electronic device 100 may save the audio collected by the first application in the first application, or convert the audio collected by the first application into text and save the text in the first application.

[0172] Exemplarily, the first application may be a memo application.

[0173] For example, Figure 4A As shown, the user can pick up the electronic device 100 and actively move close to the electronic device 100. The user can output voice, and when the electronic device 100 recognizes that the current audio is near-field voice, the electronic device 100 can automatically open the memo application and control the memo application to start collecting audio.

[0174] Optionally, before the electronic device 100 recognizes that the current audio is near-field speech, the electronic device 100 may display the audio on any user interface. Figure 4B As shown, the electronic device 100 can display a desktop.

[0175] In response to recognizing the near-field voice, the electronic device 100 may automatically open the memo application and display the memo. Figure 4C User interface 4100 is shown.

[0176] like Figure 4B As shown, prompt information 4101 is displayed on the user interface 4100, and the prompt information 4101 includes the text "near-field voice mode". The prompt information 4101 is used to prompt the user that the near-field voice mode has been started.

[0177] In some embodiments, after the electronic device 100 recognizes that the current audio is near-field speech, before the electronic device 100 automatically opens the memo application, the electronic device 100 can further confirm whether the target sound source is recognized. After the target sound source is recognized, the electronic device 100 opens the memo application again. This can improve the accuracy of confirming that the user uses the audio collection function of the electronic device 100.

[0178] In a possible implementation, the electronic device 100 may send an ultrasonic signal and confirm whether the target sound source is identified through the reflected ultrasonic signal. If the user is speaking, the ultrasonic signal hits the user's mouth, and the mouth movement causes the frequency and / or amplitude of the ultrasonic signal to change. The electronic device 100 can identify the target sound source based on the characteristics of the reflected ultrasonic signal.

[0179] Not limited to ultrasonic signals, the electronic device 100 can also determine the target sound source through other methods, which is not limited in this application.

[0180] In some embodiments, after the electronic device 100 recognizes that the current audio is near-field speech, the electronic device 100 may not display Figure 4C The user interface 4100 shown directly controls the memo application to start collecting audio and displays Figure 4D User interface 4200 is shown.

[0181] like Figure 4D As shown, the user interface 4200 can display the process of audio collection, such as a waveform, audio duration, and text information corresponding to the audio. In this way, the user can check whether the audio collected by the memo application is the same as the audio output by the user. When the audio collected by the memo application is different from the audio output by the user, the user can choose to re-enter the audio or modify the part that is different from the audio output by the user to avoid errors in the audio collection of the memo application.

[0182] In some embodiments, after the electronic device 100 fails to recognize near-field voice, for example, after the user stops outputting audio, the electronic device 100 may control the memo application to stop collecting audio.

[0183] After the user stops outputting audio, the memo application can save the collected audio within the memo application.

[0184] In a possible implementation, the memo application can directly save the collected audio within the memo application and automatically generate a voice note.

[0185] After the memo application generates a voice note, the electronic device 100 can display Figure 4E the user interface 4300 as shown. The user interface 4300 includes the name of the voice note, the generation date of the voice note, the duration of the voice note, etc. For example, the name of the voice note can be "Thoughts Memo". Optionally, the name of the voice note can be automatically extracted and generated by the memo application based on the audio content. The generation date of the voice note can be "December 18, 2023". The duration of the voice note can be 1 minute and 26 seconds.

[0186] In some embodiments, after the memo application generates a voice note, the electronic device 100 can display Figure 4F the user interface 4400 as shown. The user interface 4400 not only includes the name of the voice note, the generation date of the voice note, the duration of the voice note, but also includes a note type identifier 4401. The note type identifier 4401 is used to indicate the note type, and the note type can include but is not limited to text notes, voice notes, etc. Optionally, the note type identifier 4401 can also be automatically extracted and generated by the memo application based on the audio content.

[0187] In other possible implementations, the memo application can convert the collected audio into text, automatically generate a text note, and save the text note within the memo application.

[0188] In a possible implementation, after the memo application generates a text note, the electronic device 100 can display Figure 4E the user interface 4300 as shown.

[0189] In other possible implementations, after the memo application generates a text note, the electronic device 100 can display Figure 4G the user interface 4500 as shown. The user interface 4500 not only includes the name of the voice note, the generation date of the voice note, the duration of the voice note, but also includes a note type identifier 4501. The note type identifier 4501 is used to indicate the note type, and the note type can include but is not limited to text notes, voice notes, etc. Optionally, the note type identifier 4501 can also be automatically extracted and generated by the memo application based on the audio content.

[0190] After the electronic device 100 does not recognize the near-field voice, for example, after the user stops outputting audio, the electronic device 100 can control the memo application to stop collecting audio. After the memo application stops collecting audio, the memo application can continue to display Figure 4E or Figure 4F or Figure 4G Alternatively, after the memo application stops collecting audio, the memo application can continue to display the user interface displayed before collecting audio, such as displaying Figure 4B Alternatively, after the memo application stops collecting audio, the memo application can also display the main interface of the memo application, for example Figure 4C The user interface 4100 is shown. After the memo application stops collecting audio, the memo application may also display other user interfaces, which is not limited in this application.

[0191] In some embodiments, after the memo application generates a voice note or a text note, the electronic device 100 may also receive a user operation to view the voice note or the text note.

[0192] For example, Figure 4H As shown, the electronic device 100 may receive an input operation (eg, a single click) from a user for the "Notes" option in the user interface 4300. In response to the input operation from the user, the electronic device 100 may display Fig. 4I The user interface 4600 may include the name of the note, the creation time of the note, and the content of the note. The name of the note may be "Notes on my mind", the creation time of the note may be "December 18, 2023", and the content of the note may be "Today, the spring breeze is blowing and the flowers are blooming in the garden. Time flies, and the years are passing by. However, I have something in my mind, and I am writing it down here."

[0193] For example, Figure 4H As shown, the electronic device 100 may receive an input operation (eg, a single click) from a user for the “Notes” option in the user interface 4300. In response to the input operation from the user, the electronic device 100 may also display Figure 4J The user interface 4700 may include the name of the note, the creation time of the note, and the duration of the note. The name of the note may be "Xin Shi Bei Lu", the creation time of the note may be "December 18, 2023", and the duration of the note may be 1 minute and 26 seconds. The user interface 4700 also includes a voice note playback option, a voice note deletion option, and more options. The user can choose to share the voice note to other devices or other contacts and friends in more options.

[0194] Through the above method, the electronic device 100 can automatically open the memo application and control the memo application to collect audio and save the audio when the near-field voice is recognized. Figure 1A-Figure 1C The steps to enable the recording function in the Memo app and collect the audio output by the user through the Memo app. This saves user operations and improves the user experience.

[0195] Figure 4K-Figure 4N A schematic diagram is shown in which an electronic device 100 recognizes near-field speech, automatically collects audio, and sends the audio to an electronic device 200.

[0196] In some embodiments, the electronic device 100 may send the audio collected by the first application to the electronic device 200 , or convert the audio collected by the first application into text and then send it to the electronic device 200 .

[0197] Exemplarily, the first application may be an instant messaging application.

[0198] You can refer to Figure 4A In the description of the embodiment, the user can pick up the electronic device 100 and actively move close to the electronic device 100. The user can output voice, and when the electronic device 100 recognizes that the current audio is near-field voice, the electronic device 100 can control the instant messaging application to start collecting audio.

[0199] Optionally, before the electronic device 100 recognizes that the current audio is near-field speech, the electronic device 100 may display a chat interface of the contact. Figure 4K As shown, the electronic device 100 can display a chat interface of the contact "Lisa".

[0200] In response to recognizing the near-field voice, the electronic device 100 may display Figure 4L Prompt bar 4800 is shown.

[0201] like Figure 4L As shown, prompt information 4801 is displayed on prompt bar 4800, and prompt information 4801 includes the text "near-field voice mode". The prompt information 4801 is used to prompt the user that the near-field voice mode has been started.

[0202] In some embodiments, after the electronic device 100 recognizes that the current audio is near-field speech, the electronic device 100 may further confirm whether the target sound source is recognized. After the target sound source is recognized, the electronic device 100 opens the memo application. This can improve the accuracy of confirming that the user uses the audio collection function of the electronic device 100.

[0203] In a possible implementation, the electronic device 100 may send an ultrasonic signal and confirm whether the target sound source is identified through the reflected ultrasonic signal. If the user is speaking, the ultrasonic signal hits the user's mouth, and the mouth movement causes the frequency and / or amplitude of the ultrasonic signal to change. The electronic device 100 can identify the target sound source based on the characteristics of the reflected ultrasonic signal.

[0204] Not limited to ultrasonic signals, the electronic device 100 can also determine the target sound source through other methods, which is not limited in this application.

[0205] In some embodiments, after the electronic device 100 recognizes that the current audio is near-field speech, the electronic device 100 may not display the prompt information 4801.

[0206] like Figure 4L As shown, the prompt bar 4800 can display the process of audio collection, such as a waveform, audio duration, and text information corresponding to the audio. In this way, the user can check whether the audio collected by the instant messaging application is the same as the audio output by the user. If the audio collected by the instant messaging application is different from the audio output by the user, the user can choose to re-enter the audio or modify the part that is different from the audio output by the user to avoid errors in the audio collection of the instant messaging application.

[0207] In some embodiments, after the electronic device 100 fails to recognize near-field voice, for example, after the user stops outputting audio, the electronic device 100 can control the instant messaging application to stop collecting audio.

[0208] After the user stops outputting audio, in a possible implementation, the instant messaging application can send the collected audio to the contact "Lisa" and display Figure 4M The user interface 4900 shown includes a mark 4901, and the mark 4901 is used to indicate the audio message sent by the current user to the contact "Lisa".

[0209] After the user stops outputting audio, in other possible implementations, the instant messaging application can convert the collected audio into text, and then send the text to the contact "Lisa" and display Figure 4N The user interface 4910 shown includes an identifier 4911, and the identifier 4911 is used to indicate a text message sent by the current user to the contact "Lisa".

[0210] After the electronic device 100 does not recognize the near-field voice, for example, after the user stops outputting audio, the electronic device 100 can control the instant messaging application to stop collecting audio. Figure 4M The user interface 4900 shown or Figure 4N User interface 4910 is shown.

[0211] Figure 5 A schematic diagram of the functional modules of an audio acquisition method provided in this application.

[0212] like Figure 5 As shown, the functional modules on the electronic device 100 may include but are not limited to: a voice pickup unit, a central control unit, a near-field pattern recognition unit, a near-field voice separation unit, and an interaction unit, etc.

[0213] The voice pickup unit is used to pick up audio signals periodically / irregularly and send the picked up audio signals to the central control unit.

[0214] The central control unit is used to receive the audio signal sent by the voice pickup unit, process the audio signal, and determine whether there is a voice signal.

[0215] The central control unit is further used to send the audio signal to the near-field pattern recognition unit after determining that there is a voice signal.

[0216] In some embodiments, the central control unit is specifically used to determine the probability of the voice signal existing in the audio signal, and when the probability of the voice signal existing in the audio signal is greater than a preset value, the central control unit sends the audio signal to the near-field pattern recognition unit. In this way, noise interference can be avoided.

[0217] The near-field pattern recognition unit is used to receive the audio signal sent by the central control unit, process the audio signal, and determine whether there is near-field speech.

[0218] The near-field pattern recognition unit is further configured to send the audio signal to the near-field speech separation unit when it is determined that near-field speech exists in the audio signal.

[0219] The central control unit is also used to control the speaker to emit an ultrasonic signal after the near-field mode recognition unit determines that there is near-field speech in the audio signal, so as to determine whether the target sound source is recognized, thereby improving the accuracy of the electronic device 100 entering the near-field speech mode. The present application can emit ultrasonic signals through the speaker module without adding other devices.

[0220] The voice pickup unit is also used to control the speaker in the central control unit to emit an ultrasonic signal, receive the reflected ultrasonic signal, and send the reflected ultrasonic signal to the central control unit.

[0221] The central control unit is also used to confirm whether the target sound-emitting object is identified based on the characteristics of the emitted ultrasonic signal and the reflected ultrasonic signal. For how the central control unit identifies the target sound-emitting object, please refer to Figure 6 The description of S603 in the embodiment is not repeated in this application.

[0222] The central control unit is also used to send a message of identifying the target sound-emitting object to the near-field pattern recognition unit after identifying the target sound-emitting object.

[0223] Optional, Figure 5 Steps 6 to 10 shown in the embodiment may not be performed, and this application does not limit this.

[0224] The near-field pattern recognition unit is also used to send an audio signal to the near-field speech separation unit in response to a message sent by the central control unit that a target sound-emitting object has been recognized.

[0225] The near-field speech separation unit is used to separate the near-field speech from the audio signal after receiving the audio signal sent by the near-field pattern recognition unit.

[0226] The central control unit is also used to send a first instruction to the interaction unit after identifying the target sound-emitting object.

[0227] The interaction unit is used to display an audio collection interaction interface after receiving the first instruction sent by the central control unit, so as to prompt the user that the electronic device 100 is currently in the near-field audio collection mode.

[0228] Optionally, the interaction unit may be a functional module in the first application. The near-field speech separation unit may also be a functional module in the first application.

[0229] Figure 6 A method flow chart of an audio acquisition method provided in this application.

[0230] S601: The electronic device 100 collects audio signals periodically or irregularly.

[0231] Optionally, one or more microphones are pre-installed on the electronic device 100, and the electronic device 100 can collect audio signals periodically / irregularly through the one or more microphones.

[0232] S602: The electronic device 100 needs to determine whether the audio signal includes near-field speech.

[0233] After the electronic device 100 acquires the audio signal, the electronic device 100 needs to confirm whether the audio signal includes near-field speech.

[0234] In some embodiments, before the electronic device 100 confirms whether the audio signal is near-field speech, the electronic device 100 may first determine whether the audio signal contains a speech signal, and then confirm whether the audio signal includes near-field speech.

[0235] Exemplarily, the electronic device 100 may determine the probability of a speech signal existing in the audio signal, and when the probability of a speech signal existing in the audio signal is greater than a first threshold (e.g., 80%), the electronic device 100 further confirms whether the audio signal includes near-field speech. In this way, noise interference can be avoided.

[0236] Among them, near-field voice can refer to an audio signal emitted by a target sound-emitting object within a first preset distance from the electronic device 100.

[0237] The electronic device 100 includes a microphone array, and the microphone array includes at least two microphones, for example, a first microphone and a second microphone. The electronic device 100 can determine a first distance between the electronic device 100 and a target sound-emitting object based on an audio signal picked up by the microphone array. When the first distance is less than a first preset distance, or the first distance is less than the first preset distance for a first continuous time, the electronic device 100 can determine that near-field speech is recognized.

[0238] Optionally, the microphone array includes at least two microphones, and the time and energy of the same audio signal emitted by the same target sound-emitting object arriving at the multiple microphones are different. Based on the time difference and energy difference of the same audio signal arriving at the at least two microphones, the first distance between the electronic device 100 and the target sound-emitting object can be determined, and then it can be further determined whether it is near-field speech.

[0239] For example, the microphone array may include at least a first microphone and a second microphone, and the first microphone and the second microphone are located at different positions on the first electronic device. The electronic device 100 may collect the first audio signal through the first microphone and the second microphone. The electronic device 100 may obtain a first energy value of the first audio signal collected by the first microphone and a second energy value of the first audio signal collected by the second microphone, and the first electronic device may determine the first distance between the first electronic device and the target sound-emitting object based on the difference between the first energy value and the second energy value, and / or, the first electronic device may obtain a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone, and the first electronic device may also determine the first distance between the first electronic device and the target sound-emitting object based on the difference between the first moment and the second moment.

[0240] Exemplarily, the electronic device 100 can also determine the distance between each microphone and the target sound-emitting object. For example, when the first energy value is greater than the second energy value or the first energy value is less than the second energy value for a certain period of time, the first electronic device can determine that the distance between the first microphone and the target sound-emitting object is less than the distance between the second microphone and the target sound-emitting object, and / or, the first electronic device can also obtain the first moment of the first audio signal collected by the first microphone and the second moment of the first audio signal collected by the second microphone. When the first moment is less than the second moment or the first moment is less than the second moment for a certain period of time, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is less than the distance between the second microphone and the target sound-emitting object.

[0241] Exemplarily, the multiple microphones pre-installed on the electronic device 100 may be located at different positions. For example, the electronic device 100 is pre-installed with microphone 1, microphone 2, and microphone 3. Microphone 1 is located at the bottom of the electronic device 100, microphone 2 is located at the top of the electronic device 100, and microphone 3 is located at the back of the electronic device 100, for example, it may be located at the position where the camera module is located on the back of the electronic device 100.

[0242] In some embodiments, microphone 1 may be referred to as a first microphone and microphone 2 may be referred to as a second microphone.

[0243] Fig. 7A The schematic diagram of the positions of multiple microphones pre-installed on the electronic device 100 is exemplarily shown.

[0244] like Fig. 7A As shown, microphone 1 is located at the bottom of electronic device 100, microphone 2 is located at the top of electronic device 100, and microphone 3 is located at the back of electronic device 100, for example, it can be located at the position where the camera module is located at the back of electronic device 100. In some embodiments, one or more microphones are also used to receive ultrasonic signals to detect the target sound source.

[0245] In some embodiments, one or more speakers may be pre-installed on the electronic device 100, and the one or more speakers are used to play audio. In some embodiments, the one or more speakers are also used to emit ultrasonic signals to detect the target sound-emitting part.

[0246] like Fig. 7A As shown, the speaker 1 is located at the bottom of the electronic device 100, and the speaker 2 is located at the top of the electronic device 100. In some embodiments, the back of the electronic device 100 may also include a speaker 3, for example, the speaker 3 may be located at the position where the camera module is located on the back of the electronic device 100. In some embodiments, the back of the electronic device 100 may not include the speaker 3.

[0247] It should be noted that it is not limited to 3 microphones and 3 speakers. Fig. 7A The positions of the multiple microphones and the multiple speakers on the electronic device 100 are merely illustrated, and the present application does not limit this.

[0248] The multiple microphones are located at different positions on the electronic device 100. The closer the target sound-emitting object is to the microphone, the more energy the audio collected by the microphone is and the shorter the time is. The farther the target sound-emitting object is from the microphone, the less energy the audio collected by the microphone is and the longer the time is. Then within the first preset distance, the time and energy values ​​of the same audio signal emitted by the same target sound-emitting object arriving at the multiple microphones are significantly different, and whether the audio signal is near-field speech can be determined based on the difference in the time and energy values ​​of the same audio signal arriving at the multiple microphones.

[0249] In other embodiments, when the distance between the target sound-emitting object and the electronic device 100 exceeds the first preset distance, that is, the target sound-emitting object is far away from the electronic device 100, there is no obvious difference in the time and energy value of the same audio signal emitted by the same target sound-emitting object arriving at multiple microphones on the electronic device 100, which can be used to determine that the current audio is non-near-field speech.

[0250] Exemplarily, when the distance between the target sound-emitting object and the electronic device 100 is within a first preset distance, the target sound-emitting object outputs a first audio. The energy of the first audio received by microphone 1 is a first energy value, the energy of the first audio received by microphone 2 is a second energy value, and the energy of the first audio received by microphone 3 is a third energy value. When the first energy value, the second energy value, and the third energy value are different and meet the preset conditions, the first audio can be considered to be near-field speech. Optionally, the preset condition can be that the difference between any two energy values ​​of the first energy value, the second energy value, and the third energy value meets the preset energy difference, for example, is greater than the preset energy difference.

[0251] Exemplarily, when the distance between the target sound-emitting object and the electronic device 100 is within a first preset distance, the target sound-emitting object outputs a first audio. The moment when the first audio is received by microphone 1 is the first moment, the moment when the first audio is received by microphone 2 is the second moment, and the moment when the first audio is received by microphone 3 is the third moment. When the first moment, the second moment, and the third moment are different and meet the preset conditions, the first audio can be considered to be near-field speech. Optionally, the preset condition can be that the difference between any two moments among the first moment, the second moment, and the third moment meets the preset time difference, for example, is greater than the preset time difference.

[0252] It should be noted that the above-mentioned preset conditions may also be other judgment bases, and this application does not limit this.

[0253] For example, Figure 7B As shown, the user can pick up the electronic device 100 and actively move close to the electronic device 100. When the user moves close to the bottom of the electronic device 100 and outputs audio, the distance relationship between the target sound-emitting part (such as the mouth) and each microphone on the electronic device 100 can be obtained. For example, the distance between the target sound-emitting part and microphone 1 is distance 1, the distance between the target sound-emitting part and microphone 2 is distance 2, and the distance between the target sound-emitting part and microphone 3 is distance 3. Since the user moves close to the bottom of the electronic device 100 to output audio, distance 1 is less than distance 3, which is less than distance 2.

[0254] When the user outputs the first audio, the energy of the first audio received by microphone 1 is the first energy value, the energy of the first audio received by microphone 2 is the second energy value, and the energy of the first audio received by microphone 3 is the third energy value. Since distance 1 is less than distance 3 and less than distance 2, the first energy value, the second energy value and the third energy value are different, and the first energy value is greater than the third energy value, and the third energy value is greater than the second energy value. Optionally, when the difference between the first energy value and the third energy value meets the preset energy difference, and the difference between the third energy value and the second energy value meets the preset energy difference, for example, the difference between the first energy value and the third energy value is greater than the preset energy difference, and the difference between the third energy value and the second energy value is greater than the preset energy difference, then the first audio can be considered to be near-field speech.

[0255] When the user outputs the first audio, the moment when microphone 1 receives the first audio is the first moment, the moment when microphone 2 receives the first audio is the second moment, and the moment when microphone 3 receives the first audio is the third moment. Since distance 1 is less than distance 3 and less than distance 2, the first moment, the second moment, and the third moment are different, and the first moment is less than the third moment, and the third moment is less than the second moment. Optionally, when the difference between the first moment and the third moment meets the preset time difference, and the difference between the third moment and the second moment meets the preset time difference, for example, the difference between the first moment and the third moment is greater than the preset time difference, and the difference between the third moment and the second moment is greater than the preset time difference, then the first audio can be considered to be near-field speech.

[0256] For example, Figure 7CAs shown, the user can pick up the electronic device 100 and actively move close to the electronic device 100. When the user moves close to the top of the electronic device 100 and outputs audio, the distance relationship between the target sound-emitting part (such as the mouth) and each microphone on the electronic device 100 can be obtained. For example, the distance between the target sound-emitting part and microphone 1 is distance 1, the distance between the target sound-emitting part and microphone 2 is distance 2, and the distance between the target sound-emitting part and microphone 3 is distance 3. Since the user moves close to the top of the electronic device 100 to output audio, distance 2 is less than distance 3, which is less than distance 1.

[0257] When the user outputs the first audio, the energy of the first audio received by microphone 1 is the first energy value, the energy of the first audio received by microphone 2 is the second energy value, and the energy of the first audio received by microphone 3 is the third energy value. Since distance 2 is less than distance 3 and less than distance 1, the first energy value, the second energy value and the third energy value are different, and the second energy value is greater than the third energy value, and the third energy value is greater than the first energy value. Optionally, when the difference between the second energy value and the third energy value satisfies the preset energy difference, and the difference between the third energy value and the first energy value satisfies the preset energy difference, for example, the difference between the second energy value and the third energy value is greater than the preset energy difference, and the difference between the third energy value and the first energy value is greater than the preset energy difference, then the first audio can be considered to be near-field speech.

[0258] When the user outputs the first audio, the moment when microphone 1 receives the first audio is the first moment, the moment when microphone 2 receives the first audio is the second moment, and the moment when microphone 3 receives the first audio is the third moment. Since distance 2 is less than distance 3 and less than distance 1, the first moment, the second moment, and the third moment are different, and the second moment is less than the third moment, and the third moment is less than the first moment. Optionally, when the difference between the second moment and the third moment satisfies the preset time difference, and the difference between the third moment and the first moment satisfies the preset time difference, for example, the difference between the second moment and the third moment is greater than the preset time difference, and the difference between the third moment and the first moment is greater than the preset time difference, then the first audio can be considered to be near-field voice.

[0259] Whether the audio is near-field speech is not limited to the above-mentioned method, and whether the audio is near-field speech can also be determined based on other methods, which is not limited in this application.

[0260] After determining that the audio signal is near-field speech, the electronic device 100 may determine that the current mode is near-field speech mode, and the electronic device 100 may control the first application to collect the audio signal and execute S603.

[0261] Optionally, after determining that the audio signal of the first continuous duration is near-field speech, the electronic device 100 may determine that the current mode is near-field speech mode, and the electronic device 100 controls the first application to collect the audio signal and executes S603.

[0262] After determining that the audio signal is not near-field voice, the electronic device 100 may determine that the current mode is not near-field voice mode, and the electronic device continues to monitor whether there is near-field voice, and executes S601.

[0263] S603: The electronic device 100 needs to determine whether the target sound source is recognized.

[0264] After determining that the audio signal includes near-field speech, before collecting the audio signal, the electronic device 100 needs to determine whether the target sound source is identified. After identifying the target sound source, the electronic device 100 collects the audio signal. In this way, the accuracy of the electronic device 100 entering the near-field speech mode can be improved, and the occurrence of accidental touches can be further prevented.

[0265] In some embodiments, after determining that the audio signal is near-field speech, the electronic device 100 also needs to confirm whether the target sound-emitting part is recognized. Recognizing the target sound-emitting part may mean that the distance between the electronic device 100 and the target sound-emitting object is within a second preset distance (for example, 10 cm), and the vibration frequency of the target sound-emitting part is within a first range. Optionally, the second preset distance may be less than the first preset distance. Optionally, the second preset distance may also be equal to the first preset distance. After determining the target sound-emitting object, it is necessary to further determine whether the distance between the target sound-emitting part of the target sound-emitting object and the electronic device 100 is close enough, and whether the target sound-emitting part is active. Only after the distance is close enough and the target sound-emitting part is active, can it be determined that the target sound-emitting part is making a sound, and the electronic device 100 starts collecting audio again, which can further prevent the sound from being accidentally touched.

[0266] In some embodiments, after determining that the audio signal is near-field speech, the electronic device 100 can emit a first ultrasonic signal and receive a reflected second ultrasonic signal. The electronic device 100 can determine the second distance between the first electronic device and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the first ultrasonic signal and the second ultrasonic signal. When the second distance is less than the second preset distance and the vibration frequency of the target sound-emitting part of the target sound-emitting object is within the first range, the electronic device 100 can determine that the target sound-emitting part is making a sound, and the electronic device 100 starts collecting audio again, which can further prevent the sound from being accidentally touched.

[0267] The first range may be between the first frequency value and the second frequency value. For example, the first range may be 20 Hz-40 Hz.

[0268] Optionally, when it is determined based on the first ultrasonic signal and the second ultrasonic signal for a continuous first preset time period that the second distance is greater than the second preset distance and / or the vibration frequency of the target sound-emitting part of the target sound-emitting object is not within the first range, the electronic device 100 exits the near-field voice mode and stops collecting audio.

[0269] In some embodiments, after determining that the audio signal is near-field speech, the electronic device 100 may emit an ultrasonic signal with preset characteristics, and determine whether the target sound-emitting part is identified and the distance between the electronic device 100 and the target sound-emitting object based on the characteristics of the reflected ultrasonic signal. Optionally, the preset characteristics of the emitted ultrasonic signal may include but are not limited to a preset frequency, a preset amplitude, a transmission time, etc. The characteristics of the reflected ultrasonic signal may include but are not limited to the frequency and amplitude of the reflected ultrasonic signal, the reflection time, etc.

[0270] Because when the target sound-producing part (such as the mouth) outputs audio, the target sound-producing part moves at a certain frequency and amplitude. The ultrasonic signal hits the moving target sound-producing part and is reflected, causing the frequency and amplitude of the ultrasonic signal to change. For example, the frequency and amplitude of the ultrasonic signal will change.

[0271] The electronic device 100 may determine the distance between the electronic device 100 and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the emitted ultrasonic signal and the reflected ultrasonic signal.

[0272] Optionally, the electronic device 100 may determine the distance between the electronic device 100 and the target sound-emitting object based on the difference between the time when the ultrasonic signal is emitted and the time when the ultrasonic signal is reflected.

[0273] Optionally, the electronic device 100 may determine the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the difference between the frequency and / or amplitude of the emitted ultrasonic signal and the frequency and / or amplitude of the reflected ultrasonic signal.

[0274] Exemplarily, the electronic device 100 can determine the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the frequency and amplitude of the ultrasonic signal reflected by the frequency and amplitude of the ultrasonic signal emitted, and compare the frequency and amplitude of the ultrasonic signal emitted by the electronic device 100, and determine whether the target sound-emitting part is recognized based on the vibration frequency of the target sound-emitting part. Exemplarily, when the vibration frequency of the target sound-emitting part is within the first range, it can be determined that the target sound-emitting part is recognized.

[0275] Exemplarily, when the difference between the frequency and amplitude of the reflected ultrasonic signal and the frequency and amplitude of the ultrasonic signal emitted by the electronic device 100 satisfies a preset value, such as being greater than a preset value, the electronic device 100 can determine that the target sound source is identified.

[0276] If the ultrasonic signal hits an inactive object, the frequency and amplitude of the ultrasonic signal will basically not change, and the difference between the frequency and amplitude of the reflected ultrasonic signal and the frequency and amplitude of the ultrasonic signal emitted by the electronic device 100 does not meet the preset value, for example, is less than the preset value. The electronic device 100 can determine that the target sound-emitting part is not identified.

[0277] In some embodiments, as can be seen from the introduction of S602, a plurality of speakers are pre-installed on the electronic device 100, and the electronic device 100 can emit ultrasonic signals through the speakers to determine whether the target sound-emitting part is recognized.

[0278] Based on the introduction of S602 , it can be known that the electronic device 100 can determine the distance between the target sound-emitting object and each microphone based on the energy values ​​and time differences of the audio signals received by the multiple microphones.

[0279] After determining the distance between the target sound-emitting object and each microphone, the electronic device 100 can determine the microphone closest to the target sound-emitting object. After determining the microphone closest to the target sound-emitting object, the electronic device 100 can send an ultrasonic signal through a speaker near the microphone, and the microphone receives the reflected ultrasonic signal to determine whether the target sound-emitting part is identified, wherein the speaker near the microphone can refer to the speaker closest to the microphone. In this way, the closer the target sound-emitting part is to the speaker and the microphone, the less the signal attenuation is, and the more accurate the detection result is.

[0280] Exemplary, reference Figure 7B As described in the embodiment, when the user approaches the bottom of the electronic device 100 and outputs audio, the target sound-emitting part is closest to the microphone 1 and the speaker 1. Then the electronic device 100 can emit an ultrasonic signal through the speaker 1 and receive the reflected ultrasonic signal through the microphone 1 to confirm whether the target sound-emitting part is recognized.

[0281] Exemplary, reference Figure 7C As described in the embodiment, when the user approaches the bottom of the electronic device 100 and outputs audio, the target sound-emitting part is closest to the microphone 2 and the speaker 2. Then the electronic device 100 can emit an ultrasonic signal through the speaker 2 and receive the reflected ultrasonic signal through the microphone 2 to confirm whether the target sound-emitting part is recognized.

[0282] The ultrasonic signal is not limited to being emitted and received by the speaker and microphone closest to the target sound object, but can also be emitted and received by multiple sets of speakers and microphones at the same time, and multiple sets of detection results can be used to confirm whether the target sound source is identified. In this way, the accuracy of identifying the target sound source can also be improved.

[0283] Whether the target sound source is identified is not limited to being confirmed by ultrasonic signals, but can also be confirmed by other methods, which is not limited in this application.

[0284] In some embodiments, the electronic device 100 may not execute S603. After determining the near-field voice in S602, the electronic device 100 may directly execute S604.

[0285] S604: The electronic device 100 collects an audio signal and saves the audio signal in the first application.

[0286] After determining the near-field voice, or after determining the near-field voice and identifying the target sounding position, the electronic device 100 can collect audio signals and save the collected audio signals in the first application.

[0287] The electronic device 100 saves the collected audio signal in the first application, which may include: the electronic device 100 directly saves the audio signal in the first application. Alternatively, the electronic device 100 converts the audio signal into text information and saves the text information in the first application. For example, the first application may be a memo application. For details, refer to Figure 4A-4J Description in the examples.

[0288] The electronic device 100 stores the collected audio signal in the first application, and may also include: the electronic device 100 sends the audio signal to the electronic device 200 through the first application. Alternatively, the electronic device 100 converts the audio signal into text information, and sends the text information to the electronic device 200 through the first application. For example, the first application may be an instant messaging application. For details, refer to Figure 4K-Figure 4N Description in the examples.

[0289] In some embodiments, after collecting the audio signal, the electronic device 100 needs to process the audio signal, filter to obtain the near-field voice signal, and filter out the far-field voice signal, so as to avoid the interference of the far-field voice signal.

[0290] Based on the description in S602, it can be known that the electronic device 100 can determine the distance between the electronic device 100 and the target sound-emitting object based on the audio signal collected by the microphone array. For example, the electronic device 100 simultaneously receives a second audio signal sent by the target sound-emitting object and a third audio signal output by other sound-emitting objects. When the electronic device 100 determines that the distance between the electronic device 100 and the target sound-emitting object is less than the first preset distance based on the second audio signal and the third audio signal, and the distance between the electronic device 100 and the other sound-emitting object is greater than the first preset distance, the electronic device 100 can only save the second audio signal and not save the third audio signal. For how the electronic device 100 determines the distance to the sound-emitting object, please refer to the description in S602, and this application will not repeat it here.

[0291] Exemplarily, based on the description in S602, the difference between near-field speech and far-field speech is that when the distance between the target sound-emitting object and the electronic device 100 is within the first preset distance, the energy value and time difference of the same audio emitted by the same sound-emitting object reaching multiple microphones on the electronic device 100 are significantly different, and the current audio can be judged as near-field speech based on this. When the distance between the target sound-emitting object and the electronic device 100 exceeds the first preset distance, that is, the target sound-emitting object is far away from the electronic device 100, the time and energy of the same audio emitted by the same target sound-emitting object reaching multiple microphones on the electronic device 100 are not significantly different, and the current audio can be judged as far-field speech based on this.

[0292] Exemplarily, energy values ​​of the same audio signal received by multiple microphones on the electronic device 100 are obtained, and the difference between any two of the multiple energy values ​​satisfies a preset energy difference, for example, is greater than the preset energy difference, and / or the moments of the same audio signal received by multiple microphones on the electronic device 100 are obtained, and the difference between any two of the multiple moments satisfies a preset time difference, for example, is greater than the preset time difference, then it can be determined that the audio is near-field speech.

[0293] Exemplarily, energy values ​​of the same audio signal received by multiple microphones on the electronic device 100 are obtained, and the difference between any two of the multiple energy values ​​does not satisfy a preset energy difference, for example, is less than the preset energy difference, and / or the moments of the same audio signal received by multiple microphones on the electronic device 100 are obtained, and the difference between any two of the multiple moments does not satisfy a preset time difference, for example, is less than the preset time difference, then it can be determined that the audio is far-field speech.

[0294] Based on the above features, near-field speech can be separated from the audio signal collected by the electronic device 100 .

[0295] If, in the audio signal collected by the electronic device 100, the near-field speech can be represented by s1(t), the transmission path of the near-field speech is f1(t,n), and the far-field speech can be represented by s2(t), the transmission path of the far-field speech is f2(t,n). Wherein, t represents time, f1(t,n) represents the transmission path of the near-field speech to the nth microphone, and f2(t,n) represents the transmission path of the far-field speech to the nth microphone. Then the audio signal collected by the electronic device 100 can be represented by formula (1).

[0296] y(t,n)= s1(t)*f1(t,n) + s2(t)*f2(t,n) Formula (1)

[0297] As shown in formula (1), y(t,n) represents the audio signal collected by the nth microphone, s1(t) represents the near-field speech collected by the electronic device 100, f1(t,n) represents the transmission path of the near-field speech, s2(t) represents the far-field speech collected by the electronic device 100, and f2(t,n) represents the transmission path of the far-field speech.

[0298] The audio signals received by n microphones on the electronic device 100 can be expressed by formula (2).

[0299] Y(t)=[y(t,1),y(t,2),…,y(t,N)] Formula (2)

[0300] As shown in formula (2), Y(t) represents the audio signal received by n microphones on the electronic device 100, y(t,1) represents the audio signal received by the first microphone on the electronic device 100, y(t,2) represents the audio signal received by the second microphone on the electronic device 100, and y(t,N) represents the audio signal received by the nth microphone on the electronic device 100.

[0301] By performing frequency domain transformation on Y(t), we can obtain Y(t,f) as shown in formula (3).

[0302] Y(t,f)=F(Y(t)) Formula (3)

[0303] As shown in formula (3), Y(t,f) represents the audio signal received by n microphones on the electronic device 100 in the frequency domain, and f represents the frequency. Y(t) represents the audio signal received by n microphones on the electronic device 100 in the time domain.

[0304] The electronic device 100 can separate the near-field speech from the audio signal collected by the electronic device 100 by using the following formula (4).

[0305]

[0306] As shown in formula (4), s1(t) represents the near-field speech separated from the audio signal collected by the electronic device 100, Y(t,f) represents the audio signal received by n microphones on the electronic device 100 in the frequency domain, and W(t,f) represents the solution matrix, which is used to separate the near-field speech from the audio signal collected by the electronic device 100. The solution matrix can represent the difference relationship between the energy values ​​of the same audio emitted by the same sound-emitting object arriving at multiple microphones on the electronic device 100 when the audio is near-field speech, and / or the difference relationship between the moments when the same audio emitted by the same sound-emitting object arrives at multiple microphones on the electronic device 100. Exemplarily, the difference relationship between the energy values ​​of the same audio emitted by the same sound-emitting object arriving at multiple microphones on the electronic device 100 can be: the energy value of the same audio signal received by multiple microphones on the electronic device 100, the difference between any two energy values ​​among the multiple energy values ​​satisfies the preset energy difference value, for example, is greater than the preset energy difference value. The difference relationship between the moments when the same audio emitted by the same sound-emitting object arrives at multiple microphones on the electronic device 100 can be: the moments when the same audio signal is received by multiple microphones on the electronic device 100 are obtained, and the difference between any two moments among the multiple moments satisfies a preset time difference, for example, is greater than the preset time difference.

[0307] Optionally, the solution matrix W(t, f) may be obtained by the electronic device 100 through deep learning, and the solution matrix W(t, f) may also be updated periodically / irregularly.

[0308] It should be noted that the above formulas (1) to (4) are only used to explain how the present application separates near-field speech from audio signals. Near-field speech can also be separated from audio signals by other methods, and the present application does not limit this.

[0309] S605: The electronic device 100 needs to continue monitoring whether the target sound source is identified.

[0310] After determining the near-field voice, or after determining the near-field voice and identifying the target sounding position, the electronic device 100 can collect audio signals and save the collected audio signals in the first application.

[0311] The electronic device 100 needs to continue monitoring whether the target sound source is identified. If the target sound source is not identified, the electronic device 100 exits the near-field voice mode, stops saving the collected audio signal in the first application, and executes S601.

[0312] When the target sound source continues to be identified, the electronic device 100 continues to store the collected audio signal in the first application and executes S604.

[0313] In other embodiments, when the electronic device 100 is connected to a Bluetooth headset, the Bluetooth headset can collect audio through the microphone on the Bluetooth headset and send the audio to the electronic device 100 through the Bluetooth connection. The electronic device 100 can also send the audio to the Bluetooth headset through the Bluetooth connection, and the Bluetooth headset plays the audio sent by the electronic device 100 through the speaker on the Bluetooth headset.

[0314] Figure 7D-7F A schematic diagram showing another group of electronic devices 100 recognizing near-field speech and automatically collecting audio.

[0315] In some embodiments, the electronic device 100 may save the audio collected by the first application in the first application, or convert the audio collected by the first application into text and save the text in the first application.

[0316] Exemplarily, the first application may be a memo application.

[0317] For example, Fig.7D As shown, the user can pick up the Bluetooth headset and put the Bluetooth headset close to the user's mouth, and detect a voice signal. When the Bluetooth headset is detected to be in a handheld state and a voice signal is detected, the Bluetooth headset can be determined to be in the handset mode. In response to the Bluetooth headset being in the handset mode, the Bluetooth headset can collect audio signals and send the audio signals to the electronic device 100 that establishes a Bluetooth connection therewith, and the audio signals collected by the Bluetooth headset are saved in the first application through the electronic device 100.

[0318] In some embodiments, the Bluetooth headset may also include multiple microphones and multiple speakers, and the Bluetooth headset may also determine whether there is near-field speech in a manner similar to the above-mentioned electronic device 100. After determining that there is near-field speech, the Bluetooth headset may collect audio signals and send the audio signals to the electronic device 100 with which the Bluetooth connection is established, and the audio signals collected by the Bluetooth headset are saved in the first application through the electronic device 100.

[0319] Optionally, before the Bluetooth headset sends the collected audio signal to the electronic device 100, the electronic device 100 may be displayed on any user interface. Figure 4B As shown, the electronic device 100 can display a desktop.

[0320] In response to the audio signal sent by the Bluetooth headset, the electronic device 100 can automatically open the memo application and display the Fig. 7E User interface 7100 is shown.

[0321] like Fig. 7EAs shown, a prompt message 7101 is displayed on the user interface 4100, and the prompt message 7101 includes the text "Bluetooth headset near-field voice mode". The prompt message 7101 is used to prompt the user that the near-field voice mode is currently being identified through the Bluetooth headset.

[0322] In some embodiments, before the Bluetooth headset sends the collected audio signal to the electronic device 100, the Bluetooth headset can further confirm whether the target sound source is identified. After the target sound source is identified, the Bluetooth headset sends the collected audio signal to the electronic device 100. This can improve the accuracy of confirming that the user uses the audio collection function of the Bluetooth headset.

[0323] In a possible implementation, the Bluetooth headset can send an ultrasonic signal and confirm whether the target sound source is identified through the reflected ultrasonic signal and the transmitted ultrasonic signal. For example, if the user is talking, the ultrasonic signal hits the user's mouth, and the mouth movement causes the frequency and / or amplitude of the ultrasonic signal to change. The Bluetooth headset can identify the target sound source based on the characteristics of the reflected ultrasonic signal.

[0324] Not limited to ultrasonic signals, Bluetooth headsets can also determine the target sound location through other methods, which is not limited in this application.

[0325] In some embodiments, after the electronic device 100 receives the audio signal sent by the Bluetooth headset, the electronic device 100 may not display Fig. 7E The user interface 7100 shown directly controls the memo application to start saving the audio and displays Figure 4D User interface 4200 is shown.

[0326] Afterwards, the memo application can display the real-time received audio signal, and after the user stops outputting audio, save the received audio signal in the memo application, or convert the received audio signal into text information and save the text information in the memo application. The user can also view the saved audio signal or the text information corresponding to the audio signal in the memo application. For details, please refer to Figure 4E-Figure 4J The description in the embodiments will not be repeated in this application.

[0327] In some embodiments, the electronic device 100 may send the audio signal sent by the Bluetooth headset to the electronic device 200 through the first application, or convert the audio signal sent by the Bluetooth headset into text and then send it to the electronic device 200 through the first application.

[0328] Exemplarily, the first application may be an instant messaging application.

[0329] You can refer to Fig.7DAs described in the embodiment, the user can pick up the Bluetooth headset, bring the Bluetooth headset close to the user's mouth, and detect a voice signal. When the Bluetooth headset is detected to be close to the user's mouth and a voice signal is detected, the Bluetooth headset can be determined to be in the handset mode. In response to the Bluetooth headset being in the handset mode, the Bluetooth headset can collect an audio signal and send the audio signal to the electronic device 100 that establishes a Bluetooth connection therewith, and the audio signal collected by the Bluetooth headset is saved in the first application through the electronic device 100.

[0330] Optionally, before the electronic device 100 receives the audio signal sent by the Bluetooth headset, the electronic device 100 may display a chat interface of the contact. Figure 4K As shown, the electronic device 100 can display a chat interface of the contact "Lisa".

[0331] In response to the audio signal sent by the Bluetooth headset, the electronic device 100 can display Figure 7F Prompt bar 7200 is shown.

[0332] like Figure 7F As shown, prompt information 7201 is displayed on the prompt bar 7200, and the prompt information 7201 includes the text "Bluetooth headset near-field voice mode". The prompt information 7201 is used to prompt the user that the near-field voice mode is currently being identified through the Bluetooth headset.

[0333] In some embodiments, before the Bluetooth headset sends the collected audio signal to the electronic device 100, the Bluetooth headset can further confirm whether the target sound source is identified. After the target sound source is identified, the Bluetooth headset sends the collected audio signal to the electronic device 100. This can improve the accuracy of confirming that the user uses the audio collection function of the Bluetooth headset.

[0334] In a possible implementation, the Bluetooth headset can send an ultrasonic signal and confirm whether the target sound source is identified through the reflected ultrasonic signal. If the user is speaking, the ultrasonic signal hits the user's mouth, and the mouth movement will cause the frequency and / or amplitude of the ultrasonic signal to change. The Bluetooth headset can identify the target sound source based on the characteristics of the reflected ultrasonic signal.

[0335] Not limited to ultrasonic signals, Bluetooth headsets can also determine the target sound location through other methods, which is not limited in this application.

[0336] In some embodiments, after the electronic device 100 receives the audio signal sent by the Bluetooth headset, the electronic device 100 may not display Figure 7F Prompt message 7201 is shown.

[0337] After receiving the audio signal sent by the Bluetooth headset, the instant messaging application can display the real-time received audio signal, and after the user stops outputting the audio, send the received audio signal to the electronic device 200 through the first application, or convert the audio signal sent by the Bluetooth headset into text, and then send it to the electronic device 200 through the first application. Figure 4K-Figure 4N The description in the embodiments will not be repeated in this application.

[0338] Figure 8 A schematic diagram of the functional modules of another audio acquisition method provided in this application.

[0339] like Figure 8 As shown, the functional modules on the Bluetooth headset may include but are not limited to: a voice pickup unit, a central control unit, a near-field pattern recognition unit, a near-field voice separation unit, a Bluetooth communication unit, etc. The functional modules on the electronic device 100 may include but are not limited to: an interaction unit, a Bluetooth communication unit.

[0340] The voice pickup unit is used to pick up audio signals periodically / irregularly and send the picked up audio signals to the central control unit.

[0341] The central control unit is used to receive the audio signal sent by the voice pickup unit, process the audio signal, and determine whether the Bluetooth headset is in the handset mode.

[0342] The Bluetooth headset being in the handset mode may mean that: the Bluetooth headset recognizes a voice signal and the Bluetooth headset is in a handheld state.

[0343] In some embodiments, a sensor is pre-installed on the Bluetooth headset, and whether the Bluetooth headset is in a handheld state can be determined based on sensor data collected by the sensor.

[0344] In other embodiments, whether the Bluetooth headset is in the handheld state may also be determined based on the motion trajectory of the Bluetooth headset.

[0345] Whether the Bluetooth headset is in the handheld state may also be determined based on other methods, which is not limited in this application.

[0346] In some embodiments, the central control unit is specifically used to determine the probability of a voice signal existing in the audio signal, and when the probability of a voice signal existing in the audio signal is greater than a preset value, the central control unit determines that a voice signal is recognized. In this way, noise interference can be avoided.

[0347] The central control unit is also used to send the audio signal to the near-field mode recognition unit after determining that the Bluetooth headset is in the handset mode.

[0348] The near-field pattern recognition unit is used to receive the audio signal sent by the central control unit, process the audio signal, and determine whether there is near-field speech.

[0349] The central control unit is also used to control the speaker to emit an ultrasonic signal when the near-field mode recognition unit determines that there is near-field speech in the audio signal to determine whether the target sound source is recognized, thereby improving the accuracy of the Bluetooth headset entering the near-field speech mode. This application can emit ultrasonic signals through the speaker module without adding other devices.

[0350] The voice pickup unit is also used to control the speaker in the central control unit to emit an ultrasonic signal, receive the reflected ultrasonic signal, and send the reflected ultrasonic signal to the central control unit.

[0351] The central control unit is also used to confirm whether the target sound-emitting object is identified based on the characteristics of the emitted ultrasonic signal and the reflected ultrasonic signal. For how the central control unit identifies the target sound-emitting object, please refer to Figure 6 The description of S603 in the embodiment is not repeated in this application.

[0352] The central control unit is also used to send a message of identifying the target sound-emitting object to the near-field pattern recognition unit after identifying the target sound-emitting object.

[0353] Optional, Figure 5 Steps 6 to 10 shown in the embodiment may not be performed, and this application does not limit this.

[0354] The near-field pattern recognition unit is also used to send an audio signal to the near-field speech separation unit in response to a message sent by the central control unit that a target sound-emitting object has been recognized.

[0355] The near-field speech separation unit is used to separate the near-field speech from the audio signal after receiving the audio signal sent by the near-field pattern recognition unit.

[0356] The near-field voice separation unit is also used to send the near-field voice to the Bluetooth communication unit on the Bluetooth headset.

[0357] The Bluetooth communication unit on the Bluetooth headset is used to send near-field voice to the Bluetooth communication unit on the electronic device 100 .

[0358] The Bluetooth communication unit on the Bluetooth headset is also used to send a first instruction to the interaction unit on the electronic device 100 after receiving the near-field voice sent by the Bluetooth communication unit on the electronic device 100.

[0359] The interaction unit on the electronic device 100 is used to display the audio collection interaction interface after receiving the first instruction sent by the Bluetooth communication unit on the electronic device 100, so as to prompt the user that the current Bluetooth headset is in the near-field audio collection mode.

[0360] Optionally, the interaction unit may be a functional module in the first application on the electronic device 100 .

[0361] Optionally, the near-field speech separation unit may also be a functional module on the electronic device 100, which is not limited in this application.

[0362] Fig. 9 A schematic diagram of a method flow of another audio acquisition method provided in this application.

[0363] S901: The electronic device 100 establishes a Bluetooth communication connection with the Bluetooth headset.

[0364] After the electronic device 100 and the Bluetooth headset establish a Bluetooth communication connection, the Bluetooth headset can collect audio through the microphone on the Bluetooth headset and send the audio to the electronic device 100 through the Bluetooth connection. The electronic device 100 can also send audio to the Bluetooth headset through the Bluetooth connection, and the Bluetooth headset plays the audio sent by the electronic device 100 through the speaker on the Bluetooth headset.

[0365] S902. The Bluetooth headset collects audio signals periodically or irregularly.

[0366] Optionally, one or more microphones are pre-installed on the Bluetooth headset, and the Bluetooth headset can collect audio signals periodically / irregularly through the one or more microphones.

[0367] S903: The Bluetooth headset needs to determine whether the Bluetooth headset is in receiver mode.

[0368] The Bluetooth headset being in the handset mode may mean that: the Bluetooth headset recognizes a voice signal and / or the Bluetooth headset is in a handheld state.

[0369] In some embodiments, a sensor is pre-installed on the Bluetooth headset, and whether the Bluetooth headset is in a handheld state can be determined based on sensor data collected by the sensor.

[0370] In other embodiments, whether the Bluetooth headset is in the handheld state may also be determined based on the motion trajectory of the Bluetooth headset.

[0371] Whether the Bluetooth headset is in the handheld state may also be determined based on other methods, which is not limited in this application.

[0372] In some embodiments, the Bluetooth headset recognizes a voice signal, which may refer to the Bluetooth headset determining the probability of the voice signal existing in the audio signal. For example, when the probability of determining that the voice signal exists in the audio signal is greater than a first threshold (e.g., 80%), the Bluetooth headset can determine that the voice signal is recognized, thereby avoiding noise interference.

[0373] In some embodiments, the Bluetooth headset recognizes a voice signal, which may also mean that the Bluetooth headset recognizes that the energy value of the collected audio signal is greater than a preset value.

[0374] When the Bluetooth headset is in the handset mode, execute S904.

[0375] When the Bluetooth headset is not in the handset mode, S902 is executed, and the Bluetooth headset continues to collect audio and monitors whether it is in the handset mode to determine whether the Bluetooth headset enters the near-field audio collection mode.

[0376] Optionally, S903 may also be executed by the electronic device 100. After the electronic device 100 determines that the Bluetooth headset is in the handset mode, the electronic device 100 sends a message to the Bluetooth headset indicating that the Bluetooth headset is in the handset mode to reduce the amount of calculation of the Bluetooth headset.

[0377] Optionally, the Bluetooth headset may not execute S903.

[0378] S904: The Bluetooth headset needs to determine whether the audio signal includes near-field voice.

[0379] After the Bluetooth headset collects the audio signal and determines that the Bluetooth headset is in the handset mode, the Bluetooth headset needs to further determine whether the audio signal includes near-field speech to improve the accuracy of the Bluetooth headset entering the near-field audio collection mode.

[0380] In some embodiments, when the Bluetooth headset recognizes that the energy value of the collected audio signal is greater than a preset value, the Bluetooth headset determines whether the audio signal includes near-field speech.

[0381] Among them, near-field speech may refer to an audio signal emitted by a target sound-emitting object within a first preset distance from the Bluetooth headset. Optionally, the Bluetooth headset is pre-installed with multiple microphones, and the time and energy values ​​of the same audio signal emitted by the same target sound-emitting object arriving at the multiple microphones are significantly different. Based on the difference in the time and energy values ​​of the same audio signal arriving at the multiple microphones, it can be determined whether the audio signal includes near-field speech.

[0382] The specific implementation of how the Bluetooth headset recognizes near-field voice is similar to the specific implementation of how the electronic device 100 recognizes near-field voice. For details, please refer to Figure 6 The description of S602 in the embodiment is not repeated in this application.

[0383] In a case where it is determined that the audio signal includes near-field speech, S905 is performed.

[0384] If it is determined that the audio signal does not include near-field speech, execute S902 or S903, the Bluetooth headset continues to collect audio and monitors whether it is in the handset mode and whether near-field speech is recognized to determine whether the Bluetooth headset enters the near-field audio collection mode.

[0385] Optionally, the Bluetooth headset may not execute S904.

[0386] Optionally, S904 may also be executed by the electronic device 100. After the electronic device 100 recognizes that the audio signal includes near-field speech, the electronic device 100 sends a message to the Bluetooth headset indicating that the near-field speech has been recognized, so as to reduce the amount of calculation of the Bluetooth headset.

[0387] S905: The Bluetooth headset needs to determine whether the target sound source is recognized.

[0388] After determining that the audio signal includes near-field speech, before collecting the audio signal, the Bluetooth headset needs to determine whether the target sound source is identified. After identifying the target sound source, the Bluetooth headset collects the audio signal. In this way, the accuracy of the Bluetooth headset entering the near-field speech mode can be improved, and the occurrence of accidental touch can be further prevented.

[0389] In some embodiments, when the Bluetooth headset recognizes that the energy value of the collected audio signal is greater than a preset value, the Bluetooth headset determines whether the target sound source is recognized.

[0390] In some embodiments, after determining that the audio signal is near-field speech, the Bluetooth headset can emit an ultrasonic signal with preset characteristics, and determine whether the target sound source is recognized based on the characteristics of the reflected ultrasonic signal.

[0391] Identifying the target sound-emitting part may mean that: the distance between the Bluetooth headset and the target sound-emitting object is within a second preset distance (for example, 10 cm), and the vibration frequency of the target sound-emitting part is within a first range. Optionally, the second preset distance may be less than the first preset distance. Optionally, the second preset distance may also be equal to the first preset distance. After determining the target sound-emitting object, it is necessary to further determine whether the distance between the target sound-emitting part of the target sound-emitting object and the Bluetooth headset is close enough, and whether the target sound-emitting part is active. Only when the distance is close enough and the target sound-emitting part is active, can it be determined that the target sound-emitting part is making a sound, and the Bluetooth headset starts collecting audio, which can further prevent the sound from being accidentally touched.

[0392] In some embodiments, after determining that the audio signal is near-field speech, the Bluetooth headset can emit a first ultrasonic signal and receive a reflected second ultrasonic signal. The Bluetooth headset can determine the second distance between the first electronic device and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the first ultrasonic signal and the second ultrasonic signal. When the second distance is less than the second preset distance and the vibration frequency of the target sound-emitting part of the target sound-emitting object is within the first range, the Bluetooth headset can determine that the target sound-emitting part is making a sound, and the Bluetooth headset starts collecting audio again, which can further prevent the sound from being accidentally touched.

[0393] The first range may be between the first frequency value and the second frequency value. For example, the first range may be 20 Hz-40 Hz.

[0394] Optionally, when it is determined based on the first ultrasonic signal and the second ultrasonic signal for a continuous first preset time period that the second distance is greater than the second preset distance and / or the vibration frequency of the target sound-emitting part of the target sound-emitting object is not within the first range, the electronic device 100 exits the near-field voice mode and stops collecting audio.

[0395] In some embodiments, after determining that the audio signal is near-field speech, the Bluetooth headset may emit an ultrasonic signal with preset characteristics, and determine whether the target sound source is identified and the distance between the Bluetooth headset and the target sound source based on the characteristics of the reflected ultrasonic signal. Optionally, the preset characteristics of the emitted ultrasonic signal may include but are not limited to a preset frequency, a preset amplitude, a transmission time, etc. The characteristics of the reflected ultrasonic signal may include but are not limited to the frequency and amplitude of the reflected ultrasonic signal, the reflection time, etc.

[0396] Because when the target sound-producing part (such as the mouth) outputs audio, the target sound-producing part moves at a certain frequency and amplitude. The ultrasonic signal hits the moving target sound-producing part and is reflected, causing the frequency and amplitude of the ultrasonic signal to change. For example, the frequency and amplitude of the ultrasonic signal will change.

[0397] The Bluetooth headset can determine the distance between the Bluetooth headset and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the emitted ultrasonic signal and the reflected ultrasonic signal.

[0398] Optionally, the Bluetooth headset may determine the distance between the Bluetooth headset and the target sound-emitting object based on the difference between the time when the ultrasonic signal is emitted and the time when the ultrasonic signal is reflected.

[0399] Optionally, the Bluetooth headset can determine the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the difference between the frequency and / or amplitude of the emitted ultrasonic signal and the frequency and / or amplitude of the reflected ultrasonic signal.

[0400] Exemplarily, the Bluetooth headset can determine the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the frequency and amplitude of the ultrasonic signal reflected by the frequency and amplitude of the ultrasonic signal emitted, and compare it with the frequency and amplitude of the ultrasonic signal emitted by the Bluetooth headset, and determine whether the target sound-emitting part is recognized based on the vibration frequency of the target sound-emitting part. Exemplarily, when the vibration frequency of the target sound-emitting part is within the first range, it can be determined that the target sound-emitting part is recognized.

[0401] Exemplarily, when the difference between the frequency and amplitude of the reflected ultrasonic signal and the frequency and amplitude of the ultrasonic signal emitted by the Bluetooth headset meets a preset value, for example, is greater than a preset value, the Bluetooth headset can determine that the target sound source is identified.

[0402] If the ultrasonic signal hits an inactive object, the frequency and amplitude of the ultrasonic signal will basically not change. The difference between the frequency and amplitude of the reflected ultrasonic signal and the frequency and amplitude of the ultrasonic signal emitted by the Bluetooth headset does not meet the preset value, for example, is less than the preset value. The Bluetooth headset can determine that the target sound source is not identified.

[0403] The specific implementation of how the Bluetooth headset identifies the target sound source is similar to the specific implementation of how the electronic device 100 identifies the target sound source. For details, please refer to Figure 6 The description of S603 in the embodiment is not repeated in this application.

[0404] When the target vocalization site is identified, S906 is executed.

[0405] If the target sound source is not identified, execute S902 or S903 or S904, the Bluetooth headset continues to collect audio and monitors whether it is in the earpiece mode and whether the near-field voice is recognized and whether the target sound source is recognized to determine whether the Bluetooth headset enters the near-field audio collection mode.

[0406] Optionally, the Bluetooth headset may not execute S905.

[0407] Optionally, the Bluetooth headset can execute any one, any two, or all of S905, S906, and S907, which is not limited in this application.

[0408] S906 . The Bluetooth headset sends the collected audio signal to the electronic device 100 .

[0409] After determining that the Bluetooth headset is in the handset mode, or after determining near-field speech, or after identifying the target sound source, the electronic device 100 can collect audio signals and send the collected audio signals to the electronic device 100 via the Bluetooth connection.

[0410] In some embodiments, after collecting the audio signal, the Bluetooth headset needs to process the audio signal, filter to obtain the near-field voice signal, and filter out the far-field voice signal, so as to avoid interference from the far-field voice signal.

[0411] Based on the description in S904, the difference between near-field speech and far-field speech is that when the distance between the target sound-emitting object and the Bluetooth headset is within the first preset distance, the energy value and time when the same audio emitted by the same sound-emitting object reaches multiple microphones on the Bluetooth headset are significantly different, and the current audio can be judged as near-field speech based on this. When the distance between the target sound-emitting object and the Bluetooth headset exceeds the first preset distance, that is, the target sound-emitting object is far away from the Bluetooth headset, the time and energy value when the same audio emitted by the same target sound-emitting object reaches multiple microphones on the Bluetooth headset are not significantly different, and the current audio can be judged as far-field speech based on this.

[0412] Exemplarily, energy values ​​of the same audio signal received by multiple microphones on a Bluetooth headset are obtained, and the difference between any two of the multiple energy values ​​satisfies a preset energy difference, for example, is greater than the preset energy difference, and / or, the moments of the same audio signal received by multiple microphones on the Bluetooth headset are obtained, and the difference between any two of the multiple moments satisfies a preset time difference, for example, is greater than the preset time difference, then it can be determined that the audio is near-field speech.

[0413] Exemplarily, energy values ​​of the same audio signal received by multiple microphones on a Bluetooth headset are obtained, and the difference between any two of the multiple energy values ​​does not satisfy a preset energy difference, for example, is less than a preset energy difference, and / or the moments of the same audio signal received by multiple microphones on the Bluetooth headset are obtained, and the difference between any two of the multiple moments does not satisfy a preset time difference, for example, is less than a preset time difference, then it can be determined that the audio is far-field speech.

[0414] Based on the above features, near-field speech can be separated from the audio signal collected by the Bluetooth headset.

[0415] Optionally, the step of filtering the audio signal collected by the Bluetooth headset to obtain a near-field voice signal may also be performed by the electronic device 100, which is not limited in the present application.

[0416] The specific implementation of how the Bluetooth headset separates the near-field voice from the collected audio signal is similar to the specific implementation of how the electronic device 100 separates the near-field voice from the collected audio signal. For details, please refer to Figure 6 The description of S604 in the embodiment is not repeated in this application.

[0417] S907: The electronic device 100 saves the audio signal in the first application.

[0418] In response to the audio signal sent by the Bluetooth headset, the electronic device 100 saves the audio signal in the first application.

[0419] The electronic device 100 saves the collected audio signal in the first application, which may include: the electronic device 100 directly saves the audio signal in the first application. Alternatively, the electronic device 100 converts the audio signal into text information and saves the text information in the first application. For example, the first application may be a memo application. For details, refer to Figure 7B , Figure 4D-Figure 4J Description in the examples.

[0420] The electronic device 100 stores the collected audio signal in the first application, and may also include: the electronic device 100 sends the audio signal to the electronic device 200 through the first application. Alternatively, the electronic device 100 converts the audio signal into text information, and sends the text information to the electronic device 200 through the first application. For example, the first application may be an instant messaging application. For details, refer to Figure 7C , Figure 4M-Figure 4N Description in the examples.

[0421] S908: The Bluetooth headset continues to monitor whether the Bluetooth headset recognizes the target sound source.

[0422] After the Bluetooth headset sends the collected audio signal to the electronic device 100, the Bluetooth headset needs to continue to monitor whether the Bluetooth headset recognizes the target sound source.

[0423] If the target sound source is not identified, the Bluetooth headset exits the near-field voice mode and stops sending audio signals to the electronic device 100 , and S902 is executed.

[0424] When the target sound source continues to be identified, the Bluetooth headset continues to send the collected audio signal to the electronic device 100 and executes S906.

[0425] Fig.10 A flowchart of another audio acquisition method provided in this application.

[0426] S1001. A first electronic device collects a first audio signal output by a target sound-emitting object through a microphone array.

[0427] S1002: The first electronic device determines a first distance between the first electronic device and a target sound-emitting object based on a first audio signal.

[0428] S1003: When the first distance is less than a first preset distance, the first electronic device obtains a second audio signal output by the target sound-emitting object through the microphone array.

[0429] S1004: The first electronic device saves the second audio signal in the first application.

[0430] Optionally, the electronic device may store the first audio signal or may not store the first audio signal.

[0431] Optionally, the first electronic device stores the second audio signal in the first application. The first electronic device may store the second audio signal in the first application directly, or convert the second audio signal into text information and then store the text information in the first application.

[0432] The electronic device can determine the first distance between the electronic device and the target sound-emitting object based on the collected audio signal. When the first distance between the electronic device and the target sound-emitting object is greater than the first preset distance, or the first distance between the electronic device and the target sound-emitting object is greater than the first preset distance for a certain period of time, the electronic device can determine that near-field speech is recognized. The electronic device can automatically start collecting audio and automatically save the audio in the first application. The electronic device automatically picks up audio, reduces user operations, improves audio collection efficiency, and enhances user experience.

[0433] In a possible implementation manner, the first electronic device saves the second audio signal in the first application, specifically including:

[0434] In response to the first distance being less than the first preset distance, the first electronic device starts the first application and saves the second audio signal in the first application, or the first electronic device converts the second audio signal into a first text message and saves the first text message in the first application.

[0435] In this way, after recognizing the near-field voice, the electronic device can automatically save the collected audio signal or the text information corresponding to the collected audio signal in the first application.

[0436] Exemplarily, the first application may be a memo application.

[0437] In one possible implementation, the first electronic device saves the second audio signal in the first application, specifically including: in response to the first distance being less than the first preset distance, the first electronic device opens the first application, and sends the second audio signal to the second electronic device through the first application, or the first electronic device converts the second audio signal into a first text message, and sends the first text message to the second electronic device through the first application.

[0438] In this way, after recognizing the near-field voice, the electronic device can automatically send the collected audio signal to the second electronic device with which the communication connection is established, or send the text information corresponding to the collected audio signal to the second electronic device with which the communication connection is established.

[0439] Exemplarily, the first application may be an instant messaging application.

[0440] In a possible implementation, the first electronic device is connected to a third electronic device via Bluetooth; the first electronic device saves the second audio signal in the first application, specifically including: the first electronic device sends the second audio signal to the third electronic device via Bluetooth connection; the third electronic device saves the second audio signal in the first application.

[0441] Optionally, the first electronic device may be a Bluetooth headset. After recognizing the near-field voice, the Bluetooth headset may automatically send the collected audio signal to a third electronic device that has established a Bluetooth connection, or send text information corresponding to the collected audio signal to the third electronic device that has established a Bluetooth connection.

[0442] In one possible implementation, while the first electronic device obtains the second audio signal through the microphone array, the first electronic device also obtains the third audio signal output by other sound-emitting objects through the microphone array; the electronic device saves the second audio signal in the first application, specifically including: when the first electronic device determines that the distance between the first electronic device and the other sound-emitting objects exceeds a first preset distance, the electronic device saves the second audio signal in the first application.

[0443] Optionally, the second audio signal and the third audio signal may be audio signals in the same audio file. The electronic device 100 may extract the second audio signal or the third audio signal from the same audio file.

[0444] Optionally, the second audio signal and the third audio signal may also be audio signals in two audio files respectively.

[0445] Near-field voice may refer to the audio outputted by a target sound-emitting object within a first preset distance from the electronic device. Far-field voice may refer to the audio outputted by a target sound-emitting object beyond the first preset distance from the electronic device.

[0446] In this way, after recognizing near-field speech and starting to collect audio signals, the electronic device can determine whether the collected audio signal is a near-field audio signal or a far-field audio signal, and only save the near-field audio signal without saving the far-field audio signal, thereby avoiding interference from the far-field audio signal.

[0447] For details on how electronic devices distinguish between near-field speech and far-field speech, please refer to formula (1) to formula (4).

[0448] In a possible implementation, when the first distance is less than a first preset distance, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array, specifically including: the first electronic device determines the probability of a voice signal contained in the first audio signal; when the first distance is less than the first preset distance, and when the probability of the voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0449] In this way, before starting to collect audio, the electronic device can determine the probability of the voice signal contained in the audio signal. When the probability of the voice signal contained in the audio signal is greater than the first threshold, that is, the probability of recognizing that the user is speaking is high, the electronic device can collect and save the second audio signal.

[0450] In one possible implementation, a first electronic device is connected to a third electronic device via Bluetooth; the first electronic device obtains a second audio signal output by a target sound-emitting object through a microphone array, specifically including: when the first electronic device determines that the first electronic device is in a handheld state, the first distance is less than a first preset distance, and the probability of a voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0451] Optionally, a sensor is pre-installed on the first electronic device, and whether it is in a handheld state can be confirmed through a sensor signal collected by the sensor.

[0452] In this way, when the first electronic device is a Bluetooth headset, when the Bluetooth headset is in a handheld state, it can be considered that the current user has the intention to speak to the Bluetooth headset. When the Bluetooth headset is in a handheld state, the distance between the Bluetooth headset and the target sound-emitting object is within a first preset distance, and the probability of the voice signal contained in the first audio signal is greater than a first threshold, the Bluetooth headset can enter the near-field voice mode and start to automatically collect and save the audio signal, which can improve the accuracy of the Bluetooth headset entering the near-field voice mode.

[0453] In one possible implementation, the first electronic device includes a speaker; before the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array, the method also includes: the first electronic device emits a first ultrasonic signal through the speaker; the first electronic device receives the reflected second ultrasonic signal; the first electronic device determines the second distance between the first electronic device and the target sound-emitting object and the vibration frequency of the target sound-emitting part of the target sound-emitting object based on the first ultrasonic signal and the second ultrasonic signal; when the second distance is less than the second preset distance and the vibration frequency of the target sound-emitting part of the target sound-emitting object is within the first range, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

[0454] Exemplarily, the first range may be between 20 Hz and 40 Hz.

[0455] Optionally, the second preset distance may be smaller than the first preset distance. Optionally, the second preset distance may also be equal to the first preset distance.

[0456] Optionally, the target sound-emitting object may emit sound at a certain vibration frequency when emitting sound. The first electronic device may obtain the vibration frequency of the target sound-emitting part of the target sound-emitting object through the first ultrasonic signal and the second ultrasonic signal, and determine whether the target sound-emitting object is emitting sound based on the obtained vibration frequency of the target sound-emitting object. When the vibration frequency of the target sound-emitting object is within the first range, it can be considered that the target sound-emitting object is emitting sound.

[0457] In this way, when the first electronic device enters the near-field voice mode, it can be confirmed whether the first electronic device and the target sound-emitting object are close enough and whether the target sound-emitting part of the target sound-emitting object is emitting sound, thereby improving the accuracy of the first electronic device entering the near-field voice mode.

[0458] In one possible implementation, after the first electronic device saves the second audio signal in the first application, the method further includes: when it is determined based on the first ultrasonic signal and the second ultrasonic signal for a continuous first preset time period that the second distance is greater than the second preset distance and / or the vibration frequency of the target sound-emitting part of the target sound-emitting object is not within the first range, the first electronic device stops acquiring the audio signal through the microphone array.

[0459] After the first electronic device enters the near-field voice mode, the first electronic device needs to continuously monitor whether the first electronic device and the target sound-emitting object are close enough, and whether the target sound-emitting part of the target sound-emitting object is making a sound. When it is monitored that the distance between the first electronic device and the target sound-emitting object exceeds the second preset distance, and / or the target sound-emitting part of the target sound-emitting object is not making a sound, the first electronic device can automatically exit the near-field voice mode and stop collecting audio.

[0460] In a possible implementation, the microphone array includes a first microphone and a second microphone, and the first microphone and the second microphone are located at different positions on the first electronic device; the first electronic device collects a first audio signal output by a target sound-emitting object through the microphone array, specifically including: the first electronic device collects the first audio signal output by the target sound-emitting object through the first microphone and the second microphone respectively; the first electronic device determines a first distance between the first electronic device and the target sound-emitting object based on the first audio signal, specifically including: the first electronic device obtains a first energy value of the first audio signal collected by the first microphone and a second energy value of the first audio signal collected by the second microphone; the first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first energy value and the second energy value; and / or, the first electronic device obtains a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; the first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first moment and the second moment.

[0461] In this way, the electronic device can determine the distance between the first electronic device and the target sound-emitting object through the difference in energy values ​​of the same audio signal collected by multiple microphones in the microphone array and / or the difference between the moments of receiving the same audio signal.

[0462] In one possible implementation, the microphone array includes a first microphone and a second microphone, the first microphone and the second microphone are located at different positions on the first electronic device, the speaker includes a first speaker and a second speaker, the first speaker is located near the first microphone, and the second speaker is located near the second microphone; the first electronic device sends a first ultrasonic signal through the speaker, specifically including: when the first electronic device determines that the distance between the first microphone and the target sound-emitting object is less than the distance between the second microphone and the target sound-emitting object, the first electronic device sends the first ultrasonic signal through the first speaker, wherein the first speaker is the speaker closest to the target sound-emitting object.

[0463] In combination with the first aspect, in a possible implementation manner, the first electronic device receives the reflected second ultrasonic signal, which specifically includes: the first electronic device receives the reflected second ultrasonic signal through a first microphone.

[0464] In this way, the first electronic device can send out an ultrasonic signal through a speaker closest to the target sound-emitting object and receive a reflected ultrasonic signal through a microphone closest to the target sound-emitting object, thereby improving the accuracy of identifying the target sound-emitting object.

[0465] In one possible implementation, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object, specifically including: the first electronic device obtains a first energy value of the first audio signal collected by the first microphone and a second energy value of the first audio signal collected by the second microphone; when the first energy value is greater than the second energy value, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object; and / or the first electronic device obtains a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; when the first moment is less than the second moment, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object.

[0466] The above are only some embodiments and implementation methods of the present application, and the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0467] It is understandable that the various user interfaces described in the embodiments of the present application are only example interfaces and do not limit the present application scheme. In other embodiments, the user interface may adopt a different interface layout, may include more or fewer controls, and may increase or decrease other functional options, as long as they are based on the same inventive concept provided by the present application, they are all within the scope of protection of the present application.

[0468] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0469] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio acquisition method, characterized in that: The first electronic device includes a microphone array, the microphone array includes at least two microphones, and the method includes: The first electronic device collects a first audio signal output by a target sound-emitting object through the microphone array; The first electronic device determines, based on the first audio signal, a first distance between the first electronic device and the target sound-emitting object; When the first distance is less than a first preset distance, the first electronic device acquires a second audio signal output by the target sound-emitting object through the microphone array; The first electronic device saves the second audio signal in a first application.

2. The method according to claim 1, characterized in that The first electronic device saves the second audio signal in a first application, specifically including: In response to the first distance being less than the first preset distance, the first electronic device opens a first application and saves the second audio signal in the first application, or the first electronic device converts the second audio signal into a first text message and saves the first text message in the first application.

3. The method according to claim 1, characterized in that The first electronic device saves the second audio signal in a first application, specifically including: In response to the first distance being less than the first preset distance, the first electronic device opens a first application and sends the second audio signal to the second electronic device through the first application, or the first electronic device converts the second audio signal into a first text message and sends the first text message to the second electronic device through the first application.

4. The method according to claim 1, characterized in that: The first electronic device is connected to a third electronic device via Bluetooth; and the first electronic device stores the second audio signal in a first application, specifically including: The first electronic device sends the second audio signal to the third electronic device via the Bluetooth connection; The third electronic device saves the second audio signal in the first application.

5. The method according to any one of claims 1 to 4, characterized in that: While the first electronic device acquires the second audio signal through the microphone array, the first electronic device also acquires a third audio signal output by other sound-emitting objects through the microphone array; and the electronic device saves the second audio signal in the first application, specifically including: When the first electronic device determines that the distance between the first electronic device and the other sound-emitting object exceeds the first preset distance, the electronic device saves the second audio signal in the first application.

6. The method according to any one of claims 1 to 5, characterized in that: When the first distance is less than a first preset distance, the first electronic device acquires a second audio signal output by the target sound-emitting object through the microphone array, specifically including: The first electronic device determines a probability of a speech signal contained in the first audio signal; When the first distance is less than a first preset distance and the probability of the voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

7. The method according to claim 6, characterized in that The first electronic device is connected to a third electronic device via Bluetooth; the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array, specifically including: When the first electronic device determines that the first electronic device is in a handheld state, the first distance is less than a first preset distance, and the probability of a voice signal contained in the first audio signal is greater than a first threshold, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

8. The method according to any one of claims 1 to 7, characterized in that: The first electronic device includes a speaker; before the first electronic device acquires the second audio signal output by the target sound-emitting object through the microphone array, the method further includes: The first electronic device sends a first ultrasonic signal through the speaker; The first electronic device receives the reflected second ultrasonic signal; The first electronic device determines, based on the first ultrasonic signal and the second ultrasonic signal, a second distance between the first electronic device and the target sound-emitting object and a vibration frequency of a target sound-emitting part of the target sound-emitting object; When the second distance is less than a second preset distance and the vibration frequency of the target sound-emitting part of the target sound-emitting object is within a first range, the first electronic device obtains the second audio signal output by the target sound-emitting object through the microphone array.

9. The method according to claim 8, characterized in that After the first electronic device saves the second audio signal in a first application, the method further includes: When it is determined based on the first ultrasonic signal and the second ultrasonic signal for a first preset time period that the second distance is greater than the second preset distance and / or the vibration frequency of the target sound-emitting part of the target sound-emitting object is not within the first range, the first electronic device stops acquiring audio signals through the microphone array.

10. The method according to any one of claims 1 to 9, characterized in that: The microphone array includes a first microphone and a second microphone, and the first microphone and the second microphone are located at different positions on the first electronic device; The first electronic device collects a first audio signal output by a target sound-emitting object through the microphone array, specifically including: The first electronic device collects the first audio signal output by the target sound-emitting object through the first microphone and the second microphone respectively; The first electronic device determines a first distance between the first electronic device and the target sound-emitting object based on the first audio signal, specifically including: Acquiring, by the first electronic device, a first energy value of the first audio signal acquired by the first microphone and a second energy value of the first audio signal acquired by the second microphone; The first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first energy value and the second energy value; and / or, The first electronic device acquires, by the first microphone, a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; The first electronic device determines the first distance between the first electronic device and the target sound-emitting object based on the difference between the first moment and the second moment.

11. The method according to claim 8 or 9, characterized in that: The microphone array includes a first microphone and a second microphone, the first microphone and the second microphone are located at different positions on the first electronic device, the speaker includes a first speaker and a second speaker, the first speaker is located near the first microphone, and the second speaker is located near the second microphone; The first electronic device sends a first ultrasonic signal through the speaker, specifically including: When the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object, the first electronic device emits the first ultrasonic signal through the first speaker, wherein the first speaker is the speaker closest to the target sound-emitting object.

12. The method according to claim 11, characterized in that The first electronic device receives the reflected second ultrasonic signal, specifically including: The first electronic device receives the reflected second ultrasonic signal through the first microphone.

13. The method according to claim 11 or 12, characterized in that: The first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object, specifically including: Acquiring, by the first electronic device, a first energy value of the first audio signal acquired by the first microphone and a second energy value of the first audio signal acquired by the second microphone; When the first energy value is greater than the second energy value, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is less than the distance between the second microphone and the target sound-emitting object; and / or, The first electronic device acquires, by the first microphone, a first moment of the first audio signal collected by the first microphone and a second moment of the first audio signal collected by the second microphone; When the first moment is smaller than the second moment, the first electronic device determines that the distance between the first microphone and the target sound-emitting object is smaller than the distance between the second microphone and the target sound-emitting object.

14. An electronic device, characterized in that: The electronic device includes a microphone array, a memory, and a processor; wherein the microphone array, the memory, and the processor are coupled, the memory is used to store a computer program, and when the processor executes and calls the computer program, the electronic device executes the method described in any one of claims 1-13.

15. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 13.

Images