Voice interaction method and device and first electronic equipment

Abstract

The embodiment of the invention provides a voice interaction method and device and first electronic equipment. The method is applied to first electronic equipment and comprises the following steps: acquiring an environment sound signal; when it is judged that the environment sound signal comprises a sound signal input by the user within the set distance range, voice recognition is carried out on the sound signal, and a voice instruction is generated; triggering voice interaction in response to the voice instruction; or, a voice instruction is sent to the second electronic device, and the voice instruction is used for triggering the second electronic device to conduct voice interaction. According to the technical scheme provided by the embodiment of the invention, whether the environment sound signal comprises the sound signal input by the user in the set distance range or not is judged, and the sound signal input by the user in the set distance range is subjected to voice recognition to trigger voice interaction; the light-sound wakeup-free function of the electronic equipment in a near-field voice interaction scene can be realized.

Description

[Technical field]

[0001] The present application relates to the field of electronic technology, and in particular to a voice interaction method, device and a first electronic device. [Background technology]

[0002] Voice is a common way to carry information in daily life. With the development of voice technology, more and more voice recognition algorithms have emerged to perform voice recognition. According to the distance of the voice source, it is divided into far-field voice and near-field voice. When performing voice recognition, different voice interaction methods can be used for different voice source distances to obtain clear and recognizable voice information.

[0003] In near-field voice interaction scenarios, users generally trigger the voice interaction function of the electronic device by manually clicking the wake-up button of the electronic device or saying a specific wake-up word. The general wake-up-free function uses directional voice pickup and can only respond to sounds within a certain angle range of the electronic device. It cannot realize the gentle wake-up function of the electronic device in the near-field voice interaction scenario. [Summary of the invention]

[0004] In view of this, an embodiment of the present application provides a voice interaction method, an apparatus and a first electronic device to implement a soft voice wake-up function for an electronic device in a near-field voice interaction scenario.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a voice interaction method, which is applied to a first electronic device, and the method includes:

[0007] Acquire environmental sound signals;

[0008] When it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal to generate a voice command;

[0009] The voice interaction is triggered in response to the voice instruction; or, a voice instruction is sent to a second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0010] According to the voice interaction method provided in the present application, by determining whether the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal input by the user within the set distance range to trigger voice interaction, thereby realizing a silent wake-up function for electronic devices in near-field voice interaction scenarios.

[0011] In an implementation of the first aspect, determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes:

[0012] Identify one or more speakers corresponding to the ambient sound signal;

[0013] Generate the pickup distance corresponding to each speaker according to the acquired pickup model;

[0014] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0015] In an implementation of the first aspect, determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes:

[0016] Generate a pickup distance corresponding to one or more speakers according to the ambient sound signal;

[0017] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0018] Furthermore, when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal meets the conditions of near-field spectrum analysis, and it can be judged that the current ambient sound signal includes the sound signal input by the user within the set distance range based on the distance between the speaker's sound body and the first electronic device. The speaker who is in the near-field voice interaction condition can be judged from multiple speakers, which can improve the accuracy of subsequent voice recognition.

[0019] In an implementation of the first aspect, generating sound pickup distances corresponding to one or more speakers according to an ambient sound signal specifically includes:

[0020] generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal;

[0021] Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or,

[0022] The sound pickup distance corresponding to one or more speakers is generated according to the sound pressure information and the sound speed information.

[0023] Furthermore, the sound pickup distance of the speaker can be accurately obtained according to the sound pressure information and / or the sound speed information.

[0024] In an implementation of the first aspect, performing voice recognition on a sound signal to generate a voice command specifically includes:

[0025] Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more speakers to generate a voice command.

[0026] Furthermore, by performing speech recognition on the sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more speakers, the sound signal of the user closest to the first electronic device can be accurately obtained, thereby improving the accuracy of sound pickup.

[0027] In an implementation manner of the first aspect, the voice instruction includes an application operation instruction or a recording instruction.

[0028] In an implementation of the first aspect, before performing speech recognition on the sound signal, the method further includes:

[0029] Perform sound enhancement processing on the ambient sound signal to generate a sound signal.

[0030] Furthermore, performing sound enhancement processing on the ambient sound signal can remove the ambient noise and improve the quality of the sound signal.

[0031] In an implementation manner of the first aspect, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0032] In an implementation of the first aspect, obtaining an ambient sound signal specifically includes:

[0033] An ambient sound signal sent by a second electronic device is received.

[0034] In a second aspect, an embodiment of the present application provides a voice interaction device, which is applied to a first electronic device, and includes:

[0035] Microphone module, used to obtain ambient sound signals;

[0036] A determination module, used to determine whether the ambient sound signal includes a sound signal input by the user within a set distance range;

[0037] A voice recognition module, configured to perform voice recognition on the sound signal and generate a voice command when the judgment module judges that the ambient sound signal includes a sound signal input by the user within a set distance range;

[0038] A voice interaction module is used to trigger voice interaction in response to a voice instruction; or a wireless communication module is used to send a voice instruction to a second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0039] In an implementation manner of the second aspect, the judging module is specifically configured to:

[0040] Identify one or more speakers corresponding to the ambient sound signal;

[0041] Generate the pickup distance corresponding to each speaker according to the acquired pickup model;

[0042] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0043] In an implementation manner of the second aspect, the judging module is specifically configured to:

[0044] Generate a pickup distance corresponding to one or more speakers according to the ambient sound signal;

[0045] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0046] In an implementation manner of the second aspect, the judging module is specifically configured to:

[0047] generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal;

[0048] Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or,

[0049] The sound pickup distance corresponding to one or more speakers is generated according to the sound pressure information and the sound speed information.

[0050] In an implementation of the second aspect, the method further includes:

[0051] When the number of microphones in the microphone module is one, the microphone is a sonic velocity microphone;

[0052] The judgment module is specifically used to generate sound speed information of the ambient sound signal according to the ambient sound signal; and generate sound pickup distances corresponding to one or more speakers according to the sound speed information.

[0053] In an implementation of the second aspect, the method further includes:

[0054] When there are multiple microphones in the microphone module, the microphones include a sound velocity microphone and a sound pressure microphone;

[0055] The judgment module is specifically used to generate sound pressure information and sound speed information of the ambient sound signal according to the ambient sound signal; and generate the sound pickup distance corresponding to one or more speakers according to the sound pressure information and the sound speed information.

[0056] In an implementation manner of the second aspect, the microphone is always turned on to obtain ambient sound signals in real time.

[0057] In an implementation manner of the second aspect, the speech recognition module is specifically used to:

[0058] Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more speakers to generate a voice command.

[0059] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0060] In a fourth aspect, an embodiment of the present application provides a first electronic device, wherein the first electronic device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the method steps described in the first aspect.

Brief Description of the Drawings

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 A schematic diagram of a plosive sound provided in the related art when a user speaks at different distances;

[0063] Figure 2a A schematic diagram of the structure of a voice interaction system provided in one embodiment of the present application;

[0064] Figure 2b A schematic diagram of the structure of another voice interaction system provided in one embodiment of the present application;

[0065] Figure 2c A schematic diagram of the structure of another voice interaction system provided in one embodiment of the present application;

[0066] Figure 2d A schematic diagram of the structure of another voice interaction system provided in one embodiment of the present application;

[0067] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0068] Figure 4a A schematic diagram of a scenario for obtaining an ambient sound signal provided in an embodiment of the present application;

[0069] Figure 4bA schematic diagram of the relationship between sound velocity information, sound pressure information and sound pickup distance provided in an embodiment of the present application;

[0070] Figure 4c A schematic diagram of the relationship between sound speed information, sound pickup distance and sound frequency provided in an embodiment of the present application;

[0071] Figure 5 A flowchart of a voice interaction method provided according to an embodiment of the present application;

[0072] Figure 6 Schematic diagram of the structure of a voice interaction device according to an embodiment of the present application. [Specific implementation method]

[0073] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0074] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0075] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0076] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0077] In a related technology, when receiving far-field speech, the far-field speech is input into a pre-trained neural network-based speech training model, and the audio features of the received far-field speech are de-interferenced by using the audio features of the far-field speech and near-field speech contained in the speech training model to obtain the processed far-field speech and recognize it.

[0078] In another related technology, the far-field speech to be recognized received by the microphone array is processed by adaptive beamforming to obtain a signal to be recognized, and then the signal to be recognized is recognized by the acoustic model of far-field recognition, wherein the acoustic model of far-field recognition can truly match the input speech signal of far-field recognition, thereby improving the recognition performance of far-field speech recognition. The above two related technologies are both for speech recognition in far-field speech interaction scenarios, and cannot solve the technical problem of the soft voice wake-up function of electronic devices in near-field speech interaction scenarios.

[0079] In another related technology, by detecting the speech spectra of pops and microphone bursts when people speak at different distances, it is determined at which distance the user's speech meets the near-field voice interaction scenario. If it meets the near-field voice interaction scenario, the electronic device can be enabled directly without waking up. Figure 1 A schematic diagram of a plosive sound provided in the related art when a user speaks at different distances, such as Figure 1 As shown, the user reads "How is the weather today" at a normal volume at different distances. When the pickup distance is 2cm, 5cm, 10cm, 20cm, 30cm and 50cm, the amplitude of the sound is different between -1 and 1. When the pickup distance is 2cm, the amplitude of the sound is larger, and when the pickup distance is 50cm, the amplitude of the sound is smaller. Since plosive sound refers to the sound produced by the sound organ forming an obstruction in the oral cavity and then the airflow breaking through the obstruction, the amplitude of the sound when the user reads "today" and "weather" is larger, and the amplitude of the sound when reading "how is it" is smaller. In addition, compared with the normal speech sound with a pickup distance of 2 cm, the soft speech with a pickup distance of 2 cm will be regarded as the sound of popular noise and / or airflow, so the soft speech of the user at a pickup distance of 2 cm cannot be identified. In addition, in the related art, it can only be enabled for a certain starting frame of the user's speech, and a certain judgment condition (such as time, maximum energy attenuation, etc.) is used as the end judgment, and accurate frame processing cannot be achieved. For example, when the pickup distance is 2 cm, the probability of enabling after the "weather" sound of "day" ends is 45.8%. According to the above method, the user's voice command cannot be accurately recognized, and voice interaction cannot be accurately performed.

[0080] In the embodiments of the present application, near-field voice interaction refers to a scenario where the pickup distance is less than 10 cm, and far-field voice interaction refers to a scenario where the pickup distance is more than 10 cm. The following embodiments of the present application are applied to the scenario of near-field voice interaction.

[0081] In order to solve the technical problems in the related art, an embodiment of the present application provides a voice interaction system. Figure 2a A schematic diagram of the structure of a voice interaction system provided in one embodiment of the present application is shown in FIG. Figure 2aAs shown, the first electronic device 100 includes a first microphone 170C, a first processor 110 and a first functional module 101, wherein the first functional module 101 may be a speaker, an indicator light, a software application, etc. The first microphone 170C is connected to the first processor 110 for communication, and the first processor 110 is connected to the first functional module 101 for communication. In a possible implementation, the user holds the first electronic device 100, inputs an ambient sound signal through the first microphone 170C, and the first microphone 170C transmits the ambient sound signal to the first processor 110, and the first processor 110 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, the sound signal is voice recognized to generate a voice command. The first processor 110 transmits the voice command to the first functional module 101, and the first functional module 101 triggers voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus, and a first microphone 170C may be provided at the top of the stylus. The user holds the stylus and inputs an ambient sound signal through the first microphone 170C. For example, the user says "August 3, 2023, the weather is sunny" to the first microphone 170C, and the ambient sound signal includes "August 3, 2023, the weather is sunny". The first microphone 170C transmits the ambient sound signal to the first processor 110, and the first processor 110 determines whether the ambient sound signal includes the sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes the sound signal input by the user within the set distance range, the sound signal is voice recognized to generate a voice command, and at this time, the voice command is "record and save August 3, 2023, the weather is sunny". The first processor 110 transmits the voice command to the first functional module 101, and at this time, the first functional module is a recorder application (Application, APP for short). In response to the voice command, the recorder APP records and saves the "August 3, 2023, the weather is sunny" spoken by the user to generate a recording file. In one possible implementation, the first electronic device 100 sends the recording file to the second electronic device, and the second electronic device can perform natural language processing (NLP) processes such as speech transcription, keyword extraction, and schedule extraction on the recording file. In another possible implementation, the first electronic device 100 sends the recording file to the second electronic device, and the second electronic device can perform speech transcription on the recording file to generate text information, and then perform NLP processes such as keyword extraction and schedule extraction on the text information. Among them, NLP may include SyntaxNet, word2vec, etc. In one embodiment of the present application, certain specific actions of the user can be combined with the above-mentioned voice interaction method.For example, in one usage scenario, the first electronic device 100 is a smart watch, which may be provided with a first microphone 170C and a motion sensor. The user wears the watch, and after the user raises his wrist, the ambient sound signal can be input through the first microphone 170C, and the above-mentioned voice interaction method can be continued, which will not be repeated here. Among them, the flip angle of the smart watch can be detected by the motion sensor to determine whether the user has raised his wrist. In another usage scenario, the first electronic device 100 is a mobile phone, which may be provided with a first microphone 170C and a motion sensor. The user holds the mobile phone, and after the user picks up the mobile phone, the ambient sound signal can be input through the first microphone 170C, and the above-mentioned voice interaction method can be continued, which will not be repeated here. Among them, the flip angle of the mobile phone can be detected by the motion sensor to determine whether the user has picked up the mobile phone.

[0082] Figure 2b A structural diagram of another voice interaction system provided in an embodiment of the present application is shown in FIG. Figure 2bAs shown, the first electronic device 100 includes a first microphone 170C and a first wireless communication module 160, and the second electronic device 200 includes a second wireless communication module 260, a second processor 210, and a second functional module 201, wherein the second functional module 201 may be a speaker, an indicator light, a software application, etc. The first microphone 170C is communicatively connected to the first wireless communication module 160, the first wireless communication module 160 is communicatively connected to the second wireless communication module 260, the second wireless communication module 260 is communicatively connected to the second processor 210, and the second processor 210 is communicatively connected to the second functional module 201. In a possible implementation, the user holds the first electronic device 100, inputs an ambient sound signal through the first microphone 170C, the first microphone 170C transmits the ambient sound signal to the first wireless communication module 160, the first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260, the second wireless communication module 260 transmits the ambient sound signal to the second processor 210, the second processor 210 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, the sound signal is voice recognized to generate a voice command. The second processor 210 transmits the voice command to the second function module 201, and the second function module 201 triggers voice interaction in response to the voice command. In a usage scenario, the first electronic device 100 is a stylus pen, the second electronic device 200 is a mobile phone, the top of the stylus pen may be provided with a first microphone 170C, the user holds the stylus pen, and inputs the ambient sound signal through the first microphone 170C, for example, the user says "open music APP" to the first microphone 170C, then the ambient sound signal includes "open music APP". The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160, the first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260, the second wireless communication module 260 transmits the ambient sound signal to the second processor 210, the second processor 210 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, the sound signal is voice recognized to generate a voice command, at which time the voice command is "open the music APP". The second processor 210 transmits the voice command "open the music APP" to the second functional module 201, the second functional module 201 is a music APP, the music APP responds to the voice command of "open the music APP" to open, and the music APP installed on the second electronic device is automatically opened.

[0083] Figure 2c A structural diagram of another voice interaction system provided in an embodiment of the present application is shown in FIG. Figure 2cAs shown, the first electronic device 100 includes a first microphone 170C, a first processor 110 and a first wireless communication module 160, and the second electronic device 200 includes a second wireless communication module 260 and a second functional module 201, wherein the second functional module 201 may be a speaker, an indicator light, a software application, etc. The first microphone 170C is connected to the first processor 110 for communication, the first processor 110 is connected to the first wireless communication module 160 for communication, the first wireless communication module 160 is connected to the second wireless communication module 260 for communication, and the second wireless communication module 260 is connected to the second functional module 201 for communication. In a possible implementation, the user holds the first electronic device 100, inputs an ambient sound signal through the first microphone 170C, the first microphone 170C transmits the ambient sound signal to the first processor 110, and the first processor 110 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range, the sound signal is subjected to voice recognition to generate a voice command. The first processor 110 transmits the voice command to the first wireless communication module 160, the first wireless communication module 160 transmits the voice command to the second wireless communication module 260, the second wireless communication module 260 transmits the voice command to the second functional module 201, and the second functional module 201 triggers the voice interaction in response to the voice command. In a usage scenario, the first electronic device 100 is a stylus, the second electronic device 200 is a mobile phone, and the top of the stylus may be provided with a first microphone 170C. The user holds the stylus and inputs the ambient sound signal through the first microphone 170C. For example, the user says "open the music APP" to the first microphone 170C, and the ambient sound signal includes "open the music APP". The first microphone 170C transmits the ambient sound signal to the first processor 110, and the first processor 110 determines whether the ambient sound signal includes the sound signal input by the user within the set distance range; when it is determined that the ambient sound signal includes the sound signal input by the user within the set distance range, the sound signal is voice recognized to generate a voice command, and at this time, the voice command is "open the music APP". The first processor 110 transmits the voice command to the first wireless communication module 160, the first wireless communication module 160 transmits the voice command to the second wireless communication module 260, the second wireless communication module 260 transmits the voice command to the second functional module 201, the second functional module 201 is a music APP, the music APP responds to the voice command of "open the music APP" to open, and the music APP installed on the second electronic device is automatically opened.

[0084] Figure 2d A structural diagram of another voice interaction system provided in an embodiment of the present application is shown in FIG. Figure 2dAs shown, the first electronic device 100 includes a first microphone 170C, a first wireless communication module 160 and a first functional module 101, wherein the first functional module 101 may be a speaker, an indicator light, a software application, etc. The second electronic device 200 includes a second wireless communication module 260 and a second processor 210. The first microphone 170C is connected to the first wireless communication module 160 for communication, the first wireless communication module 160 is connected to the second wireless communication module 260 for communication, the second wireless communication module 260 is connected to the second processor 210 for communication, and the first wireless communication module 160 is connected to the first functional module 101 for communication. In a possible implementation, a user holds the first electronic device 100 and inputs an ambient sound signal through the first microphone 170C. The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160. The first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260. The second wireless communication module 260 transmits the ambient sound signal to the second processor 210. The second processor 210 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. When it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, the sound signal is subjected to voice recognition to generate a voice command. The second processor 210 transmits the voice command to the second wireless communication module 260. The second wireless communication module 260 transmits the voice command to the first wireless communication module 160. The first wireless communication module 160 transmits the voice command to the first functional module 101. The first functional module 101 triggers voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus, the second electronic device 200 is a mobile phone, and the top of the stylus may be provided with a first microphone 170C. The user holds the stylus and inputs an ambient sound signal through the first microphone 170C. For example, the user says "August 3, 2023, the weather is sunny" to the first microphone 170C, and the ambient sound signal includes "August 3, 2023, the weather is sunny". The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160, the first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260, the second wireless communication module 260 transmits the ambient sound signal to the second processor 210, and the second processor 210 determines whether the ambient sound signal includes the sound signal input by the user within a set distance range; when it is determined that the ambient sound signal includes the sound signal input by the user within the set distance range, the sound signal is voice recognized to generate a voice command. At this time, the voice command is "record and save August 3, 2023, the weather is sunny".The second processor 210 transmits the voice command to the second wireless communication module 260, the second wireless communication module 260 transmits the voice command to the first wireless communication module 160, the first wireless communication module 160 transmits the voice command to the first functional module 101, at this time the first functional module is a recorder APP, the recorder APP responds to the voice command, records and saves the user's words "August 3, 2023, sunny", and generates a recording file. In a possible implementation, the first electronic device 100 sends the recording file to the second electronic device 200, and the second electronic device 200 can perform NLP processes such as voice transcription, keyword extraction, and schedule extraction on the recording file.

[0085] In one embodiment of the present application, the above Figures 2a to 2d The first electronic device 100 and the second electronic device 200 may be connected via wireless communication, for example, the wireless communication method includes Bluetooth or WiFi.

[0086] For example, Figure 3 As shown, a schematic diagram of the hardware structure of an electronic device is provided. Figures 2a to 2d The first electronic device 100 and the second electronic device 200 can both adopt Figure 3 The hardware structure of the electronic equipment in the Figure 3 As shown, the electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395. It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown, or combine some components, or split some components, or arrange the components differently. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 310 for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or circulated. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 310 is reduced, and the efficiency of the system is improved.

[0088] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation of the electronic device 300. In other embodiments of the present application, the electronic device 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0089] The wireless communication function of the electronic device 300 can be implemented by antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0090] The mobile communication module 350 can provide a solution for wireless communication including 2G / 3G / 4G / 5G / 6G applied to the electronic device 300. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, filter, amplify and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the processor 310. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the same device as at least some of the modules of the processor 310. The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 370A, a receiver 370B, etc.), or displays an image or video through a display screen 394. In some embodiments, the modem processor can be an independent device. In other embodiments, the modem processor can be independent of the processor 310 and be set in the same device as the mobile communication module 350 or other functional modules.

[0091] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the electronic device 300. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering processing, and sends the processed signal to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0092] In some embodiments, the antenna 1 of the electronic device 300 is coupled to the mobile communication module 350, and the antenna 2 is coupled to the wireless communication module 360, so that the electronic device 300 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0093] The electronic device 300 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute instructions to generate or change display information. The electronic device 300 can implement a shooting function through an ISP, one or more cameras 393, a video codec, a GPU, one or more display screens 394, and an application processor.

[0094] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0095] The internal memory 321 can be used to store computer executable program codes, which include instructions. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 300 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 310 executes various functional applications and data processing of the electronic device 300 by running instructions stored in the internal memory 321, and / or instructions stored in a memory provided in the processor.

[0096] The electronic device 300 can implement audio functions such as music playing and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.

[0097] The sensor module 380 may include a pressure sensor 380A, a gyroscope sensor 380B, an air pressure sensor 380C, a magnetic sensor 380D, an acceleration sensor 380E, a distance sensor 380F, a proximity light sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, etc. Among them, the touch sensor 380K is also called a "touch control device". The touch sensor 380K can be set on the display screen 394, and the touch sensor 380K and the display screen 394 form a touch screen, also called a "touch screen". The touch sensor 380K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 394. In other embodiments, the touch sensor 380K can also be set on the surface of the electronic device 300, which is different from the position of the display screen 394.

[0098] The gyro sensor 380B can be used to determine the motion posture of the electronic device 300. In some embodiments, the angular velocity of the electronic device 300 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 380B. The gyro sensor 380B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 380B detects the angle of the electronic device 300 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 300 through reverse movement to achieve anti-shake. The gyro sensor 380B can also be used for navigation and somatosensory game scenes.

[0099] The acceleration sensor 380E can detect the magnitude of the acceleration of the electronic device 300 in all directions (generally three axes). When the electronic device 300 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0100] In some embodiments, the electronic device 300 includes but is not limited to a Or other operating systems. For example, the electronic device 300 may include a mobile phone, a tablet computer, a drawing board, a stylus or a wearable device.

[0101] Based on the above-mentioned voice interaction system, an embodiment of the present application provides a voice interaction method. Figures 4a to 4c A detailed description of the voice interaction method is given.

[0102] The first electronic device obtains an ambient sound signal. In one embodiment of the present application, the first electronic device can directly obtain the ambient sound signal through a microphone provided by itself, or can receive the ambient sound signal sent by the second electronic device. Specifically, the second electronic device obtains the ambient sound signal through a microphone provided by itself, and sends the ambient sound signal to the first electronic device. In one possible implementation, the microphone is always on to obtain the ambient sound signal in real time.

[0103] When the first electronic device determines that the ambient sound signal includes a sound signal input by the user within a set distance range, the first electronic device performs voice recognition on the sound signal and generates a voice command. Specifically, when the first electronic device determines that the acquired ambient sound signal satisfies the conditions of voice spectrum analysis and determines that the ambient sound signal satisfies the conditions of near-field spectrum analysis, it determines that the ambient sound signal includes a sound signal input by the user within a set distance range, performs voice recognition on the sound signal, and generates a voice command. In one embodiment of the present application, the portion of the ambient sound signal that satisfies the conditions of voice spectrum analysis and near-field spectrum analysis is a sound signal input by the user within a set distance range. Figure 4a A schematic diagram of a scenario for obtaining an ambient sound signal provided in an embodiment of the present application, such as Figure 4aAs shown, when the first electronic device is a stylus, a microphone is provided at the rear end of the stylus, and the user brings the rear end of the stylus close to his mouth to speak, and the first electronic device can obtain an ambient sound signal. When the first electronic device is a mobile phone, a microphone is provided at the lower end of the mobile phone, and the user brings the lower end of the mobile phone close to his mouth to speak, and the first electronic device can obtain an ambient sound signal. Among them, the number of microphones can be one or more. The ambient sound signal may include the sound of the user speaking, the sound of passers-by speaking, and various noises. The first electronic device needs to determine whether the ambient sound signal meets the conditions of speech spectrum analysis and whether the ambient sound signal meets the conditions of near-field spectrum analysis. In a possible implementation, when the first electronic device identifies the ambient sound signal based on the voiceprint recognition algorithm to obtain the speaker corresponding to the ambient sound signal, it is determined that the ambient sound signal meets the conditions of speech spectrum analysis, and it can be determined that the current ambient sound signal includes human speech. In one possible implementation, the first electronic device identifies one or more speakers corresponding to the ambient sound signal based on the voiceprint recognition algorithm; generates the pickup distance corresponding to each speaker according to the acquired pickup model; when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal meets the conditions of near-field spectrum analysis. Among them, the first electronic device can learn the near-field spectrum characteristics according to the neural network and generate the pickup model to judge whether the ambient sound signal meets the near-field characteristics. In another possible implementation, the first electronic device performs near-field spectrum analysis on the ambient sound signal based on the near-field evanescent wave effect to generate the pickup distance corresponding to one or more speakers; when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal meets the conditions of near-field spectrum analysis. In one embodiment of the present application, the set distance threshold can be set according to the actual situation, for example, the set distance threshold is 10cm. Specifically, the first electronic device generates the sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; generates the pickup distance corresponding to one or more speakers according to the sound speed information; or generates the pickup distance corresponding to one or more speakers according to the sound pressure information and the sound speed information. Among them, the near-field evanescent wave effect refers to the fact that a general omnidirectional microphone can collect sound pressure information in space. A directional microphone can be considered to be the first-order derivative of the sound pressure information to a certain extent, that is, the pressure difference information or the sound speed information. The sound speed information and the sound pressure information change differently when they are close to the sound source. Among them, the directivity of the microphone means that the microphone can pick up ambient sound from different directions. When a directional microphone picks up ambient sound from different directions, different directions can obtain different sound speed information. For example, the direction in which a stylus picks up ambient sound can be the direction along the stylus shaft pointing to the end of the stylus. Figure 4b A schematic diagram of the relationship between the sound velocity information, the sound pressure information and the sound pickup distance provided in an embodiment of the present application is shown in FIG. Figure 4bAs shown in the figure, Un is the sound speed information in m / s, P is the sound pressure information in Pa, r is the sound pickup distance in cm, that is, the distance between the detection position and the sound source, and level is the size of the sound speed information or the sound pressure information. At a long distance, the size of the sound pressure information and the sound speed information are almost the same. As the sound pickup distance decreases, the sound speed information is larger than the sound pressure information. This is mainly because the sound propagation model is closer to a spherical wave when near the sound source, while it can be considered as a plane wave at a long distance. Figure 4c A schematic diagram of the relationship between the sound speed information, the sound pickup distance and the sound frequency provided in an embodiment of the present application is shown in FIG. Figure 4c As shown, level is the size of the sound speed information, the unit is m / s, F is the sound frequency, the unit is Hz, and the size of the sound speed information of the same sound frequency when the pickup distance is 5cm is greater than the size of the sound speed information of the same sound frequency when the pickup distance is 50cm. Therefore, when the stylus uses only one microphone, the microphone can be a sound speed microphone to collect the sound speed information of the user's speech. In addition, when the first electronic device uses multiple microphones, the multiple microphones can simultaneously obtain a number of sound pressure information and a number of sound speed information. In one embodiment of the present application, the setting of the low-frequency signal and the high-frequency signal of the ambient sound signal is related to the pickup distance. For example, when the pickup distance is 5cm, the signal below 1100Hz is a low-frequency signal, and the signal of 1100Hz and above is a high-frequency signal. For another example, when the pickup distance is 10cm, the signal below 600Hz is a low-frequency signal, and the signal of 600Hz and above is a high-frequency signal. The above-mentioned near-field evanescent wave effect can make the sound source at a close distance present a greater receiving sensitivity when using a directional microphone than an omnidirectional microphone. The larger the wavelength, the more obvious the sound wave enhancement effect, and the enhancement effect of the low-frequency signal reflected in the ambient sound signal is more obvious. The distance at which the high-frequency signal triggers the near-field evanescent wave effect needs to be closer than the low-frequency signal, and the degree of enhancement is also lower than the low-frequency part. According to the above characteristics, the ambient sound signal at a close distance and the ambient sound signal at a long distance can be distinguished relatively well, and the above function can be achieved using only one directional microphone. In a possible implementation, the first electronic device performs voice recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers, and generates a voice instruction. As an optional solution, the first electronic device performs voice recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers based on a voice recognition algorithm, and generates a voice instruction. Among them, the voice instruction may include an application operation instruction or a recording instruction. In a possible implementation, before the voice recognition of the sound signal, the ambient sound signal can also be sound enhanced to generate a sound signal. For example, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0104] The first electronic device triggers voice interaction in response to the voice instruction; or sends a voice instruction to the second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0105] To implement the above technical solution, the following is a specific embodiment to illustrate the workflow of the voice interaction method. An embodiment of the present application provides a voice interaction method, Figure 5 A flow chart of a voice interaction method provided according to an embodiment of the present application is shown in FIG. Figure 5 As shown, the method includes:

[0106] Step 102: The first electronic device obtains an ambient sound signal.

[0107] In one possible implementation, one or more microphones are installed in the first electronic device. The microphone is always on to obtain ambient sound signals in real time. For example, when the first electronic device is a stylus, the stylus may be equipped with a sonic microphone. The user holds the stylus, and when the stylus is in use, the microphone is always on. When the user holds the stylus to write and wants to execute certain instructions, the user moves the stylus from the writing state to the mouth and speaks to the stylus to obtain ambient sound signals. For example, the user says "open memo" to the stylus. At this time, the ambient sound signal may include sound signals (the sound of the user speaking "open memo"), the sound of passers-by speaking, and various noises.

[0108] In a possible implementation, when the first electronic device is a smart watch or a mobile phone, when the motion sensor of the first electronic device recognizes the user's wrist raising action or the mobile phone picking up action, step 102 is simultaneously executed to perform the above-mentioned voice interaction method.

[0109] In one embodiment of the present application, the motion sensor may be an inertial measurement unit (IMU), a compass, a micro-electro-mechanical system (MEMS) accelerometer or a MEMS gyroscope. The motion sensor may include Figure 3 The gyro sensor 380B and the acceleration sensor 380E in the device.

[0110] Step 104: When the first electronic device determines that the ambient sound signal includes a sound signal input by the user within a set distance range, the first electronic device performs sound enhancement processing on the ambient sound signal to generate a sound signal.

[0111] Specifically, when the first electronic device determines that the acquired ambient sound signal satisfies the conditions of speech spectrum analysis and determines that the ambient sound signal satisfies the conditions of near-field spectrum analysis, it determines that the ambient sound signal includes a sound signal input by the user within a set distance range. In one embodiment of the present application, the portion of the ambient sound signal that satisfies the conditions of speech spectrum analysis and near-field spectrum analysis is a sound signal input by the user within a set distance range.

[0112] In a possible implementation, when the first electronic device identifies the ambient sound signal based on the voiceprint recognition algorithm and obtains the speaker corresponding to the ambient sound signal, it is determined that the ambient sound signal meets the conditions of speech spectrum analysis, indicating that the ambient sound signal includes the human speech sound. As an optional solution, when the first electronic device recognizes that the spectrum characteristics of the ambient sound signal include the Mel spectrum, it can be indicated that the ambient sound signal includes the human speech sound.

[0113] In one possible implementation, based on the near-field evanescent wave effect, near-field spectrum analysis is performed on the spectral characteristics of the ambient sound signal to generate the pickup distance corresponding to one or more speakers; when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is determined that the ambient sound signal meets the conditions for near-field spectrum analysis.

[0114] In one embodiment of the present application, the distance threshold can be set according to actual conditions, for example, the distance threshold is set to 10 cm. In a possible implementation, when the first electronic device is a smart watch or a mobile phone, the motion sensor of the first electronic device improves the sensitivity of near-field sound pickup after recognizing the user's wrist-raising action or picking up the mobile phone. For example, the set distance threshold can be increased to increase the probability of judging that the ambient sound signal meets the conditions for near-field spectrum analysis.

[0115] As an optional solution, the sound velocity microphone can collect sound velocity information corresponding to the ambient sound signal, and can generate the pickup distance corresponding to one or more speakers based on the sound velocity information.

[0116] As another optional solution, the sound pressure microphone can collect sound pressure information corresponding to the ambient sound signal, and the sound velocity microphone can collect sound velocity information corresponding to the ambient sound signal. The sound pickup distance corresponding to one or more speakers can be generated based on the sound pressure information and the sound velocity information.

[0117] Specifically, the first electronic device may store the correspondence between the sound speed information, the sound pressure information and the sound pickup distance. The correspondence between the sound speed information, the sound pressure information and the sound pickup distance is as shown in the following formula.

[0118] P(r, t) = Re{Ae i(wt-kr) r -1}Wherein, P(r, t) is the sound pressure information, Re represents the real part, k is the wave number, A is a specific constant, w is the angular frequency, r is the pickup distance, and t is the time.

[0119] Among them, u(r, t) is the sound speed information, A is a specific constant, w is the angular frequency, r is the pickup distance, ρ0 is the air density, and t is the time.

[0120] In one embodiment of the present application, for the sound pressure information, only the real part is considered, and the attenuation with distance is proportional to 1 / r; for the sound speed information, the first half of the plus sign is the imaginary part, and the second half is the real part. The second half is the same as the sound pressure information (only the constant is different), and the first half is the additional term of the sound speed information, which is proportional to 1 / r. 2 It is directly proportional to the r value, and can only provide a significant contribution when the r value is small, and can be ignored when r is large.

[0121] In one embodiment of the present application, the sound enhancement process includes noise suppression process and / or reverberation suppression process. For example, the noise suppression process may include a linear filtering method, a spectral subtraction method, a statistical model algorithm, or a subspace algorithm. The reverberation suppression process may include a weighted prediction error algorithm (weighted prediction error, referred to as WPE).

[0122] In one embodiment of the present application, the above algorithm can also be used to realize sound enhancement processing of the ambient sound signal of the user speaking softly. The main difference between speaking softly and speaking normally is that speaking softly is only the vibration of the air near the mouth, while speaking normally is the vibration of the vocal cords. Therefore, there is a significant difference in the energy of speaking softly and speaking normally, and there is a significant difference in the fundamental frequency part, but there is no significant difference in the spectrum of speaking softly. Unlike the plosive sound detection in the related art, the present application uses the spectrum information and / or sound speed information of the ambient sound signal to detect the ambient sound signal, and does not need to rely on the energy of the ambient sound signal for detection. Therefore, the present application can realize the soft voice wake-up function of the electronic device in the near-field voice interaction scenario.

[0123] Step 106: The first electronic device performs voice recognition on the sound signal to generate a voice command.

[0124] In one possible implementation, the first electronic device performs speech recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers, and generates a voice instruction. As an optional scheme, the first electronic device performs speech recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers based on a speech recognition algorithm, and generates a voice instruction. Among them, the speech recognition algorithm may include a hidden Markov model (HMM for short), WFST-based speech decoding, etc. Voice instructions may include application operation instructions or recording instructions. For example, application operation instructions include opening a music APP, recording tomorrow's schedule, setting an alarm, etc. When the speech recognition does not recognize the application operation instruction, the recording instruction is triggered, that is, the user's speech sound is recorded.

[0125] For example, the first electronic device performs voice recognition on the sound signal "open memo" and can generate a voice command "open memo APP".

[0126] Step 108: The first electronic device sends a voice command to the second electronic device.

[0127] For example, the second electronic device is a mobile phone, and the first electronic device (stylus pen) sends a voice command "open the memo APP" to the second electronic device (mobile phone).

[0128] Step 110: The second electronic device triggers voice interaction in response to the voice command.

[0129] For example, the second electronic device (mobile phone) triggers voice interaction in response to the voice command "open the memo APP", that is, the memo APP installed in the second electronic device (mobile phone) is automatically opened.

[0130] In the technical solution provided by an embodiment of the present application, an ambient sound signal is obtained; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal to generate a voice command; voice interaction is triggered in response to the voice command; or, a voice command is sent to a second electronic device, and the voice command is used to trigger the second electronic device to perform voice interaction. The technical solution provided by an embodiment of the present application can realize the soft voice-free wake-up function for electronic devices in near-field voice interaction scenarios by determining whether the ambient sound signal includes a sound signal input by the user within a set distance range, and performing voice recognition on the sound signal input by the user within the set distance range to trigger voice interaction. .

[0131] In the technical solution provided in one embodiment of the present application, the voice interaction capability of the first electronic device with a single microphone can be used to distinguish between far-field and near-field ambient sound signals, and voice enhancement can be performed on near-field sound signals in a targeted manner.

[0132] In the technical solution provided in one embodiment of the present application, the spectral features of the voice spectrum and the near-field spectrum can be used to recognize and trigger voice commands for voice interaction, thereby realizing a wake-up-free voice interaction function.

[0133] The technical solution provided in one embodiment of the present application solves the problem of high computing power required for wake-up actions and wake-up recognition in the related technology, provides a solution for automatically identifying effective ambient sound signals, and then performs processing such as voice enhancement and wake-up triggering, which can further realize time-frequency point-level judgment and post-processing such as noise reduction and writing, thereby improving the user's efficiency when using multiple electronic devices.

[0134] An embodiment of the present application provides a voice interaction device, Figure 6 FIG. 1 is a schematic diagram of a structure of a voice interaction device provided according to an embodiment of the present application. Figure 6 As shown, the device includes: a microphone module 61, a judgment module 62, a speech recognition module 63, a speech interaction module 64 and a wireless communication module 65.

[0135] The microphone module 61 is used to obtain ambient sound signals.

[0136] The determination module 62 is used to determine whether the ambient sound signal includes a sound signal input by the user within a set distance range.

[0137] The voice recognition module 63 is used to perform voice recognition on the sound signal and generate a voice command when the judgment module 62 judges that the ambient sound signal includes a sound signal input by the user within a set distance range.

[0138] The voice interaction module 64 is used to trigger voice interaction in response to a voice instruction; or the wireless communication module 65 is used to send a voice instruction to the second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0139] In one embodiment of the present application, the judgment module 62 is specifically used to: identify one or more speakers corresponding to the ambient sound signal; generate the pickup distance corresponding to each speaker based on the acquired pickup model; when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal includes the sound signal input by the user within the set distance range.

[0140] In one embodiment of the present application, the judgment module 62 is specifically used to: generate the pickup distance corresponding to one or more speakers according to the ambient sound signal; when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal includes the sound signal input by the user within the set distance range.

[0141] In one embodiment of the present application, the judgment module 62 is specifically used to: generate sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; generate the pickup distance corresponding to one or more speakers according to the sound speed information; or generate the pickup distance corresponding to one or more speakers according to the sound pressure information and the sound speed information.

[0142] In one embodiment of the present application, when the number of microphones in the microphone module 61 is one, the microphone is a sound velocity microphone; the judgment module 62 is specifically used to generate sound velocity information of the ambient sound signal based on the ambient sound signal; and generate the pickup distance corresponding to one or more speakers based on the sound velocity information.

[0143] In one embodiment of the present application, when there are multiple microphones in the microphone module 61, the microphones include a sound velocity microphone and a sound pressure microphone; the judgment module 62 is specifically used to generate sound pressure information and sound velocity information of the ambient sound signal based on the ambient sound signal; and generate the pickup distance corresponding to one or more speakers based on the sound pressure information and the sound velocity information.

[0144] In one embodiment of the present application, the microphone is always on to obtain ambient sound signals in real time.

[0145] In one embodiment of the present application, the speech recognition module 63 is specifically used to: perform speech recognition on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more speakers, and generate a speech command.

[0146] In one embodiment of the present application, the voice command includes an application operation command or a recording command.

[0147] In one embodiment of the present application, the device further includes a sound enhancement module 66 .

[0148] The sound enhancement module 66 is used to perform sound enhancement processing on the ambient sound signal to generate a sound signal.

[0149] In one embodiment of the present application, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0150] In one embodiment of the present application, the device further includes a receiving module 67 .

[0151] The receiving module 67 is used to receive the ambient sound signal sent by the second electronic device.

[0152] In the technical solution provided by an embodiment of the present application, an ambient sound signal is obtained; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal to generate a voice command; voice interaction is triggered in response to the voice command; or, a voice command is sent to a second electronic device, and the voice command is used to trigger the second electronic device to perform voice interaction. The technical solution provided by an embodiment of the present application can realize the soft voice-free wake-up function for electronic devices in near-field voice interaction scenarios by determining whether the ambient sound signal includes a sound signal input by the user within a set distance range, and performing voice recognition on the sound signal input by the user within the set distance range to trigger voice interaction.

[0153] In the description of the embodiments of the present application, for the convenience of description, the device is described by dividing it into various modules according to its functions. The division of each module is merely a division of logical functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or one or more software and / or hardware.

[0154] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in a processor element or instructions in the form of software.

[0155] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0156] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.

[0157] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0159] In the several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0160] Specifically, in one embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0161] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is run on a computer, the computer executes the method provided by the embodiment of the present application.

[0162] The embodiments described in this application are described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0165] It should also be noted that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0166] In the embodiments of the present application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0167] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0168] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0169] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0170] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0171] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice interaction method, characterized in that: The method is applied to a first electronic device, and the method includes: Acquire environmental sound signals; When it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, performing voice recognition on the sound signal to generate a voice command; In response to the voice instruction, voice interaction is triggered; or, the voice instruction is sent to a second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

2. The method according to claim 1, characterized in that The determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes: Identifying one or more speakers corresponding to the ambient sound signal; Generate the pickup distance corresponding to each speaker according to the acquired pickup model; When the sound pickup distance corresponding to one or more of the speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

3. The method according to claim 1, characterized in that The determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes: generating a sound pickup distance corresponding to one or more speakers according to the ambient sound signal; When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

4. The method according to claim 3, characterized in that The generating the sound pickup distances corresponding to one or more speakers according to the ambient sound signal specifically includes: generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or, The sound pickup distances corresponding to one or more speakers are generated according to the sound pressure information and the sound speed information.

5. The method according to any one of claims 2 to 4, characterized in that: The performing speech recognition on the sound signal to generate a speech instruction specifically includes: Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more of the speakers to generate a voice command.

6. The method according to claim 1, characterized in that The voice instructions include application operation instructions or recording instructions.

7. The method according to claim 1, characterized in that Before performing speech recognition on the sound signal, the method further includes: Perform sound enhancement processing on the ambient sound signal to generate a sound signal.

8. The method according to claim 7, characterized in that The sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

9. The method according to claim 1, characterized in that: The obtaining of the ambient sound signal specifically includes: Receive the ambient sound signal sent by the second electronic device.

10. A voice interaction device, characterized in that: Applicable to a first electronic device, comprising: Microphone module, used to obtain ambient sound signals; A determination module, used to determine whether the ambient sound signal includes a sound signal input by a user within a set distance range; a voice recognition module, configured to perform voice recognition on the sound signal and generate a voice command when the judgment module judges that the ambient sound signal includes a sound signal input by a user within a set distance range; A voice interaction module is used to trigger voice interaction in response to the voice instruction; or a wireless communication module is used to send the voice instruction to a second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

11. The device according to claim 10, characterized in that The judgment module is specifically used for: Identifying one or more speakers corresponding to the ambient sound signal; Generate the pickup distance corresponding to each speaker according to the acquired pickup model; When the sound pickup distance corresponding to one or more of the speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

12. The device according to claim 10, characterized in that The judgment module is specifically used for: generating a sound pickup distance corresponding to one or more speakers according to the ambient sound signal; When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

13. The device according to claim 12, characterized in that The judgment module is specifically used for: generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or, The sound pickup distances corresponding to one or more speakers are generated according to the sound pressure information and the sound speed information.

14. The device according to claim 13, characterized in that Also includes: When the number of microphones in the microphone module is one, the microphone is a sonic velocity microphone; The judgment module is specifically configured to generate sound speed information of the ambient sound signal according to the ambient sound signal; and generate sound pickup distances corresponding to one or more speakers according to the sound speed information.

15. The device according to claim 13, characterized in that Also includes: When there are multiple microphones in the microphone module, the microphones include a sound velocity microphone and a sound pressure microphone; The judgment module is specifically used to generate sound pressure information and sound speed information of the ambient sound signal according to the ambient sound signal; and generate sound pickup distances corresponding to one or more speakers according to the sound pressure information and sound speed information.

16. The device according to claim 15, characterized in that The microphone is always turned on to obtain the ambient sound signal in real time.

17. The device according to any one of claims 11 to 13, characterized in that The speech recognition module is specifically used for: Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more of the speakers to generate a voice command.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 9.

19. A first electronic device, characterized in that: The first electronic device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the method steps as described in any one of claims 1-9.

Images