Speech interaction method and apparatus, and first electronic device

Abstract

A speech interaction method and apparatus, and a first electronic device and a storage medium. The method comprises: a first electronic device acquiring an ambient sound signal (102); when it is determined that the ambient sound signal comprises a sound signal input by a user within a set distance range, the first electronic device performing sound enhancement processing on the ambient sound signal, so as to generate a sound signal (104); the first electronic device performing speech recognition on the sound signal, so as to generate a speech instruction (106); the first electronic device sending the speech instruction to a second electronic device (108); and the second electronic device triggering speech interaction in response to the speech instruction (110). In the method, whether the ambient sound signal comprises the sound signal input by the user within the set distance range is determined, and speech recognition is performed on the sound signal input by the user within the set distance range, so as to trigger speech interaction. The method can realize a whisper no-activation function for an electronic device in a near-field speech interaction scenario.

Description

Voice interaction method, device and first electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 4, 2023, with application number 202311136337.0 and application name “A voice interaction method, device and first electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of electronic technology, and in particular to a voice interaction method, device, and first electronic device. Background Art

[0003] Voice is a common way to carry information in daily life. With the development of voice technology, more and more voice recognition algorithms have emerged for voice recognition. According to the distance of the voice source, it is divided into far-field voice and near-field voice. When performing voice recognition, different voice interaction methods can be used for different voice source distances to obtain clear and recognizable voice information.

[0004] In near-field voice interaction scenarios, users generally trigger the voice interaction function of electronic devices by manually clicking the wake-up button of the electronic device or saying a specific wake-up word. The general wake-up-free function uses directional voice pickup and can only respond to sounds within a certain angle range of the electronic device. It cannot realize the silent wake-up function of the electronic device in the near-field voice interaction scenario.

[0005] Summary of the Invention

[0006] In view of this, an embodiment of the present application provides a voice interaction method, device and a first electronic device for realizing a soft voice wake-up function for an electronic device in a near-field voice interaction scenario.

[0007] The embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a voice interaction method, which is applied to a first electronic device and includes:

[0009] Obtaining environmental sound signals;

[0010] When it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, performing voice recognition on the sound signal to generate a voice command;

[0011] Triggering voice interaction in response to a voice instruction; or sending a voice instruction to a second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0012] According to the voice interaction method provided in the present application, by determining whether the ambient sound signal includes the sound signal input by the user within a set distance range, voice recognition is performed on the sound signal input by the user within the set distance range to trigger voice interaction, thereby realizing the silent wake-up function of the electronic device in the near-field voice interaction scenario.

[0013] In one implementation of the first aspect, determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes:

[0014] Identify one or more speakers corresponding to the ambient sound signal;

[0015] Generate the pickup distance corresponding to each speaker based on the acquired pickup model;

[0016] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0017] In one implementation of the first aspect, determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes:

[0018] Generate the pickup distance corresponding to one or more speakers according to the ambient sound signal;

[0019] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0020] Furthermore, when the pickup distance corresponding to one or more speakers is less than the set distance threshold, it is judged that the ambient sound signal meets the conditions of near-field spectrum analysis. It can be judged based on the distance between the speaker's sound body and the first electronic device that the current ambient sound signal includes the sound signal input by the user within the set distance range. It can determine the speaker in the near-field voice interaction condition from multiple speakers, and can improve the accuracy of subsequent voice recognition.

[0021] In one implementation of the first aspect, generating pickup distances corresponding to one or more speakers based on an ambient sound signal specifically includes:

[0022] generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal;

[0023] Generate the pickup distance corresponding to one or more speakers based on the sound speed information; or,

[0024] The sound pickup distance corresponding to one or more speakers is generated based on the sound pressure information and sound speed information.

[0025] Furthermore, the sound pickup distance of the speaker can be accurately obtained according to the sound pressure information and / or sound speed information.

[0026] In one implementation of the first aspect, performing voice recognition on a sound signal to generate a voice command specifically includes:

[0027] Speech recognition is performed on a sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers to generate a voice command.

[0028] Furthermore, by performing speech recognition on the sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more speakers, the sound signal of the user closest to the first electronic device can be accurately obtained, thereby improving the accuracy of sound pickup.

[0029] In an implementation manner of the first aspect, the voice instruction includes an application operation instruction or a recording instruction.

[0030] In an implementation of the first aspect, before performing speech recognition on the sound signal, the method further includes:

[0031] Perform sound enhancement processing on the ambient sound signal to generate a sound signal.

[0032] Furthermore, performing sound enhancement processing on the ambient sound signal can remove the ambient noise and improve the quality of the sound signal.

[0033] In an implementation manner of the first aspect, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0034] In an implementation of the first aspect, obtaining an ambient sound signal specifically includes:

[0035] Receive an ambient sound signal sent by a second electronic device.

[0036] In a second aspect, an embodiment of the present application provides a voice interaction device, which is applied to a first electronic device and includes:

[0037] Microphone module, used to obtain ambient sound signals;

[0038] a determination module, configured to determine whether the ambient sound signal includes a sound signal input by the user within a set distance range;

[0039] a voice recognition module configured to perform voice recognition on the sound signal and generate a voice command when the judgment module determines that the ambient sound signal includes a sound signal input by the user within a set distance range;

[0040] A voice interaction module is used to trigger voice interaction in response to a voice instruction; or a wireless communication module is used to send a voice instruction to a second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0041] In an implementation of the second aspect, the judgment module is specifically configured to:

[0042] Identify one or more speakers corresponding to the ambient sound signal;

[0043] Generate the pickup distance corresponding to each speaker based on the acquired pickup model;

[0044] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0045] In an implementation of the second aspect, the judgment module is specifically configured to:

[0046] Generate the pickup distance corresponding to one or more speakers according to the ambient sound signal;

[0047] When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0048] In an implementation of the second aspect, the judgment module is specifically configured to:

[0049] generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal;

[0050] Generate the pickup distance corresponding to one or more speakers based on the sound speed information; or,

[0051] The sound pickup distance corresponding to one or more speakers is generated based on the sound pressure information and sound speed information.

[0052] In an implementation of the second aspect, the method further includes:

[0053] When the number of microphones in the microphone module is one, the microphone is a sonic microphone;

[0054] The judgment module is specifically used to generate sound speed information of the ambient sound signal according to the ambient sound signal; and generate the sound pickup distance corresponding to one or more speakers according to the sound speed information.

[0055] In an implementation of the second aspect, the method further includes:

[0056] When the microphone module includes multiple microphones, the microphones include a sound velocity microphone and a sound pressure microphone;

[0057] The judgment module is specifically used to generate sound pressure information and sound speed information of the ambient sound signal according to the ambient sound signal; and generate the sound pickup distance corresponding to one or more speakers according to the sound pressure information and sound speed information.

[0058] In an implementation of the second aspect, the microphone is always on to acquire the ambient sound signal in real time.

[0059] In an implementation of the second aspect, the speech recognition module is specifically configured to:

[0060] Speech recognition is performed on a sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers to generate a voice command.

[0061] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0062] In a fourth aspect, an embodiment of the present application provides a first electronic device, wherein the first electronic device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the method steps described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0064] FIG1 is a schematic diagram of a plosive sound produced by a user speaking at different distances provided in the related art;

[0065] FIG2a is a schematic structural diagram of a voice interaction system provided in one embodiment of the present application;

[0066] FIG2 b is a schematic structural diagram of another voice interaction system provided in an embodiment of the present application;

[0067] FIG2c is a schematic diagram of the structure of another voice interaction system provided in one embodiment of the present application;

[0068] FIG2 d is a schematic diagram of the structure of another voice interaction system provided in one embodiment of the present application;

[0069] FIG3 is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0070] FIG4a is a schematic diagram of a scenario for obtaining an ambient sound signal according to an embodiment of the present application;

[0071] FIG4b is a schematic diagram showing the relationship between sound velocity information, sound pressure information, and sound pickup distance according to an embodiment of the present application;

[0072] FIG4c is a schematic diagram showing the relationship between sound velocity information, sound pickup distance, and sound frequency according to an embodiment of the present application;

[0073] FIG5 is a flow chart of a voice interaction method according to an embodiment of the present application;

[0074] FIG6 is a schematic structural diagram of a voice interaction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0076] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0077] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0078] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0079] In a related technology, when receiving far-field speech, the far-field speech is input into a pre-trained neural network-based speech training model. The audio features of the far-field speech and near-field speech contained in the speech training model are used to denoise the audio features of the received far-field speech, obtain the processed far-field speech, and recognize it.

[0080] In another related technology, the far-field speech to be recognized received by the microphone array is subjected to adaptive beamforming processing to obtain a signal to be recognized. The signal to be recognized is then recognized using an acoustic model for far-field recognition. The acoustic model for far-field recognition can truly match the input speech signal for far-field recognition, thereby improving the recognition performance of far-field speech recognition. Both of the above related technologies are for speech recognition in far-field voice interaction scenarios, and cannot solve the technical problem of the soft voice wake-up function of electronic devices in near-field voice interaction scenarios.

[0081] Another related technology uses the detection of plosive sounds (pops) and microphone blasts from people speaking at different distances to determine which distance the user is speaking from meets the near-field voice interaction scenario. If the user meets the near-field voice interaction scenario, the electronic device can be enabled directly without waking up. Figure 1 shows a schematic diagram of plosive sounds from a user speaking at different distances, as provided in the related art. As shown in Figure 1, the user reads "How's the weather today?" at a normal volume at different distances. The amplitude of the sound varies between -1 and 1 when the pickup distance is 2cm, 5cm, 10cm, 20cm, 30cm, and 50cm. The amplitude of the sound is larger at a pickup distance of 2cm, and smaller at a pickup distance of 50cm. Because plosive sounds are produced when the vocal organs form an obstruction in the mouth and then the airflow breaks through the obstruction, the amplitude of the sound is larger when the user reads "today" and "weather" and smaller when reading "how about it." In addition, compared with normal speech at a pickup distance of 2 cm, soft speech at a pickup distance of 2 cm will be regarded as the sound of popular noise and / or airflow. Therefore, the user's soft speech at a pickup distance of 2 cm cannot be recognized. In addition, the related art can only start enabling for a certain starting frame of the user's speech, and use a certain judgment condition (such as time, maximum energy attenuation, etc.) as the end judgment, which cannot achieve accurate frame processing. For example, when the pickup distance is 2 cm, the probability of enabling after the "day" of the "weather" sound ends is 45.8%. According to the above method, the user's voice command cannot be accurately recognized, and voice interaction cannot be accurately performed.

[0082] In the embodiments of the present application, near-field voice interaction refers to a scenario where the pickup distance is less than 10 cm, and far-field voice interaction refers to a scenario where the pickup distance is more than 10 cm. The following embodiments of the present application are applied to the scenario of near-field voice interaction.

[0083] To address technical issues in related technologies, an embodiment of the present application provides a voice interaction system. FIG2a is a schematic structural diagram of a voice interaction system provided by an embodiment of the present application. As shown in FIG2a, a first electronic device 100 includes a first microphone 170C, a first processor 110, and a first functional module 101. The first functional module 101 may be a speaker, an indicator light, a software application, or the like. The first microphone 170C is in communication with the first processor 110, which is in communication with the first functional module 101. In one possible implementation, a user holds the first electronic device 100 and inputs an ambient sound signal through the first microphone 170C. The first microphone 170C transmits the ambient sound signal to the first processor 110. The first processor 110 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If the ambient sound signal is determined to include a sound signal input by the user within the set distance range, the first processor 110 performs voice recognition on the sound signal and generates a voice command. The first processor 110 transmits the voice command to the first functional module 101, which triggers voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus pen, and the tip of the stylus pen may be provided with a first microphone 170C. A user holding the stylus pen inputs an ambient sound signal through first microphone 170C. For example, if the user says "August 3, 2023, sunny" into first microphone 170C, the ambient sound signal will include "August 3, 2023, sunny." First microphone 170C transmits the ambient sound signal to first processor 110, which determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If the ambient sound signal does include a sound signal input by the user within the set distance range, it performs voice recognition on the sound signal and generates a voice command. In this case, the voice command is "Record and save 'August 3, 2023, sunny.'" First processor 110 transmits the voice command to first functional module 101, which is a recorder application (APP). In response to the voice command, the recorder APP records the user's voice "August 3, 2023, sunny" and generates a recording file. In one possible implementation, the first electronic device 100 sends the recording file to the second electronic device, and the second electronic device can perform natural language processing (NLP) processes such as speech transcription, keyword extraction, and schedule extraction on the recording file. In another possible implementation, the first electronic device 100 sends the recording file to the second electronic device, and the second electronic device can perform speech transcription on the recording file to generate text information, and then perform NLP processes such as keyword extraction and schedule extraction on the text information. Among them, NLP may include SyntaxNet, word2vec, etc.In one embodiment of the present application, certain specific actions of the user can be combined with the above-mentioned voice interaction method. For example, in one usage scenario, the first electronic device 100 is a smart watch, and the smart watch can be provided with a first microphone 170C and a motion sensor. The user wears the watch. After the user raises his wrist, the ambient sound signal can be input through the first microphone 170C, and the above-mentioned voice interaction method can be continued. It will not be repeated here. Among them, the flip angle of the smart watch can be detected by the motion sensor to determine whether the user has raised his wrist. In another usage scenario, the first electronic device 100 is a mobile phone, and the mobile phone can be provided with a first microphone 170C and a motion sensor. The user holds the mobile phone. After the user picks up the mobile phone, the ambient sound signal can be input through the first microphone 170C, and the above-mentioned voice interaction method can be continued. It will not be repeated here. Among them, the flip angle of the mobile phone can be detected by the motion sensor to determine whether the user has picked up the mobile phone.

[0084] FIG2 b is a schematic diagram of the structure of another voice interaction system provided in an embodiment of the present application. As shown in FIG2 b , the first electronic device 100 includes a first microphone 170C and a first wireless communication module 160, and the second electronic device 200 includes a second wireless communication module 260, a second processor 210, and a second functional module 201. The second functional module 201 may be a speaker, an indicator light, a software application, etc. The first microphone 170C is communicatively coupled to the first wireless communication module 160, the first wireless communication module 160 is communicatively coupled to the second wireless communication module 260, the second wireless communication module 260 is communicatively coupled to the second processor 210, and the second processor 210 is communicatively coupled to the second functional module 201. In one possible implementation, a user holds the first electronic device 100 and inputs an ambient sound signal through the first microphone 170C. The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160, which then transmits the ambient sound signal to the second wireless communication module 260. The second wireless communication module 260 then transmits the ambient sound signal to the second processor 210. The second processor 210 then determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If the ambient sound signal includes a sound signal input by the user within the set distance range, the second processor 210 performs voice recognition on the sound signal and generates a voice command. The second processor 210 transmits the voice command to the second functional module 201, which triggers voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus and the second electronic device 200 is a mobile phone. The stylus may be provided with the first microphone 170C at the top. The user holds the stylus and inputs the ambient sound signal through the first microphone 170C. For example, if the user says "Open the music app" into the first microphone 170C, the ambient sound signal includes "Open the music app." The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160. The first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260. The second wireless communication module 260 transmits the ambient sound signal to the second processor 210. The second processor 210 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. When the second processor 210 determines that the ambient sound signal includes a sound signal input by the user within the set distance range, it performs voice recognition on the sound signal and generates a voice command. In this case, the voice command is "Open the music app." The second processor 210 transmits the voice command "Open the music app" to the second functional module 201. The second functional module 201 is a music app. The music app opens in response to the voice command "Open the music app." At this time, the music app installed on the second electronic device automatically opens.

[0085] Figure 2c is a schematic diagram of the structure of another voice interaction system provided by an embodiment of the present application. As shown in Figure 2c, the first electronic device 100 includes a first microphone 170C, a first processor 110, and a first wireless communication module 160. The second electronic device 200 includes a second wireless communication module 260 and a second functional module 201, wherein the second functional module 201 can be a speaker, an indicator light, a software application, etc. The first microphone 170C is in communication with the first processor 110, the first processor 110 is in communication with the first wireless communication module 160, the first wireless communication module 160 is in communication with the second wireless communication module 260, and the second wireless communication module 260 is in communication with the second functional module 201. In one possible implementation, a user holds the first electronic device 100 and inputs an ambient sound signal through the first microphone 170C. The first microphone 170C transmits the ambient sound signal to the first processor 110. The first processor 110 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range, the first processor 110 performs voice recognition on the sound signal and generates a voice command. The first processor 110 transmits the voice command to the first wireless communication module 160, which then transmits the voice command to the second wireless communication module 260. The second wireless communication module 260 then transmits the voice command to the second functional module 201. The second functional module 201 triggers a voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus and the second electronic device 200 is a mobile phone. The stylus may be provided with a first microphone 170C at its tip. The user holds the stylus and inputs an ambient sound signal through the first microphone 170C. For example, if the user says "Open Music App" into the first microphone 170C, the ambient sound signal includes "Open Music App." The first microphone 170C transmits the ambient sound signal to the first processor 110, which determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If the ambient sound signal does include a sound signal input by the user within the set distance range, the first processor 110 performs voice recognition on the sound signal and generates a voice command. In this case, the voice command is "Open Music App." The first processor 110 transmits the voice command to the first wireless communication module 160, the first wireless communication module 160 transmits the voice command to the second wireless communication module 260, the second wireless communication module 260 transmits the voice command to the second functional module 201, the second functional module 201 is the music APP, the music APP responds to the voice command "Open the music APP" to open, and the music APP installed on the second electronic device is automatically opened.

[0086] FIG2 d is a schematic diagram of the structure of another voice interaction system provided in an embodiment of the present application. As shown in FIG2 d , the first electronic device 100 includes a first microphone 170C, a first wireless communication module 160, and a first functional module 101, wherein the first functional module 101 may be a speaker, an indicator light, a software application, etc. The second electronic device 200 includes a second wireless communication module 260 and a second processor 210. The first microphone 170C is communicatively coupled to the first wireless communication module 160, the first wireless communication module 160 is communicatively coupled to the second wireless communication module 260, the second wireless communication module 260 is communicatively coupled to the second processor 210, and the first wireless communication module 160 is connected to the first functional module 101. In one possible implementation, a user holds the first electronic device 100 and inputs an ambient sound signal through the first microphone 170C. The first microphone 170C transmits the ambient sound signal to the first wireless communication module 160. The first wireless communication module 160 transmits the ambient sound signal to the second wireless communication module 260. The second wireless communication module 260 transmits the ambient sound signal to the second processor 210. The second processor 210 determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If the second processor 210 determines that the ambient sound signal includes a sound signal input by the user within the set distance range, it performs voice recognition on the sound signal and generates a voice command. The second processor 210 transmits the voice command to the second wireless communication module 260. The second wireless communication module 260 transmits the voice command to the first wireless communication module 160. The first wireless communication module 160 transmits the voice command to the first functional module 101. The first functional module 101 triggers voice interaction in response to the voice command. In one usage scenario, the first electronic device 100 is a stylus, and the second electronic device 200 is a mobile phone. The stylus may be equipped with a first microphone 170C at its tip. A user holding the stylus inputs an ambient sound signal through first microphone 170C. For example, if the user says "August 3, 2023, sunny" into first microphone 170C, the ambient sound signal will include "August 3, 2023, sunny." First microphone 170C transmits the ambient sound signal to first wireless communication module 160, which in turn transmits the ambient sound signal to second wireless communication module 260. Second wireless communication module 260 transmits the ambient sound signal to second processor 210, which determines whether the ambient sound signal includes a sound signal input by the user within a set distance range. If so, second processor 210 performs voice recognition on the sound signal and generates a voice command. In this case, the voice command is "Record and save 'August 3, 2023, sunny'."The second processor 210 transmits the voice command to the second wireless communication module 260, which transmits the voice command to the first wireless communication module 160. The first wireless communication module 160 transmits the voice command to the first functional module 101. In this case, the first functional module is a recorder app. The recorder app responds to the voice command and records and saves the user's voice "The weather will be sunny on August 3, 2023" to generate a recording file. In one possible implementation, the first electronic device 100 sends the recording file to the second electronic device 200, which can perform NLP processes such as voice transcription, keyword extraction, and schedule extraction on the recording file.

[0087] In one embodiment of the present application, the first electronic device 100 and the second electronic device 200 in Figures 2a to 2d can be connected via wireless communication, for example, the wireless communication method includes Bluetooth or WiFi.

[0088] For example, as shown in FIG3 , a schematic diagram of the hardware structure of an electronic device is provided. The first electronic device 100 and the second electronic device 200 in FIG2a to 2d can both adopt the hardware structure of the electronic device in FIG3 . As shown in FIG3 , the electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display 394, and a subscriber identification module (SIM) card interface 395, etc. It will be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0089] Processor 310 may include one or more processing units. For example, processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 310 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 310 is a high-speed cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 310. If processor 310 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces processor 310's latency, and thus improves system efficiency.

[0090] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. It will be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation of the electronic device 300. In other embodiments of the present application, the electronic device 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0091] The wireless communication functionality of electronic device 300 can be implemented using antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, a modem processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0092] The mobile communication module 350 can provide wireless communication solutions, including 2G / 3G / 4G / 5G / 6G, for the electronic device 300. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 350 can receive electromagnetic waves via antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 350 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 350 may be located within the processor 310. In some embodiments, at least some of the functional modules of the mobile communication module 350 may be located within the same device as at least some of the modules of the processor 310. The modem processor may include a modulator and a demodulator. The modulator is used to modulate a low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator is used to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs audio signals through an audio device (not limited to speaker 370A, receiver 370B, etc.) or displays images or video through display 394. In some embodiments, the modem processor may be a standalone device. In other embodiments, the modem processor may be independent of the processor 310 and be located in the same device as the mobile communication module 350 or other functional modules.

[0093] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to the electronic device 300. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0094] In some embodiments, antenna 1 of electronic device 300 is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, so that electronic device 300 can communicate with a network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0095] The electronic device 300 implements display functionality through a GPU, display screen 394, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute instructions to generate or change display information. The electronic device 300 can implement a camera function through an ISP, one or more cameras 393, a video codec, a GPU, one or more display screens 394, and an application processor.

[0096] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 via the external memory interface 320 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0097] The internal memory 321 can be used to store computer executable program codes, which include instructions. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 300 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 310 executes various functional applications and data processing of the electronic device 300 by running instructions stored in the internal memory 321 and / or instructions stored in a memory provided in the processor.

[0098] The electronic device 300 can implement audio functions such as music playback and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.

[0099] The sensor module 380 may include a pressure sensor 380A, a gyroscope sensor 380B, an air pressure sensor 380C, a magnetic sensor 380D, an acceleration sensor 380E, a distance sensor 380F, a proximity light sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, and the like. The touch sensor 380K is also referred to as a "touch control device." The touch sensor 380K may be disposed on the display screen 394. The touch sensor 380K and the display screen 394 form a touch screen, also referred to as a "touch screen." The touch sensor 380K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor may transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations may be provided via the display screen 394. In other embodiments, the touch sensor 380K may also be disposed on the surface of the electronic device 300, at a location different from that of the display screen 394.

[0100] The gyroscope sensor 380B can be used to determine the motion posture of the electronic device 300. In some embodiments, the angular velocity of the electronic device 300 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 380B. The gyroscope sensor 380B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 380B detects the angle of the electronic device 300 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 300 through reverse movement to achieve anti-shake. The gyroscope sensor 380B can also be used for navigation and somatosensory game scenes.

[0101] Accelerometer 380E can detect the magnitude of acceleration of electronic device 300 in all directions (generally three axes). When electronic device 300 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0102] In some embodiments, the electronic device 300 includes but is not limited to a device equipped with Hongmeng system. Or devices with other operating systems. For example, the electronic device 300 may include a mobile phone, a tablet computer, a drawing board, a stylus, or a wearable device.

[0103] Based on the above-mentioned voice interaction system, an embodiment of the present application provides a voice interaction method, which is described in detail below in conjunction with Figures 4a to 4c.

[0104] The first electronic device obtains an ambient sound signal. In one embodiment of the present application, the first electronic device can directly obtain the ambient sound signal through its own microphone, or it can receive the ambient sound signal sent by the second electronic device. Specifically, the second electronic device obtains the ambient sound signal through its own microphone and sends the ambient sound signal to the first electronic device. In one possible implementation, the microphone is always on to obtain the ambient sound signal in real time.

[0105] When the first electronic device determines that the ambient sound signal includes a sound signal input by the user within a set distance range, it performs speech recognition on the sound signal and generates a voice command. Specifically, when the first electronic device determines that the acquired ambient sound signal meets the conditions for speech spectrum analysis and the conditions for near-field spectrum analysis, it determines that the ambient sound signal includes a sound signal input by the user within the set distance range, performs speech recognition on the sound signal, and generates a voice command. In one embodiment of the present application, the portion of the ambient sound signal that meets the conditions for speech spectrum analysis and near-field spectrum analysis is the sound signal input by the user within the set distance range. Figure 4a is a schematic diagram of a scenario for acquiring an ambient sound signal according to one embodiment of the present application. As shown in Figure 4a, when the first electronic device is a stylus, a microphone is provided at the rear end of the stylus. When the user brings the rear end of the stylus close to their mouth and speaks, the first electronic device can acquire the ambient sound signal. When the first electronic device is a mobile phone, a microphone is provided at the lower end of the mobile phone. When the user brings the lower end of the mobile phone close to their mouth and speaks, the first electronic device can acquire the ambient sound signal. The number of microphones can be one or more. The ambient sound signal can include the user's voice, the voices of passersby, and various noises. The first electronic device needs to determine whether the ambient sound signal meets the conditions for speech spectrum analysis and near-field spectrum analysis. In one possible implementation, when the first electronic device identifies the ambient sound signal based on a voiceprint recognition algorithm and determines the speaker corresponding to the ambient sound signal, it determines that the ambient sound signal meets the conditions for speech spectrum analysis and thus determines that the current ambient sound signal includes human speech. In one possible implementation, the first electronic device identifies one or more speakers corresponding to the ambient sound signal based on the voiceprint recognition algorithm; generates a pickup distance corresponding to each speaker based on the acquired pickup model; and determines that the ambient sound signal meets the conditions for near-field spectrum analysis when the pickup distance corresponding to one or more speakers is less than a set distance threshold. The first electronic device can learn near-field spectrum features using a neural network to generate a pickup model to determine whether the ambient sound signal meets near-field characteristics. In another possible implementation, the first electronic device performs near-field spectrum analysis on the ambient sound signal based on the near-field evanescent wave effect to generate the pickup distances corresponding to one or more speakers; and determines that the ambient sound signal meets the conditions for near-field spectrum analysis when the pickup distances corresponding to one or more speakers are less than a set distance threshold. In one embodiment of the present application, a distance threshold can be set based on actual conditions, for example, a distance threshold of 10 cm. Specifically, the first electronic device generates sound pressure information and / or sound velocity information of the ambient sound signal based on the ambient sound signal; generates pickup distances corresponding to one or more speakers based on the sound velocity information; or generates pickup distances corresponding to one or more speakers based on the sound pressure information and sound velocity information.Among them, the near-field evanescent wave effect means that a general omnidirectional microphone can collect sound pressure information in space. A directional microphone can be considered to be the first-order derivative of the sound pressure information to a certain extent, that is, pressure difference information or sound speed information, and the sound speed information and sound pressure information change differently when close to the sound source. Among them, the directionality of the microphone means that the microphone can pick up ambient sound from different directions. When a directional microphone picks up ambient sound from different directions, different sound speed information can be obtained in different directions. For example, the direction in which the stylus picks up ambient sound can be along the stylus shaft pointing to the end of the stylus. Figure 4b is a schematic diagram of the relationship between the sound speed information, sound pressure information and the pickup distance provided in an embodiment of the present application. As shown in Figure 4b, Un is the sound speed information, in m / s, P is the sound pressure information, in Pa, r is the pickup distance, in cm, that is, the distance between the detection position and the sound source, and level is the size of the sound speed information or sound pressure information. At a long distance, the magnitude of the sound pressure information and the sound speed information are almost the same. As the pickup distance decreases, the sound speed information becomes larger than the sound pressure information. This is mainly because the propagation model of sound is closer to a spherical wave when near the sound-emitting body, while it can be considered as a plane wave at a long distance. Figure 4c is a schematic diagram of the relationship between the sound speed information, the pickup distance and the sound frequency provided in an embodiment of the present application. As shown in Figure 4c, level is the magnitude of the sound speed information, in m / s, and F is the sound frequency, in Hz. When the pickup distance is 5 cm, the magnitude of the sound speed information of the same sound frequency is greater than the magnitude of the sound speed information of the same sound frequency when the pickup distance is 50 cm. Therefore, when the stylus uses only one microphone, the microphone can be a sound speed microphone to collect the sound speed information of the user's speech. In addition, when the first electronic device uses multiple microphones, the multiple microphones can simultaneously obtain a plurality of sound pressure information and a plurality of sound speed information. In an embodiment of the present application, the setting of the low-frequency signal and the high-frequency signal of the ambient sound signal is related to the pickup distance. For example, when the pickup distance is 5cm, signals below 1100Hz are low-frequency signals, and signals at 1100Hz and above are high-frequency signals. For another example, when the pickup distance is 10cm, signals below 600Hz are low-frequency signals, and signals at 600Hz and above are high-frequency signals. The above-mentioned near-field evanescent wave effect can make the receiving sensitivity of a close-range sound source greater when using a directional microphone than an omnidirectional microphone. The larger the wavelength, the more obvious the sound wave enhancement effect, and the enhancement effect of the low-frequency signal reflected in the ambient sound signal is more obvious. The distance at which the high-frequency signal triggers the near-field evanescent wave effect needs to be closer than the low-frequency signal, and the degree of enhancement is also lower than the low-frequency part. Based on the above characteristics, it is possible to better distinguish between close-range ambient sound signals and long-range ambient sound signals, and the above function can be achieved using only one directional microphone. In one possible implementation, the first electronic device performs voice recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers to generate a voice command.As an optional solution, the first electronic device performs voice recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers based on a voice recognition algorithm to generate a voice command. The voice command may include an application operation command or a recording command. In one possible implementation, before performing voice recognition on the sound signal, sound enhancement processing may be performed on the ambient sound signal to generate the sound signal. For example, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0106] The first electronic device triggers voice interaction in response to the voice instruction; or sends a voice instruction to the second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0107] To implement the above technical solution, the following is an example of the workflow of the voice interaction method through a specific embodiment. An embodiment of the present application provides a voice interaction method. FIG5 is a flowchart of a voice interaction method provided according to an embodiment of the present application. As shown in FIG5, the method includes:

[0108] Step 102: The first electronic device obtains an ambient sound signal.

[0109] In one possible implementation, one or more microphones are installed in the first electronic device. The microphone is always on to obtain ambient sound signals in real time. For example, when the first electronic device is a stylus, the stylus may be equipped with a sonic microphone. The user holds the stylus, and when the stylus is in use, the microphone is always on. When the user holds the stylus to write and wants to execute certain instructions, the user moves the stylus from the writing state to the mouth and speaks to the stylus to obtain ambient sound signals. For example, the user says "open memo" to the stylus. At this time, the ambient sound signal may include sound signals (the sound of the user speaking "open memo"), the sound of passers-by speaking, and various noises.

[0110] In one possible implementation, when the first electronic device is a smart watch or a mobile phone, the motion sensor of the first electronic device simultaneously executes step 102 to perform the above-mentioned voice interaction method when recognizing the user's wrist raising action or picking up the mobile phone action.

[0111] In one embodiment of the present application, the motion sensor may be an inertial measurement unit (IMU), a compass, a micro-electro-mechanical system (MEMS) accelerometer, or a MEMS gyroscope. The motion sensor may include the gyroscope sensor 380B and the acceleration sensor 380E in FIG. 3 .

[0112] Step 104 : When the first electronic device determines that the ambient sound signal includes a sound signal input by the user within a set distance range, the first electronic device performs sound enhancement processing on the ambient sound signal to generate a sound signal.

[0113] Specifically, when the first electronic device determines that the acquired ambient sound signal satisfies the conditions for speech spectrum analysis and the conditions for near-field spectrum analysis, it determines that the ambient sound signal includes a sound signal input by the user within the set distance range. In one embodiment of the present application, the portion of the ambient sound signal that satisfies the conditions for speech spectrum analysis and the conditions for near-field spectrum analysis is the sound signal input by the user within the set distance range.

[0114] In one possible implementation, when the first electronic device identifies the ambient sound signal based on a voiceprint recognition algorithm and determines the speaker corresponding to the ambient sound signal, it determines that the ambient sound signal meets the conditions for speech spectrum analysis, indicating that the ambient sound signal includes human speech. As an alternative, when the first electronic device identifies that the spectral characteristics of the ambient sound signal include mel-spectrograms, it may indicate that the ambient sound signal includes human speech.

[0115] In one possible implementation, based on the near-field evanescent wave effect, near-field spectrum analysis is performed on the spectral characteristics of the ambient sound signal to generate the pickup distance corresponding to one or more speakers; when the pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal meets the conditions for near-field spectrum analysis.

[0116] In one embodiment of the present application, a distance threshold can be set based on actual conditions, for example, the distance threshold can be set to 10 cm. In one possible implementation, when the first electronic device is a smartwatch or a mobile phone, the motion sensor of the first electronic device increases the sensitivity of near-field sound pickup after recognizing the user's wrist-raising motion or picking up the phone. For example, the set distance threshold can be increased to increase the probability of determining that the ambient sound signal meets the conditions for near-field spectrum analysis.

[0117] As an optional solution, a sound velocity microphone can collect sound velocity information corresponding to the ambient sound signal, and can generate the pickup distance corresponding to one or more speakers based on the sound velocity information.

[0118] As another optional solution, the sound pressure microphone can collect the sound pressure information corresponding to the ambient sound signal, and the sound velocity microphone can collect the sound velocity information corresponding to the ambient sound signal. The pickup distance corresponding to one or more speakers can be generated based on the sound pressure information and sound velocity information.

[0119] Specifically, the first electronic device may store a correspondence between sound speed information, sound pressure information, and sound pickup distance. The correspondence between sound speed information, sound pressure information, and sound pickup distance is shown in the following formula.

[0120] P(r, t) = Re{Ae i(wt-kr) r -1}Wherein, P(r, t) is the sound pressure information, Re represents the real part, k is the wave number, A is a specific constant, w is the angular frequency, r is the pickup distance, and t is the time.

[0121] Among them, u(r, t) is the sound speed information, A is a specific constant, w is the angular frequency, r is the pickup distance, ρ0 is the air density, and t is time.

[0122] In one embodiment of the present application, for the sound pressure information, only the real part is considered, which decays with distance and is proportional to 1 / r; for the sound speed information, the first half of the plus sign is the imaginary part, and the second half is the real part. The second half is the same as the sound pressure information (only the difference is a constant), and the first half is the additional term of the sound speed information, which is proportional to 1 / r2. It can provide a more obvious contribution only when the r value is small, and can be ignored when r is large.

[0123] In one embodiment of the present application, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing. For example, the noise suppression processing may include linear filtering, spectral subtraction, statistical model algorithms, or subspace algorithms. The reverberation suppression processing may include a weighted prediction error (WPE) algorithm.

[0124] In one embodiment of the present application, the above algorithm can also be used to enhance the sound of the ambient sound signal of the user speaking softly. The main difference between speaking softly and speaking normally is that speaking softly is just the vibration of the air near the mouth, while speaking normally is the vibration of the vocal cords. Therefore, there is a significant difference in the energy of speaking softly and speaking normally, and there is a significant difference in the fundamental frequency part, but there is no significant difference in the spectrum of speaking softly. Unlike the plosive sound detection in the related art, this application uses the spectrum information and / or sound speed information of the ambient sound signal to detect the ambient sound signal, and does not need to rely on the energy of the ambient sound signal for detection. Therefore, this application can realize the soft-voice wake-up function of electronic devices in near-field voice interaction scenarios.

[0125] Step 106: The first electronic device performs voice recognition on the sound signal to generate a voice command.

[0126] In one possible implementation, the first electronic device performs speech recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers, and generates a voice instruction. As an optional solution, the first electronic device performs speech recognition on the sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers based on a speech recognition algorithm, and generates a voice instruction. The speech recognition algorithm may include a Hidden Markov Model (HMM), WFST-based speech decoding, etc. The voice instruction may include application operation instructions or recording instructions. For example, the application operation instructions include opening a music APP, recording tomorrow's schedule, setting an alarm, etc. When the speech recognition does not recognize the application operation instruction, the recording instruction is triggered, that is, the user's speech sound is recorded.

[0127] For example, the first electronic device performs voice recognition on the sound signal "open memo" and may generate a voice command "open memo APP".

[0128] Step 108: The first electronic device sends a voice command to the second electronic device.

[0129] For example, the second electronic device is a mobile phone, and the first electronic device (stylus) sends a voice command "open the memo APP" to the second electronic device (mobile phone).

[0130] Step 110: The second electronic device triggers voice interaction in response to the voice instruction.

[0131] For example, the second electronic device (mobile phone) triggers voice interaction in response to the voice command "open the memo APP", that is, the memo APP installed in the second electronic device (mobile phone) is automatically opened.

[0132] In the technical solution provided by one embodiment of the present application, an ambient sound signal is obtained; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal to generate a voice instruction; in response to the voice instruction, a voice interaction is triggered; or a voice instruction is sent to a second electronic device, the voice instruction being used to trigger the second electronic device to perform voice interaction. The technical solution provided by an embodiment of the present application determines whether the ambient sound signal includes a sound signal input by the user within a set distance range, and performs voice recognition on the sound signal input by the user within the set distance range to trigger voice interaction, thereby realizing a soft voice wake-up function for electronic devices in near-field voice interaction scenarios.

[0133] In the technical solution provided in one embodiment of the present application, the voice interaction capability of the first electronic device with a single microphone can be used to distinguish between far-field and near-field ambient sound signals, and voice enhancement can be performed on near-field sound signals in a targeted manner.

[0134] In the technical solution provided in one embodiment of the present application, the spectral features of the voice spectrum and the near-field spectrum can be used to identify and trigger voice commands for voice interaction, thereby realizing a wake-up-free voice interaction function.

[0135] The technical solution provided in one embodiment of the present application solves the problem of large computing power processing required for wake-up actions and wake-up recognition in related technologies, provides a solution for automatically identifying effective ambient sound signals, and then performs voice enhancement and wake-up triggering and other processing, which can further realize time-frequency point-level judgment and noise reduction, writing and other post-processing, thereby improving the user's efficiency when using multiple electronic devices.

[0136] An embodiment of the present application provides a voice interaction device. Figure 6 is a structural schematic diagram of a voice interaction device provided according to an embodiment of the present application. As shown in Figure 6, the device includes: a microphone module 61, a judgment module 62, a voice recognition module 63, a voice interaction module 64 and a wireless communication module 65.

[0137] The microphone module 61 is used to obtain ambient sound signals.

[0138] The determination module 62 is configured to determine whether the ambient sound signal includes a sound signal input by the user within a set distance range.

[0139] The voice recognition module 63 is configured to perform voice recognition on the sound signal and generate a voice command when the judgment module 62 judges that the ambient sound signal includes a sound signal input by the user within a set distance range.

[0140] The voice interaction module 64 is used to trigger voice interaction in response to a voice instruction; or the wireless communication module 65 is used to send a voice instruction to the second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

[0141] In one embodiment of the present application, the judgment module 62 is specifically used to: identify one or more speakers corresponding to the ambient sound signal; generate the pickup distance corresponding to each speaker based on the acquired pickup model; when the pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0142] In one embodiment of the present application, the judgment module 62 is specifically used to: generate the pickup distance corresponding to one or more speakers based on the ambient sound signal; when the pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

[0143] In one embodiment of the present application, the judgment module 62 is specifically used to: generate sound pressure information and / or sound speed information of the ambient sound signal based on the ambient sound signal; generate the pickup distance corresponding to one or more speakers based on the sound speed information; or generate the pickup distance corresponding to one or more speakers based on the sound pressure information and sound speed information.

[0144] In one embodiment of the present application, when the number of microphones in the microphone module 61 is one, the microphone is a speed-of-sound microphone; the judgment module 62 is specifically used to generate speed information of the ambient sound signal based on the ambient sound signal; and generate the pickup distance corresponding to one or more speakers based on the speed-of-sound information.

[0145] In one embodiment of the present application, when there are multiple microphones in the microphone module 61, the microphones include a sound velocity microphone and a sound pressure microphone; the judgment module 62 is specifically used to generate sound pressure information and sound velocity information of the ambient sound signal based on the ambient sound signal; and generate the pickup distance corresponding to one or more speakers based on the sound pressure information and sound velocity information.

[0146] In one embodiment of the present application, the microphone is always on to obtain ambient sound signals in real time.

[0147] In one embodiment of the present application, the speech recognition module 63 is specifically configured to perform speech recognition on a sound signal corresponding to the smallest pickup distance among the pickup distances corresponding to one or more speakers, and generate a speech instruction.

[0148] In one embodiment of the present application, the voice command includes an application operation command or a recording command.

[0149] In one embodiment of the present application, the device further includes a sound enhancement module 66 .

[0150] The sound enhancement module 66 is used to perform sound enhancement processing on the ambient sound signal to generate a sound signal.

[0151] In one embodiment of the present application, the sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

[0152] In one embodiment of the present application, the device further includes a receiving module 67 .

[0153] The receiving module 67 is configured to receive an ambient sound signal sent by the second electronic device.

[0154] In the technical solution provided by one embodiment of the present application, an ambient sound signal is obtained; when it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, voice recognition is performed on the sound signal to generate a voice instruction; voice interaction is triggered in response to the voice instruction; or a voice instruction is sent to a second electronic device, the voice instruction being used to trigger the second electronic device to perform voice interaction. The technical solution provided by the embodiment of the present application, by determining whether the ambient sound signal includes a sound signal input by the user within a set distance range, and triggering voice interaction by voice recognition of the sound signal input by the user within the set distance range, can realize a quiet wake-up function for electronic devices in near-field voice interaction scenarios.

[0155] In the description of the embodiments of the present application, for the convenience of description, the device is described as being divided into various modules according to their functions. The division of each module is merely a division of logical functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0156] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; or they can all be implemented in the form of hardware; or some modules can be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. During the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0157] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0158] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.

[0159] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0161] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this application.

[0162] Specifically, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0163] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program product is run on a computer, it enables the computer to execute the method provided in the embodiment of the present application.

[0164] The embodiment description in this application is described with reference to the flow chart and / or block diagram according to the method, equipment (device) and computer program product of embodiment of the present application.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing equipment to produce a machine, so that the instruction executed by the processor of computer or other programmable data processing equipment produces the device for realizing the function specified in one flow chart flow chart or multiple flow charts and / or one block or multiple blocks of block diagram.

[0165] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0167] It should also be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0168] In the embodiments of the present application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the element.

[0169] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0170] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0171] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0173] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A voice interaction method, characterized in that: The method is applied to a first electronic device, and the method includes: Acquire environmental sound signals; When it is determined that the ambient sound signal includes a sound signal input by the user within a set distance range, performing voice recognition on the sound signal to generate a voice command; In response to the voice instruction, voice interaction is triggered; or, the voice instruction is sent to a second electronic device, where the voice instruction is used to trigger the second electronic device to perform voice interaction.

2. The method according to claim 1, characterized in that The determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes: Identifying one or more speakers corresponding to the ambient sound signal; Generate the pickup distance corresponding to each speaker according to the acquired pickup model; When the sound pickup distance corresponding to one or more of the speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

3. The method according to claim 1, characterized in that The determining that the ambient sound signal includes a sound signal input by a user within a set distance range specifically includes: generating a sound pickup distance corresponding to one or more speakers according to the ambient sound signal; When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

4. The method according to claim 3, characterized in that The generating the sound pickup distances corresponding to one or more speakers according to the ambient sound signal specifically includes: generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or, The sound pickup distances corresponding to one or more speakers are generated according to the sound pressure information and the sound speed information.

5. The method according to any one of claims 2 to 4, characterized in that: The performing speech recognition on the sound signal to generate a speech instruction specifically includes: Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more of the speakers to generate a voice command.

6. The method according to claim 1, characterized in that The voice instructions include application operation instructions or recording instructions.

7. The method according to claim 1, characterized in that Before performing speech recognition on the sound signal, the method further includes: Perform sound enhancement processing on the ambient sound signal to generate a sound signal.

8. The method according to claim 7, characterized in that The sound enhancement processing includes noise suppression processing and / or reverberation suppression processing.

9. The method according to claim 1, characterized in that: The obtaining of the ambient sound signal specifically includes: Receive the ambient sound signal sent by the second electronic device.

10. A voice interaction device, characterized in that: Applicable to a first electronic device, comprising: Microphone module, used to obtain ambient sound signals; A determination module, used to determine whether the ambient sound signal includes a sound signal input by a user within a set distance range; The speech recognition module is used to, when the judgment module determines that the ambient sound signal includes a sound input by the user within a set distance range, When a sound signal is received, voice recognition is performed on the sound signal to generate a voice command; A voice interaction module is used to trigger voice interaction in response to the voice instruction; or a wireless communication module is used to send the voice instruction to a second electronic device, and the voice instruction is used to trigger the second electronic device to perform voice interaction.

11. The device according to claim 10, characterized in that The judgment module is specifically used for: Identifying one or more speakers corresponding to the ambient sound signal; Generate the pickup distance corresponding to each speaker according to the acquired pickup model; When the sound pickup distance corresponding to one or more of the speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

12. The device according to claim 10, characterized in that The judgment module is specifically used for: generating a sound pickup distance corresponding to one or more speakers according to the ambient sound signal; When the sound pickup distance corresponding to one or more speakers is less than a set distance threshold, it is determined that the ambient sound signal includes a sound signal input by the user within the set distance range.

13. The device according to claim 12, characterized in that The judgment module is specifically used for: generating sound pressure information and / or sound speed information of the ambient sound signal according to the ambient sound signal; Generate the pickup distance corresponding to one or more speakers according to the sound speed information; or, The sound pickup distances corresponding to one or more speakers are generated according to the sound pressure information and the sound speed information.

14. The device according to claim 13, characterized in that Also includes: When the number of microphones in the microphone module is one, the microphone is a sonic velocity microphone; The judgment module is specifically configured to generate sound speed information of the ambient sound signal according to the ambient sound signal; and generate sound pickup distances corresponding to one or more speakers according to the sound speed information.

15. The device according to claim 13, characterized in that Also includes: When there are multiple microphones in the microphone module, the microphones include a sound velocity microphone and a sound pressure microphone; The judgment module is specifically used to generate sound pressure information and sound speed information of the ambient sound signal according to the ambient sound signal; and generate sound pickup distances corresponding to one or more speakers according to the sound pressure information and sound speed information.

16. The device according to claim 15, characterized in that The microphone is always turned on to obtain the ambient sound signal in real time.

17. The device according to any one of claims 11 to 13, characterized in that The speech recognition module is specifically used for: Speech recognition is performed on a sound signal corresponding to the smallest sound pickup distance among the sound pickup distances corresponding to one or more of the speakers to generate a voice command.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 9.

19. A first electronic device, characterized in that: The first electronic device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the first electronic device is triggered to execute the method steps as described in any one of claims 1-9.

Images