A voice wake-up method, electronic device, wearable device and system

Abstract

The present application provides a method, electronic device, wearable device and system for voice wake-up, the system includes an electronic device and a wearable device, the electronic device communicates with the wearable device through a short-range wireless connection, the electronic device is used to collect voice signals in the environment; the electronic device is also used to send a query request to the wearable device when the voice signal meets a preset condition, the query request is used to request information about the user speaking; the wearable device is used to send a query result to the electronic device, the query result includes information about the user speaking; the electronic device is also used to enter a wake-up state when it is determined that the user is speaking based on the information about the user speaking. The embodiments of the present application help to improve the accuracy of the electronic device when performing voice wake-up.

Description

Technical Field

[0001] The present application relates to the field of terminals, and more specifically, to a voice wake-up method, an electronic device and a wearable device. Background Art

[0002] Although the current intelligent voice devices on the market have made great progress in voice wake-up, noise reduction, and recognition accuracy, they are still not very good at recognizing human voices in the presence of background noise. In particular, when the user is far away from the device to be woken up and there is background noise, the wake-up rate will be even lower and there will be a high number of false wake-ups. Summary of the invention

[0003] The present application provides a voice wake-up method, electronic device, wearable device and system, which help to improve the accuracy of voice wake-up of electronic devices.

[0004] In a first aspect, a system is provided, which includes an electronic device and a wearable device, wherein the electronic device communicates with the wearable device via a short-range wireless connection, and the electronic device is used to collect voice signals in the environment; the electronic device is also used to send a query request to the wearable device when the voice signal meets a preset condition, and the query request is used to request information about a user who is speaking; the wearable device is used to send a query result to the electronic device, and the query result includes information about a user who is speaking; and the electronic device is also used to enter a wake-up state when it is determined that a user is speaking based on the information about the user who is speaking.

[0005] In the embodiment of the present application, after the electronic device determines that the voice signal contains the wake-up word and the voiceprint information matches the preset voiceprint information, it can query the wearable device for information about the user speaking. When the electronic device determines that the user is speaking, it performs a wake-up operation to enter the wake-up state. Using a wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0006] In some possible implementations, the preset condition is that the voice signal contains a wake-up word; or, the preset condition is that the voiceprint information of the voice signal matches the preset voiceprint information; or, the preset condition is that the voice signal contains a wake-up word and the voiceprint information of the voice signal matches the preset voiceprint information.

[0007] In some possible implementations, the information that the user is speaking includes data detected by a sensor of the wearable device, and the electronic device is further used to determine that the user is speaking based on the data detected by the sensor of the wearable device.

[0008] In the embodiment of the present application, the electronic device can be used to have strong computing power. The wearable device only needs to send the data detected by the sensor to the electronic device, which will make the judgment. This helps to reduce the delay of the electronic device during the voice wake-up process and also helps to improve the accuracy of voice wake-up.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the information that the user is speaking is used to indicate a first confidence and a second confidence, the first confidence being the confidence that the user is wearing the wearable device, and the second confidence being the confidence that the user is speaking; the electronic device is specifically used to: enter a wake-up state when the first confidence is greater than or equal to a first preset value and the second confidence is greater than or equal to a second preset value.

[0010] In the embodiment of the present application, the electronic device can determine whether to perform a wake-up operation based on the first confidence level and the second confidence level indicated in the information that the user is speaking. The electronic device can perform a wake-up operation when it is determined that the user is wearing a wearable device and the user is speaking, thereby entering a wake-up state. Using a wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the wearable device includes an acoustic sensor, and the wearable device is specifically used to: determine the first confidence level based on the frequency of a sound signal of a human heartbeat or breathing detected by the acoustic sensor within a preset detection period; and determine the second confidence level based on the intensity of the sound signal within a preset frequency range.

[0012] In the embodiment of the present application, the wearable device can use an acoustic sensor to detect the sound signal of the human heartbeat or breathing to determine the first confidence level and the second confidence level. In this way, the electronic device can determine whether the user is speaking based on the first confidence level and the second confidence level. Using a wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0013] In some possible implementations, the information that the user is speaking may include the frequency of the sound signal of the human heartbeat or breathing detected by the acoustic sensor of the wearable device within a preset period, and the intensity of the sound signal within a preset frequency range. The electronic device may determine the first confidence level and the second confidence level based on the information that the user is speaking. Thus, the electronic device determines whether the user is speaking based on the first confidence level and the second confidence level.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the wearable device includes a photoplethysmography (PPG) sensor, and the wearable device is specifically used to: determine the first confidence level based on the frequency of the PPG signal detected by the PPG sensor within a preset detection period; and determine the second confidence level based on the intensity of the PPG signal within a preset frequency range.

[0015] In the embodiment of the present application, the wearable device can use the PPG signal detected by the PPG sensor to determine the first confidence level and the second confidence level. In this way, the electronic device can determine whether the user is speaking based on the first confidence level and the second confidence level. Using the wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0016] In some possible implementations, the information that the user is speaking may include the frequency of the PPG signal detected by the PPG sensor of the wearable device within a preset detection period and the intensity of the PPG signal within a preset frequency range. The electronic device may determine the first confidence level and the second confidence level based on the information that the user is speaking. Thus, the electronic device determines whether the user is speaking based on the first confidence level and the second confidence level.

[0017] In combination with the first aspect, in some implementations of the first aspect, the wearable device includes an acoustic sensor, and the wearable device is specifically used to: collect sound signals of human heartbeat or breathing through the acoustic sensor; input the sound signals into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model is obtained by collecting noise signals when the user is not wearing the wearable device, the second model is obtained by collecting sound signals when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting sound signals when the user is wearing the wearable device and is speaking.

[0018] In the embodiment of the present application, the wearable device can input the sound signal collected by the sound sensor into the first model, the second model or the third model by machine learning, so as to obtain the first confidence level and the second confidence level. In this way, the electronic device can judge whether the user is speaking by the first confidence level and the second confidence level. Using the wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0019] In some possible implementations, the first model, the second model, and the third model are stored in the electronic device. The information that the user is speaking includes a sound signal of a human heartbeat or breathing collected by an acoustic sensor of the wearable device. The electronic device can input the sound signal into the first model, the second model, and the third model to obtain a first confidence level and a second confidence level. Thus, the electronic device determines whether the user is speaking according to the first confidence level and the second confidence level.

[0020] In combination with the first aspect, in some implementations of the first aspect, the wearable device includes a PPG sensor, and the wearable device is specifically used to: collect a PPG signal through the PPG sensor; input the PPG signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a PPG signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a PPG signal when the user is wearing the wearable device and is speaking.

[0021] In the embodiment of the present application, the wearable device can input the PPG signal collected by the PPG sensor into the first model, the second model or the third model by machine learning, so as to obtain the first confidence level and the second confidence level. In this way, the electronic device can judge whether the user is speaking by the first confidence level and the second confidence level. By using the wearable device to assist the electronic device in voice wake-up, the accuracy of voice wake-up can be improved.

[0022] In some possible implementations, the first model, the second model, and the third model are stored in the electronic device. The information that the user is speaking includes a PPG signal collected by a PPG sensor of the wearable device. The electronic device can input the PPG signal into the first model, the second model, and the third model to obtain a first confidence level and a second confidence level. Thus, the electronic device determines whether the user is speaking according to the first confidence level and the second confidence level.

[0023] In combination with the first aspect, in some implementations of the first aspect, the account logged in on the electronic device is associated with the account logged in on the wearable device.

[0024] In some possible implementations, the account logged in on the electronic device and the account logged in on the wearable device may be the same account; or, the account logged in on the electronic device and the account logged in on the wearable device are accounts in the same family group; or, the account logged in on the wearable device may be an account authorized by the account logged in on the electronic device.

[0025] In a second aspect, a voice wake-up method is provided, which is applied to an electronic device, which communicates with a wearable device via a short-range wireless connection, and the method includes: the electronic device collects a voice signal in the environment; when the voice signal meets a preset condition, the electronic device sends a query request to the wearable device, and the query request is used to request information about what the user is saying; the electronic device receives the query result sent by the wearable device, and the query result includes information about what the user is saying; when it is determined that the user is saying what the user is saying based on the information about what the user is saying, the electronic device enters a wake-up state.

[0026] In the embodiment of the present application, after the electronic device determines that the voice signal contains the wake-up word and the voiceprint information matches the preset voiceprint information, it can query the wearable device for information about the user speaking. When the electronic device determines that the user is speaking, it performs a wake-up operation to enter the wake-up state. Using a wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0027] In some possible implementations, the preset condition is that the voice signal contains a wake-up word; or, the preset condition is that the voiceprint information of the voice signal matches the preset voiceprint information; or, the preset condition is that the voice signal contains a wake-up word and the voiceprint information of the voice signal matches the preset voiceprint information.

[0028] In some possible implementations, the information that the user is speaking includes data detected by a sensor of the wearable device, and the electronic device determines that the user is speaking based on the information that the user is speaking, including: the electronic device determines that the user is speaking based on the data detected by the sensor of the wearable device.

[0029] In some possible implementations, the information that the user is speaking is used to indicate that the user is speaking.

[0030] In combination with the second aspect, in certain implementations of the second aspect, the information that the user is speaking is used to indicate a first confidence and a second confidence, the first confidence being the confidence that the user is wearing the wearable device, and the second confidence being the confidence that the user is speaking, and when it is determined that the user is speaking based on the information that the user is speaking, entering a wake-up state includes: when the first confidence is greater than or equal to a first preset value and the second confidence is greater than or equal to a second preset value, entering a wake-up state.

[0031] In an embodiment of the present application, the electronic device can determine whether to perform a wake-up operation based on the first confidence level and the second confidence level indicated in the information that the user is speaking. The electronic device can perform a wake-up operation when it determines that the user is wearing a wearable device and the user is speaking, thereby entering a wake-up state. Using a wearable device to assist the electronic device in voice wake-up helps to improve the accuracy of voice wake-up.

[0032] In some possible implementations, the information that the user is speaking may include the frequency of the sound signal of the human heartbeat or breathing detected by the acoustic sensor of the wearable device within a preset period, and the intensity of the sound signal within a preset frequency range. The method also includes: the electronic device determines the first confidence level according to the frequency of the sound signal of the human heartbeat or breathing detected by the acoustic sensor within a preset detection period; the electronic device determines the second confidence level according to the intensity of the sound signal within the preset frequency range.

[0033] In some possible implementations, the information that the user is speaking may include the frequency of the PPG signal detected by the PPG sensor of the wearable device within a preset detection period and the strength of the PPG signal within a preset frequency range. The method also includes: the electronic device determines the first confidence level according to the frequency of the PPG signal detected by the PPG sensor within the preset detection period; the electronic device determines the second confidence level according to the strength of the PPG signal within the preset frequency range.

[0034] In some possible implementations, the first model, the second model, and the third model are stored in an electronic device. The first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking. The information that the user is speaking includes a sound signal of a human heartbeat or breathing collected by an acoustic sensor of the wearable device. The method also includes: the electronic device inputs the sound signal into the first model, the second model, and the third model to obtain a first confidence level and a second confidence level.

[0035] In some possible implementations, the first model, the second model, and the third model are stored in the electronic device. The first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a PPG signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a PPG signal when the user is wearing the wearable device and speaking. The information that the user is speaking includes a PPG signal collected by a PPG sensor of the wearable device. The method also includes: the electronic device inputs the PPG signal into the first model, the second model, and the third model to obtain the first confidence level and the second confidence level.

[0036] In combination with the second aspect, in some implementations of the second aspect, the account logged in on the electronic device is associated with the account logged in on the wearable device.

[0037] In some possible implementations, the account logged in on the electronic device and the account logged in on the wearable device may be the same account; or, the account logged in on the electronic device and the account logged in on the wearable device are accounts in the same family group; or, the account logged in on the wearable device may be an account authorized by the account logged in on the electronic device.

[0038] In a third aspect, a voice wake-up method is provided, which is applied to a wearable device, and the wearable device communicates with an electronic device via a short-range wireless connection. The method includes: receiving a query request sent by the electronic device, and the query request is used to request information about what the user is speaking; and sending a query result to the electronic device, and the query result includes information about what the user is speaking.

[0039] In combination with the third aspect, in certain implementations of the third aspect, the information that the user is speaking is used to indicate a first confidence and a second confidence, the first confidence being the confidence that the user is wearing the wearable device, and the second confidence being the confidence that the user is speaking.

[0040] In combination with the third aspect, in certain implementations of the third aspect, the wearable device includes an acoustic sensor, and before sending the query result to the electronic device, the method also includes: determining the first confidence level based on the frequency of the sound signal of human heartbeat or breathing detected by the acoustic sensor within a preset detection period; and determining the second confidence level based on the intensity of the sound signal within a preset frequency range.

[0041] In combination with the third aspect, in certain implementations of the third aspect, the wearable device includes a PPG sensor, and before sending the query result to the electronic device, the method also includes: determining the first confidence level based on the frequency of the PPG signal detected by the PPG sensor within a preset detection period; and determining the second confidence level based on the intensity of the PPG signal within a preset frequency range.

[0042] In combination with the third aspect, in some implementations of the third aspect, the wearable device includes an acoustic sensor, and before sending the query result to the electronic device, the method also includes: collecting sound signals of human heartbeat or breathing through the acoustic sensor; inputting the sound signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model being obtained by collecting noise signals when the user is not wearing the wearable device, the second model being obtained by collecting sound signals when the user is wearing the wearable device and not speaking, and the third model being obtained by collecting sound signals when the user is wearing the wearable device and is speaking.

[0043] In combination with the third aspect, in some implementations of the third aspect, the wearable device includes a PPG sensor, and before sending the query result to the electronic device, the method also includes: collecting a PPG signal through the PPG sensor; inputting the PPG signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model being obtained by collecting a noise signal when the user is not wearing the wearable device, the second model being obtained by collecting a PPG signal when the user is wearing the wearable device and not speaking, and the third model being obtained by collecting a PPG signal when the user is wearing the wearable device and is speaking.

[0044] In combination with the third aspect, in some implementations of the third aspect, the account logged in on the electronic device and the account logged in on the wearable device are associated.

[0045] In a fourth aspect, a voice wake-up device is provided, which is included in an electronic device, and has the functions of the electronic device in the second aspect and the possible implementation of the second aspect. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0046] In a fifth aspect, a voice wake-up device is provided, which is included in a wearable device, and has the function of implementing the wearable device in the third aspect and the possible implementation of the third aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0047] In a sixth aspect, an electronic device is provided, comprising: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device performs the voice wake-up method in any possible implementation of the second aspect.

[0048] In a seventh aspect, a wearable device is provided, comprising: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the wearable device, the wearable device performs the voice wake-up method in any possible implementation of the third aspect.

[0049] In an eighth aspect, a chip system is provided, wherein the chip system is located in an electronic device, and the chip system includes a system on chip (SOC), and the SOC is used to control a microphone to collect voice signals in the environment in which the electronic device is located; the SOC is also used to control a wireless communication module to send a query request to the wearable device when it is determined that the voice signal meets a preset condition, and the query request is used to request information about what the user is speaking; the SOC is also used to control the wireless communication module to receive a query result sent by the wearable device, and the query result includes information about what the user is speaking; the SOC is also used to enter a wake-up state when it is determined that the user is speaking based on the information about what the user is speaking.

[0050] In a ninth aspect, a chip system is provided, which is located in a wearable device, and the wearable device includes a SOC, and the SOC is used to control the wireless communication module to receive a query request sent by an electronic device, and the query request is used to request information about what the user is speaking; the SOC is also used to control the wireless communication module to send a query result to the electronic device, and the query result includes information about what the user is speaking.

[0051] In the tenth aspect, a computer storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the voice wake-up method in any possible implementation of the second aspect; or, the wearable device executes the voice wake-up method in any possible implementation of the third aspect.

[0052] In the eleventh aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the voice wake-up method in any possible implementation of the second aspect; or, the wearable device executes the voice wake-up method in any possible implementation of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0054] Figure 2 It is a set of graphical user interfaces provided by the embodiments of the present application.

[0055] Figure 3 It is a schematic flowchart of the voice wake-up method provided in an embodiment of the present application.

[0056] Figure 4 is a schematic diagram of a wearable device provided in an embodiment of the present application.

[0057] Figure 5 is another schematic diagram of the wearable device provided in an embodiment of the present application.

[0058] Figure 6 This is another schematic flowchart of the voice wake-up method provided in an embodiment of the present application.

[0059] Figure 7 It is a schematic block diagram of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0061] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0062] The following describes an electronic device designed by the embodiments of the present application, a user interface for such an electronic device, and an embodiment for using such an electronic device. In some embodiments, the electronic device may be a portable electronic device including other functions, such as a mobile phone, a tablet computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to carrying Or a portable electronic device with other operating systems. The portable electronic device may also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the electronic device may not be a portable electronic device, but a desktop computer. In some embodiments, the electronic device may be a smart home appliance, such as a smart speaker, a smart home device, and the like.

[0063] For example, Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0064] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0065] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0066] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0067] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0068] In the embodiment of the present application, the processor 110 may include a wake-up processing module and a voiceprint processing module described in the following embodiments.

[0069] Exemplarily, the wake-up processing module in the processor 110 may analyze whether the voice signal in the environment contains the wake-up word, thereby determining whether it is a false wake-up.

[0070] Exemplarily, the voiceprint processing module in the processor 110 may analyze the similarity between the voiceprint information in the speech signal and the voiceprint preset by the user.

[0071] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0072] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple groups of I2S buses. The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. The UART interface is a universal serial data bus for asynchronous communication. The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. The USB interface 130 is an interface that complies with the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc.

[0073] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0074] The charging management module 140 is used to receive charging input from a charger.

[0075] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0076] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0077] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0078] In an embodiment of the present application, the electronic device 100 can send a query request to the wearable device through the wireless communication module 160, where the query request is used to request the wearable device to determine whether the user is speaking; and can also receive query results from the wearable device through the wireless communication module 160.

[0079] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0080] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor. The display screen 194 is used to display images, videos, etc. The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture static images or videos.

[0081] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0082] Exemplarily, the wake-up processing module and voiceprint processing module may also be included in an NPU computing processor.

[0083] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0084] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0085] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0086] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0087] In the embodiment of the present application, the electronic device 100 can receive voice signals in the environment through the microphone 170C.

[0088] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0089] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B.

[0090] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0091] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can detect the opening and closing of the flip leather case by using the magnetic sensor 180D.

[0092] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0093] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0094] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0095] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the sensed brightness of the ambient light.

[0096] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0097] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.

[0098] The touch sensor 180K is also called a “touch panel.” The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen.”

[0099] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of a human vocal part.

[0100] Before introducing the embodiments of the present application, several concepts in voice wake-up are first introduced.

[0101] Wake-up word: a string of characters used to wake up an electronic device. For example, the wake-up word is "Xiaoyi Xiaoyi".

[0102] Voice wake-up operation: Voice wake-up operation includes two parts: wake-up operation and recognition operation.

[0103] Among them, the wake-up operation refers to the user saying the wake-up word to wake up the electronic device, so that the electronic device is in a state of waiting for voice commands; or the wake-up operation refers to the user saying the wake-up word, so that the electronic device enters the wake-up state; or the wake-up operation can be that the voiceprint information of the voice signal received by the electronic device matches the preset voiceprint information, so that the electronic device enters the wake-up state.

[0104] Voice command: A voice command used to control an electronic device to perform a corresponding voice operation. For example, a voice operation can be "help me book a flight from Beijing to Shanghai tomorrow morning", "navigate home", "play music", etc.

[0105] Recognition operation: After the electronic device is awakened, the user speaks a voice command to control the electronic device to perform the corresponding voice operation.

[0106] At present, when waking up a device, there is the following problem: if the user wakes up the waking device while the playback device is playing audio, the audio of the playback device may interfere with the waking device, causing the waking device to be woken up by mistake or fail to wake up. For example, when a smart device (such as a mobile phone) is placed next to a TV, because the TV series says "Xiao Yi Xiao Yi", which is the same as the wake-up word "Xiao Yi Xiao Yi" of the smart device, it will cause the smart device to be woken up by mistake.

[0107] In the embodiment of the present application, the characteristic of the human body as a conductor of sound is utilized, and a sensor on the side of the wearable device close to the human skin is used to detect voice signals in the human body to determine whether the user is speaking, which helps to improve the accuracy of voice wake-up of the electronic device.

[0108] Figure 2 A set of graphical user interfaces (GUIs) provided by an embodiment of the present application is shown.

[0109] like Figure 2 As shown, the user sends a voice command "Xiaoyi Xiaoyi" containing a wake-up word to the mobile phone. After receiving the user's voice command, the mobile phone can determine whether the voice command contains the wake-up word and whether the voiceprint information in the voice command matches the voiceprint information preset in the mobile phone. If the mobile phone determines that the voice command contains the wake-up word and the voiceprint information matches the voiceprint information preset in the mobile phone, then the mobile phone can send a query request to the user's wearable device, and the query request is used to query whether the user is speaking; the wearable device can use a sensor close to the side of the human body to determine whether the user is speaking; the wearable device can send the query result to the mobile phone, and the query result includes information that the user is speaking; after the mobile phone determines that the user is speaking based on the information that the user is speaking, it performs a wake-up operation, thereby entering the wake-up state. As shown Figure 2As shown, the mobile phone can reply to the user "I heard a moving voice calling me". In the embodiment of the present application, when the mobile phone replies to the user "I heard a moving voice calling me", it can indicate that the mobile phone has entered the awake state.

[0110] In an embodiment of the present application, when performing a wake-up operation on an electronic device, the electronic device can continue to send a query request to the wearable device after determining that the voice command contains a wake-up word and the voiceprint information of the voice command matches the voiceprint information preset in the electronic device. The wearable device can assist the electronic device in determining whether the user is speaking. When the electronic device determines that the user is speaking based on the query result, the electronic device can perform a wake-up operation. The way in which the wearable device detects the sound in the human body helps to improve the accuracy of voice wake-up of the electronic device.

[0111] Figure 3 FIG. 3 is a schematic flow chart of a method 300 for voice wake-up according to an embodiment of the present application. Figure 3 As shown, the method 300 can be performed by an electronic device and a wearable device, and the method 300 includes:

[0112] S301, the electronic device receives a voice command in an environment.

[0113] For example, Figure 2 As shown, the mobile phone can receive the voice command "Xiaoyi Xiaoyi" in the environment.

[0114] S302: The electronic device determines whether the voice command contains a wake-up word.

[0115] For example, S302 may be performed by Figure 1 The wake-up processing module in the processor 110 executes.

[0116] In one embodiment, the wake-up processing module may be a digital signal processor (DSP). The DSP may process the voice command, and thus analyze whether the voice command contains a wake-up word.

[0117] In one embodiment, the wake-up processing module may include an automatic speech recognition (ASR) module and a natural language understanding (NLU) module.

[0118] Among them, the main function of the ASR module is to recognize the user's voice as text content, and the main function of the NLU module is to understand the user's intent and perform slot analysis.

[0119] For example, Figure 2 As shown, the user issues a voice command "Xiaoyi Xiaoyi". After receiving the voice command, the mobile phone can send the voice command to the ASR module, and the ASR module converts the voice into text information (for example, "Xiaoyi Xiaoyi"). Thus, the electronic device can determine that the voice command contains the wake-up word.

[0120] S303: The electronic device determines whether the voiceprint information of the voice command matches the voiceprint information preset in the electronic device.

[0121] For example, S303 may be performed by Figure 1 The voiceprint processing module in the processor 110 executes.

[0122] The user can save a recording containing the wake-up word in the electronic device in advance, and the electronic device can analyze the user's voiceprint information after obtaining the recording. After the electronic device receives the voice command containing the wake-up word, the electronic device can compare the voice command with the voiceprint of the recording to determine the similarity between the voiceprint information in the voice command and the voiceprint information in the recording. If the similarity is greater than the preset similarity, the electronic device can determine that the voiceprint information of the voice command matches the voiceprint information preset in the electronic device.

[0123] It should be understood that there is no actual order between the above S302 and the above S303. S302 may be executed first and then S303, or S303 may be executed first and then S302.

[0124] It should also be understood that the above S302 and S303 can refer to the process of judging whether the voice command contains a wake-up word and whether the voiceprint information of the voice command matches the preset voiceprint information in the prior art, and the embodiments of the present application are not limited to this. Exemplarily, the electronic device can generate a voice feature vector corresponding to the voice command based on the voice command; the electronic device can match the voice feature vector with the user feature vector. If the match is successful, the electronic device sends a query request to the wearable device.

[0125] It should also be understood that in the embodiment of the present application, the electronic device can send a query request to the wearable device after determining that the voice signal meets the preset condition. The preset condition can be that the voice command in S302 and S303 contains the wake-up word and the voiceprint information of the voice command matches the preset voiceprint information; or, the preset condition can also be that the voice command contains the wake-up word; or, the preset condition can also be that the voiceprint information of the voice command matches the preset voiceprint information.

[0126] For example, when the phone is in the locked screen state and in the off screen state, the phone detects the user's voice command "open the camera". At this time, although the voice command does not contain the wake-up word, the phone can determine that the voiceprint information of the voice command matches the preset voiceprint information. The phone can determine that the user wants to wake up the phone before opening the camera. Then the phone can also send a query request to the wearable device after determining that the voiceprint information of the voice command matches the preset voiceprint information.

[0127] S304, if the electronic device determines that the voice command contains a wake-up word and the voiceprint information of the voice command matches the preset voiceprint information, then the electronic device can send a query request to the wearable device, and the wearable device receives the query request sent by the electronic device, where the query request is used to request information about what the user is saying.

[0128] In one embodiment, before the electronic device sends a query request to the wearable device, the electronic device and the wearable device establish a connection through near field communication. Near field communication includes but is not limited to Wi-Fi, Bluetooth (BT), near field communication technology (NFC) and other near field communication technologies.

[0129] In the embodiment of the present application, the method of sending the query request is not specifically limited. For example, the query request may be a message newly defined by the Wi-Fi protocol or the Bluetooth protocol, and the message may carry a field for requesting information about the user speaking.

[0130] Alternatively, the query request may also be carried in an existing Wi-Fi protocol or Bluetooth protocol message.

[0131] For example, the query request can be carried in a Bluetooth low energy (BLE) data packet. The BLE data packet can be a directional broadcast packet, and the electronic device can know the media access control (MAC) address of the wearable device in advance. Then when the electronic device determines that the voice command contains a wake-up word and the voiceprint information of the voice command matches the preset voiceprint information, the electronic device can send a BLE data packet to the wearable device through the MAC address of the wearable device. The BLE data packet can carry a field that is used to request information about what the user is speaking.

[0132] In one embodiment, the electronic device and the wearable device can be devices under the same account. For example, Huawei account A is logged in on the electronic device, and Huawei account A is also logged in on the wearable device. Then the electronic device can know the address information of the wearable device in advance, so that the electronic device can send the query request to the wearable device. Alternatively, the electronic device can also send the query request to the cloud server, which forwards it to the wearable device.

[0133] S305: The wearable device determines information that the user is speaking based on the query request.

[0134] In the embodiment of the present application, the information that the user is speaking may be the confidence that the user is speaking. The confidence that the user is speaking may be determined by a sensor on the wearable device. Before determining the confidence that the user is speaking, the confidence of the sound signal may be determined first, wherein the confidence of the sound signal may also be understood as the confidence that the user is wearing the wearable device.

[0135] In one embodiment, Figure 4 A schematic diagram of a wearable device is shown. An acoustic sensor can be set on the side of the wearable device close to the skin, and the acoustic sensor can detect sounds in the body through the skin.

[0136] In the embodiment of the present application, the principle of the acoustic sensor is to utilize the characteristic that the human body is also a sound conductor, and to determine the parameters corresponding to the human heartbeat or breathing sound, such as the heartbeat frequency, through the acoustic sensor.

[0137] One method to determine the confidence of a sound signal is to use the frequency and amplitude range of the human heartbeat and breathing sounds as detection rules. For example, the resting heart rate of an adult is 60-100 beats / minute, and it can reach up to 200 beats / minute during exercise, so the cycle frequency of the heartbeat sound is between 1Hz and 3.3Hz; the sound frequency of the heartbeat sound itself is between 20Hz and 500Hz, that is, if a sound of 20Hz-500Hz is detected with a cycle of 1Hz-3.3Hz, the electronic device can determine that the confidence of the sound signal is 1. This method is relatively simple.

[0138] Another method to determine the confidence of a sound signal is a machine learning method. Based on a sound detection model, the heartbeat or breathing sounds collected by the acoustic sensor on the wearable device are used as training data to train a model that can detect heartbeat or breathing sounds. When there is a heartbeat or breathing sound in the input sound signal, the model can output the confidence of whether the human heartbeat or breathing sound is in the input sound signal (the confidence is the confidence of the sound signal). The model has high detection accuracy and strong anti-interference ability.

[0139] After determining the confidence level of the sound signal, the wearable device may continue to determine the confidence level of what the user is saying.

[0140] One method to determine the confidence level of speaking is to extract the speech signal from the background sounds of the human body (heartbeat or breathing sounds, etc.) and calculate the confidence level that the wearer is speaking at the time.

[0141] For example, if the fundamental frequency of an adult female voice is 350 Hz to 3 kHz, the fundamental frequency of an adult male voice is 100 Hz to 900 Hz. Similarly, a sound detection rule may be used to set different sound detection frequencies according to the gender of the user.

[0142] In one embodiment, the wearable device can set the sound detection frequency range to [a, b]. Then the confidence that the user is speaking can be determined by the following formula (1):

[0143] S=P / P0 (1)

[0144] Among them, S is the confidence that the user is speaking, P is the average strength of the sound signal detected in the range of [a, b] during the detection period, and P0 is the preset basic sound strength.

[0145] Another method to determine the confidence of speaking is a machine learning method, which combines the confidence of the sound signal (S1) and the confidence of the user speaking (S2). The sound signal (including heartbeat, breathing and speaking sound) collected by the acoustic sensor on the wearable device when the user is wearing the wearable device and speaking can be used as a training data set A (S1=1, S2=1), the sound signal (including heartbeat, breathing sound) collected when the user is wearing the wearable device and not speaking can be used as a training data set B (S1=1, S2=0), and the noise data (excluding heartbeat, breathing and speaking sound) collected when the user is not wearing the wearable device can be used as a training data set C (S1=0, S2=0). A model that can simultaneously detect heartbeat, breathing sound and speaking sound can be trained. When the sound signal or noise signal detected by the acoustic sensor is input into the model, the model can output S1 and S2 at the same time.

[0146] The above describes a process of determining the confidence level of a user speaking by setting an acoustic sensor on a side of a wearable device close to the skin; the following describes a process of determining the confidence level of a user speaking by using a photoplethysmograph (PPG) sensor in a wearable device.

[0147] Figure 5 A schematic diagram of a PPG sensor is shown, Figure 5As shown, the PPG sensor includes a light emitting component 501 and a light receiving component 502. In the light emitting component 501 of the PPG sensor of the wearable device, a green light emitting diode (LED) is used in combination with a photosensitive photodiode to illuminate the blood. Since different volumes of blood in the blood vessels absorb green light differently, when the heart beats, the blood flow rate increases, and the amount of green light absorbed increases accordingly; when the heart is in the interval between beats, the blood flow decreases, and the amount of green light absorbed also decreases accordingly. Therefore, the heart rate can be measured based on the absorbance of the blood.

[0148] Specifically, when a light beam of a certain wavelength is irradiated onto the surface of the skin, the light beam will pass through the skin and be transmitted to the light receiving component 502. During this process, the light intensity detected by the light receiving component 502 will be weakened due to the attenuation effect of the skin muscle and blood absorption. Among them, the reflection of light by the human skin, bones, meat, fat, etc. is a fixed value, and the capillaries are constantly increasing and decreasing with the pulse volume under the action of the heart. When the heart contracts, the peripheral blood volume is the largest, the light absorption is also the largest, and the light intensity detected by the light receiving component 502 is the smallest; while when the heart is in diastole, on the contrary, the detected light intensity is the largest, causing the light intensity received by the light receiving component 502 to change pulsatingly.

[0149] Since PPG sensors have been widely used to detect heart rate, when users speak, their blood cells will also vibrate, resulting in the difference in the frequency and amplitude of the blood cell vibrations detected when speaking and when the user is not speaking. This allows wearable devices to determine the confidence that the user is speaking.

[0150] In one embodiment, before using the PPG sensor to determine the confidence that the user is speaking, the confidence of the sound signal may be determined first.

[0151] One method for determining the confidence of a sound signal may be to use the vibration frequency and amplitude range of human blood cells as detection rules. The specific process may refer to the above-mentioned process of determining the confidence of a sound signal using an acoustic sensor. For the sake of brevity, it will not be repeated here.

[0152] Another method to determine the confidence of a sound signal is a machine learning method, which is based on a PPG signal detection model. The frequency and amplitude of the PPG signal (or the frequency and amplitude of blood cell vibration) collected by the PPG sensor on the actual wearable device is used as training data to train a model that can detect the frequency and amplitude of the PPG signal. When the frequency and amplitude of the PPG signal detected by the PPG sensor are input into the model, the model can output the confidence of the sound signal. The model has high detection accuracy and strong anti-interference ability.

[0153] After determining the confidence level of the sound signal, the wearable device may then determine the confidence level that the user is speaking.

[0154] One method to determine the confidence that the wearer is speaking is to extract the speech signal from the PPG signal and calculate the confidence that the wearer is speaking at the time.

[0155] The process of determining the confidence that the user is speaking by the frequency of the voice signal can refer to the above process of extracting the voice signal from the human background sound (heartbeat or breathing sound, etc.) and calculating and outputting the confidence that the wearer is speaking at the time. For the sake of brevity, it will not be repeated here.

[0156] Another method to determine the confidence that the user is speaking is a machine learning method, which combines the confidence of the sound signal (S1) and the confidence that the user is speaking (S2). That is, the PPG signal collected by the PPG sensor on the wearable device when the user is wearing the wearable device and speaking can be used as the training data set D (S1 = 1, S2 = 1), the PPG signal collected when the user is wearing the wearable device and not speaking can be used as the training data set E (S1 = 1, S2 = 0), and the noise data (excluding the PPG signal) collected when the user is not wearing the wearable device can be used as the training data set F (S1 = 0, S2 = 0). A model that can detect PPG signals at the same time can be trained. When the PPG signal detected by the wearable device is input into the model, the model can output S1 and S2 at the same time.

[0157] The above is explained by taking the example of using a wearable device to determine the confidence of a sound signal and the confidence that a user is speaking. In the embodiment of the present application, the confidence of a sound signal and the confidence that a user is speaking may also be determined by an electronic device.

[0158] Exemplarily, the query result sent by the wearable device to the electronic device may include data collected by a sensor of the wearable device (e.g., an acoustic sensor or a PPG sensor). For example, the query result may include a sound signal collected by the acoustic sensor. When the electronic device receives the sound signal, the electronic device may input the sound signal into training data set A, training data set B, and training data set C stored in the electronic device, so that the electronic device can obtain the confidence of the sound signal and the confidence that the user is speaking.

[0159] For another example, the query result may include a PPG signal collected by a PPG sensor. After the electronic device receives the PPG signal, the electronic device may input the PPG signal into training data sets D, E, and F stored in the electronic device, so that the electronic device may obtain the confidence of the sound signal and the confidence that the user is speaking.

[0160] It should be understood that the process by which the electronic device determines the confidence of the sound signal and the confidence that the user is speaking through the data collected by the sensor of the wearable device can also refer to the process by which the wearable device determines the confidence of the sound signal and the confidence that the user is speaking, and for the sake of brevity, it will not be repeated here.

[0161] S306, the wearable device sends a query result to the electronic device, and the electronic device receives the query result sent by the wearable device, where the query result includes information that the user is speaking.

[0162] In one embodiment, the information that the user is speaking includes the confidence level of the sound signal and the confidence level that the user is speaking.

[0163] In one embodiment, if the wearable device determines that the confidence of the sound signal is lower than the first preset value, the wearable device can directly indicate in the query result that the user is not wearing the wearable device. At this time, the electronic device can determine whether to enter the awake state according to the existing technology. Alternatively, if the wearable device determines that the confidence of the sound signal is lower than the first preset value, the wearable device can indicate in the query result that the confidence that the user is speaking is unknown, and the electronic device can determine whether to enter the awake state according to the existing technology.

[0164] Alternatively, if the wearable device determines that the confidence level of the sound signal is lower than a first preset value, the wearable device may not send the query result to the electronic device. If the electronic device does not receive the query result within a preset time, the electronic device may determine that the user is not wearing the wearable device. At this time, the electronic device may determine whether to enter the awake state according to the prior art.

[0165] In an embodiment of the present application, after the wearable device determines that the confidence of the sound signal is greater than or equal to the first preset value and the confidence that the user is speaking is greater than or equal to the second preset value, it can indicate in the query result that the user is speaking; or, the wearable device can also carry the confidence of the sound signal and the confidence that the user is speaking in the query result and send it to the electronic device, and the electronic device determines whether the user is speaking. For example, if the electronic device determines that the confidence of the sound signal is greater than or equal to the first preset value and the confidence that the user is speaking is greater than or equal to the second preset value, then the electronic device can determine that the user is speaking, thereby entering the awakening state.

[0166] In the embodiment of the present application, the method of sending the query result is not specifically limited. For example, the query result may be a message newly defined by the Wi-Fi protocol or the Bluetooth protocol, and the message may carry a field for indicating information that the user is speaking.

[0167] Alternatively, the query result may also be carried in an existing Wi-Fi protocol or Bluetooth protocol message.

[0168] Exemplarily, the query result can be carried in a BLE data packet. The BLE data packet can be a directional broadcast packet, and the wearable device can know the MAC address of the electronic device in advance. Then the wearable device can send a BLE data packet to the electronic device according to the MAC address of the electronic device. The BLE data packet can carry a field for requesting information about the user speaking.

[0169] For example, information that the user is speaking can be indicated by 2 bits, "11" means that the user is wearing the wearable device and the user is speaking; "10" means that the user is wearing the wearable device and the user is not speaking; "00" means that the user is not wearing the wearable device.

[0170] For another example, information that the user is speaking may also be indicated by 1 bit, where "1" indicates that the user is speaking; and "0" indicates that the user is not speaking.

[0171] In one embodiment, the information that the user is speaking includes data collected by a sensor of the wearable device (eg, an acoustic sensor or a PPG sensor).

[0172] S307: If the electronic device determines that the user is speaking based on the information that the user is speaking, the electronic device enters a wake-up state.

[0173] It should be understood that in the embodiment of the present application, when the electronic device determines that the user is speaking, the electronic device can perform a wake-up operation, and performing the wake-up operation can cause the electronic device to enter a wake-up state from a non-wake-up state.

[0174] In one embodiment, if the query results received by the electronic device include the confidence of the sound signal sent by the wearable device and the confidence that the user is speaking, the electronic device can perform a wake-up operation when it is determined that the confidence of the sound signal is greater than or equal to a first preset value and the confidence that the user is speaking is greater than or equal to a second preset value, thereby entering a wake-up state.

[0175] In one embodiment, if the query result received by the electronic device indicates that the user is speaking, the electronic device can directly perform a wake-up operation to enter a wake-up state.

[0176] It should be understood that if the query result received by the electronic device includes the confidence of the sound signal sent by the wearable device and the confidence that the user is speaking, and the confidence of the sound signal is less than the first preset value, the electronic device may not enter the wake-up state.

[0177] Alternatively, if the confidence level of the sound signal is greater than or equal to a first preset value and the confidence level that the user is speaking is less than a second preset value, the electronic device may not enter the wake-up state.

[0178] It should also be understood that if the electronic device does not receive the query result within the preset time, the electronic device can be informed that the user is not wearing the wearable device, and the electronic device can determine whether to enter the wake-up state according to the existing technology.

[0179] In one embodiment, the information that the user is speaking includes data collected by a sensor of the wearable device (e.g., an acoustic sensor or a PPG sensor). After receiving the data collected by the sensor of the wearable device, the electronic device can determine the confidence of the sound signal and the confidence that the user is speaking based on the data collected by the sensor of the wearable device. Then, it can be determined whether the user is speaking based on these two confidences.

[0180] In an embodiment of the present application, by detecting human voice, the wearable device can send information that the user is speaking to the electronic device, and the electronic device determines whether the user is speaking based on the information that the user is speaking, which helps to improve the accuracy of voice wake-up.

[0181] Figure 6 FIG. 6 is a schematic flow chart of another voice wake-up method 600 provided in an embodiment of the present application. Figure 6 As shown, the method 600 may be executed by an electronic device, and the method 600 includes:

[0182] S601, receiving a voice command in an environment.

[0183] S602: If the electronic device determines that the voice command meets the preset condition, it sends a query request to the wearable device, where the query request is used to request information about what the user is saying.

[0184] It should be understood that the process of the electronic device determining that the voice command meets the preset conditions can refer to the process S302-303 in the above method 300, and for the sake of brevity, it will not be repeated here.

[0185] It should also be understood that the process of the electronic device sending a query request to the wearable device can refer to S304 in the above method 300, and for the sake of brevity, it will not be repeated here.

[0186] S603, the electronic device determines whether a query result is received within a preset time period.

[0187] In one embodiment, if the electronic device does not receive the query result within a preset time, the electronic device can determine that the wearable device is far away from the electronic device, or the electronic device can also determine that the user is not wearing the wearable device, and then the electronic device can determine whether to enter the wake-up state according to the existing technology. For example, the electronic device enters the wake-up state when it determines that the voice command contains a wake-up word and the voiceprint information of the voice command matches the preset voiceprint information.

[0188] S604: If the electronic device receives the query result within the preset time period and the query result includes information that the user is speaking, the electronic device determines whether the user is speaking based on the information that the user is speaking.

[0189] In one embodiment, the information that the user is speaking includes the confidence of the sound signal and the confidence that the user is speaking. Then the electronic device can enter the awakening state when it is determined that the confidence of the sound signal is greater than or equal to a first preset value and the confidence that the user is speaking is greater than or equal to a second preset value.

[0190] In one embodiment, the information that the user is speaking directly indicates whether the user is speaking. For example, when the information that the user is speaking (for example, the field value carried in the BLE data packet is "1") indicates that the user is speaking, the electronic device may enter the awake state; when the information that the user is speaking (for example, the field value carried in the BLE data packet is "0") indicates that the user is not speaking, the electronic device may not enter the awake state.

[0191] In one embodiment, the information that the user is speaking may also be data collected by a sensor of the wearable device (e.g., an acoustic sensor or a PPG sensor). Then the electronic device can determine whether the user is speaking based on the data collected by the sensor of the wearable device. For example, the electronic device can determine the confidence of the sound signal and the confidence that the user is speaking based on the data collected by the sensor of the wearable device, and thus determine whether the user is speaking based on the two confidences.

[0192] It should be understood that the process of the electronic device determining whether the user is speaking based on the information that the user is speaking can refer to the process S307 in the above method 300, and for the sake of brevity, it will not be repeated here.

[0193] S605: If the electronic device determines that the user is speaking, the electronic device enters a wake-up state.

[0194] In one embodiment, if the electronic device determines that the user is not speaking based on the information that the user is speaking, the electronic device may not enter the awake state.

[0195] In an embodiment of the present application, by detecting human voice, the wearable device can send information that the user is speaking to the electronic device, and the electronic device determines whether the user is speaking based on the information that the user is speaking, which helps to improve the accuracy of voice wake-up.

[0196] An electronic device is also provided in an embodiment of the present application. The electronic device may include: Figure 1 The processor 110 and the wireless communication module 160 are shown. The wireless communication module 160 can be used for the steps of sending a query request to the wearable device in S602 and receiving the query result sent by the wearable device in S604; the processor 110 can be used for executing S603, S604 to determine whether the user is speaking according to the information that the user is speaking and step S605.

[0197] Figure 7 A schematic block diagram of a wearable device provided in an embodiment of the present application is shown. The wearable device may include a processor 710 and a wireless communication module 720, and the wireless communication module 720 may be used to perform the steps of receiving a query request sent by an electronic device in S304 and sending a query result to the electronic device in S306. The processor 710 may be used to perform the step of determining information that the user is speaking in S305.

[0198] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0201] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0203] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0204] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice wake-up system, comprising an electronic device and a wearable device, wherein the electronic device communicates with the wearable device via a short-range wireless connection, characterized in that: The electronic device is used to collect voice signals in the environment; The electronic device is further configured to send a query request to the wearable device when the voice signal meets a preset condition, wherein the query request is used to request information about what the user is saying; The wearable device is used to send a query result to the electronic device, wherein the query result includes information that the user is speaking; The electronic device is further configured to enter a wake-up state when it is determined that the user is speaking according to the information that the user is speaking; The information that the user is speaking is used to indicate a first confidence level and a second confidence level, the first confidence level being the confidence level that the user is wearing the wearable device, and the second confidence level being the confidence level that the user is speaking; The electronic device is specifically configured to: enter a wake-up state when the first confidence level is greater than or equal to a first preset value and the second confidence level is greater than or equal to a second preset value; The wearable device includes an acoustic sensor, and the wearable device is specifically used for: Collecting the sound signal of human heartbeat or breathing through the acoustic sensor; Inputting the sound signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, wherein the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking; or The wearable device includes a photoplethysmography (PPG) sensor, and the wearable device is specifically used for: Collecting PPG signals through the PPG sensor; The PPG signal is input into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a PPG signal when the user is wearing the wearable device and is not speaking, and the third model is obtained by collecting a PPG signal when the user is wearing the wearable device and is speaking.

2. The voice wake-up system according to claim 1, characterized in that: The account logged in on the electronic device is associated with the account logged in on the wearable device.

3. An electronic device that communicates with a wearable device via a short-range wireless connection, characterized in that: The electronic device comprises: one or more processors; one or more memories; The one or more memories store one or more computer programs, and the one or more computer programs include instructions. When the instructions are executed by the one or more processors, the electronic device performs the following steps: Collect voice signals in the environment; When the voice signal meets a preset condition, sending a query request to the wearable device, wherein the query request is used to request information about what the user is saying; Receiving a query result sent by the wearable device, wherein the query result includes information that the user is speaking; When it is determined that the user is speaking according to the information that the user is speaking, entering a wake-up state; The information that the user is speaking is used to indicate a first confidence level and a second confidence level, the first confidence level is the confidence level that the user is wearing the wearable device, and the second confidence level is the confidence level that the user is speaking. When the instruction is executed by the one or more processors, the electronic device performs the following steps: When the first confidence level is greater than or equal to a first preset value and the second confidence level is greater than or equal to a second preset value, entering a wake-up state; The first confidence and the second confidence are obtained by inputting a sound signal of a human heartbeat or breathing into a first model, a second model and a third model, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking; or The first confidence and the second confidence are obtained by inputting the photoplethysmography (PPG) signal into the first model, the second model and the third model. The first model is obtained by collecting the noise signal when the user is not wearing the wearable device, the second model is obtained by collecting the PPG signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting the PPG signal when the user is wearing the wearable device and speaking.

4. The electronic device according to claim 3, characterized in that: The account logged in on the electronic device is associated with the account logged in on the wearable device.

5. A wearable device that communicates with an electronic device via a short-range wireless connection, characterized in that: The wearable device comprises: one or more processors; one or more memories; The one or more memories store one or more computer programs, and the one or more computer programs include instructions, and when the instructions are executed by the one or more processors, the wearable device performs the following steps: receiving a query request sent by the electronic device, wherein the query request is used to request information about a user speaking; Sending a query result to the electronic device, wherein the query result includes information that the user is speaking; The information that the user is speaking is used to indicate a first confidence level and a second confidence level, the first confidence level being the confidence level that the user is wearing the wearable device, and the second confidence level being the confidence level that the user is speaking; The wearable device includes an acoustic sensor. When the instructions are executed by the one or more processors, the wearable device performs the following steps: Collecting the sound signal of human heartbeat or breathing through the acoustic sensor; Inputting the sound signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, wherein the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking; or The wearable device includes a PPG sensor. When the instructions are executed by the one or more processors, the wearable device performs the following steps: Collecting PPG signals through the PPG sensor; The PPG signal is input into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a PPG signal when the user is wearing the wearable device and is not speaking, and the third model is obtained by collecting a PPG signal when the user is wearing the wearable device and is speaking.

6. The wearable device according to claim 5, characterized in that: The account logged in on the electronic device is associated with the account logged in on the wearable device.

7. A voice wake-up method, the method is applied to an electronic device, the electronic device communicates with a wearable device via a short-range wireless connection, characterized in that: The method comprises: Collect voice signals in the environment; When the voice signal meets a preset condition, sending a query request to the wearable device, wherein the query request is used to request information about what the user is saying; Receiving a query result sent by the wearable device, wherein the query result includes information that the user is speaking; When it is determined that the user is speaking according to the information that the user is speaking, entering a wake-up state; The information that the user is speaking is used to indicate a first confidence level and a second confidence level, the first confidence level is the confidence level that the user is wearing the wearable device, and the second confidence level is the confidence level that the user is speaking, and when it is determined that the user is speaking according to the information that the user is speaking, entering the awakening state includes: When the first confidence level is greater than or equal to a first preset value and the second confidence level is greater than or equal to a second preset value, entering a wake-up state; The first confidence and the second confidence are obtained by inputting a sound signal of a human heartbeat or breathing into a first model, a second model and a third model, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking; or The first confidence and the second confidence are obtained by inputting the photoplethysmography (PPG) signal into the first model, the second model and the third model. The first model is obtained by collecting the noise signal when the user is not wearing the wearable device, the second model is obtained by collecting the PPG signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting the PPG signal when the user is wearing the wearable device and speaking.

8. The method according to claim 7, characterized in that The account logged in on the electronic device is associated with the account logged in on the wearable device.

9. A voice wake-up method, the method is applied to a wearable device, the wearable device communicates with an electronic device via a short-range wireless connection, characterized in that: The method comprises: receiving a query request sent by the electronic device, wherein the query request is used to request information about a user speaking; Sending a query result to the electronic device, wherein the query result includes information that the user is speaking; The information that the user is speaking is used to indicate a first confidence level and a second confidence level, the first confidence level being the confidence level that the user is wearing the wearable device, and the second confidence level being the confidence level that the user is speaking; The wearable device includes an acoustic sensor, and before sending the query result to the electronic device, the method further includes: Collecting the sound signal of human heartbeat or breathing through the acoustic sensor; Inputting the sound signal into a first model, a second model, and a third model to obtain the first confidence and the second confidence, wherein the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a sound signal when the user is wearing the wearable device and not speaking, and the third model is obtained by collecting a sound signal when the user is wearing the wearable device and speaking; or The wearable device includes a PPG sensor. Before sending the query result to the electronic device, the method further includes: Collecting PPG signals through the PPG sensor; The PPG signal is input into a first model, a second model, and a third model to obtain the first confidence and the second confidence, the first model is obtained by collecting a noise signal when the user is not wearing the wearable device, the second model is obtained by collecting a PPG signal when the user is wearing the wearable device and is not speaking, and the third model is obtained by collecting a PPG signal when the user is wearing the wearable device and is speaking.

10. The method according to claim 9, characterized in that The account logged in on the electronic device is associated with the account logged in on the wearable device.

11. A computer-readable storage medium, characterized in that: including computer instructions, When the computer instruction is executed on an electronic device, the electronic device executes the voice wake-up method as claimed in claim 7 or 8; or, When the computer instructions are executed on the wearable device, the wearable device executes the voice wake-up method as claimed in claim 9 or 10.

12. A computer program product, characterized in that When the computer program product is run on an electronic device, the electronic device executes the voice wake-up method as claimed in claim 7 or 8; or, When the computer program product runs on a wearable device, the wearable device is enabled to perform the voice wake-up method according to any one of claims 9 or 10.

Images