Voice assistant wake-up method and apparatus

Abstract

The application discloses a voice assistant wake-up method and device, the method comprising: when a first electronic device receives voice data of a user, obtaining sensor data, determining state awareness data according to the sensor data, and then receiving state awareness data from at least one second electronic device; determining a target wake-up device according to the state awareness data of the first electronic device and the state awareness data of the at least one second electronic device, and waking up a voice assistant when the target wake-up device is the first electronic device. According to the state awareness data of multiple electronic devices, the application determines the voice assistant on the electronic device that the user needs to wake up, can make the electronic device better perceive the user's intention, so that the voice assistant on the electronic device that meets the user's needs can be woken up from multiple electronic devices, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for waking up a voice assistant. Background Technology

[0002] With the widespread application of voice recognition technology, voice assistants in mobile terminals have gradually become a frequently used function. Users can issue voice commands to the voice assistant to control it to perform various operations on the mobile terminal.

[0003] However, existing voice assistants all require user activation for voice interaction. Furthermore, the technology has evolved from single-device systems to distributed devices across multiple devices and scenarios, such as Internet of Things (IoT) devices centered around voice assistants (e.g., smartphones, smart speakers, tablets, laptops, smart TVs, smart air conditioners, etc.). When a user is in a scenario involving multiple devices, figuring out how to activate the voice assistant on the desired device based on the user's voice data is a pressing issue that needs to be addressed. Summary of the Invention

[0004] This application provides a voice assistant wake-up method and apparatus. Based on the state perception data of multiple electronic devices, the method determines the voice assistant on the electronic device that the user needs to wake up. This allows the electronic devices to better perceive the user's intent, thereby enabling the user to wake up the voice assistant on the electronic device that the user needs from among multiple electronic devices, thus improving the user experience.

[0005] In a first aspect, embodiments of this application provide a voice assistant wake-up method, applied to a first electronic device, the method comprising:

[0006] When user voice data is received, sensor data is acquired;

[0007] Determine state perception data based on the sensor data;

[0008] Receive state-aware data from at least one second electronic device;

[0009] The target wake-up device is determined based on the state awareness data of the first electronic device and the state awareness data of the at least one second electronic device;

[0010] If the target wake-up device is the first electronic device, the voice assistant is woken up.

[0011] Secondly, embodiments of this application provide a voice assistant wake-up device, applied to an electronic device, the device comprising:

[0012] The state awareness module is used to acquire sensor data when it receives voice data from the user;

[0013] The state perception module is also used to determine state perception data based on the sensor data;

[0014] A communication module for receiving status-aware data from at least one second electronic device;

[0015] The decision module is used to determine the target wake-up device based on the state perception data of the first electronic device and the state perception data of the at least one second electronic device;

[0016] The wake-up module is used to wake up the voice assistant if the target wake-up device is the first device.

[0017] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0019] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package.

[0020] As can be seen in this embodiment, when the first electronic device receives the user's voice data, it acquires sensor data, determines state perception data based on the sensor data, and then receives state perception data from at least one second electronic device. Based on the state perception data of the first electronic device and the state perception data of at least one second electronic device, it determines the target wake-up device. If the target wake-up device is the first electronic device, it wakes up the voice assistant. This application determines the voice assistant on the electronic device that the user needs to wake up based on the state perception data of multiple electronic devices, allowing the electronic devices to better perceive the user's intent. This enables the user to wake up the voice assistant on the electronic device requested by the user from among multiple electronic devices, thereby improving the user experience. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating an application scenario for waking up a voice assistant, as provided in an embodiment of this application.

[0025] Figure 4 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0026] Figure 5 This is a flowchart illustrating a voice assistant wake-up method provided in an embodiment of this application;

[0027] Figure 5a This is a schematic diagram of a multi-device scenario provided in an embodiment of this application;

[0028] Figure 5b This is a schematic diagram of a holding state provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram of the structure of a voice assistant wake-up device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0032] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0033] The electronic devices involved in the embodiments of this application may be portable electronic devices that also include other functions such as personal digital assistant and / or music player functions, such as mobile phones, tablet computers, wearable electronic devices with wireless communication capabilities (such as smartwatches), etc. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft systems, or other operating systems. The aforementioned portable electronic devices may also be other portable electronic devices, such as laptop computers. It should also be understood that in some other embodiments, the aforementioned electronic device may not be a portable electronic device, but a desktop computer.

[0034] The first part describes the software and hardware operating environment of the technical solution disclosed in this application.

[0035] For example, Figure 1A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a compass 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0036] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0037] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent components or integrated into one or more processors. In some embodiments, electronic device 100 may also include one or more processors 110. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. In other embodiments, processor 110 may also include a memory for storing instructions and data. For example, the memory in processor 110 may be a cache memory. This memory can store instructions or data that processor 110 has just used or is reusing. If processor 110 needs to reuse the instruction or data, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the electronic device 100 in processing data or executing instructions.

[0038] In some embodiments, the processor 110 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. The USB interface 130 is a USB standard-compliant interface, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transfer between the electronic device 100 and peripheral devices. The USB interface 130 can also be used to connect headphones for audio playback.

[0039] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0040] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0041] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0042] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0043] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0044] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0045] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and UWB. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0046] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0047] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (miniled), a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or more display screens 194.

[0048] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0049] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0050] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or more cameras 193.

[0051] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0052] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0053] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0054] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0055] Internal memory 121 can be used to store one or more computer programs, which include instructions. Processor 110 can execute the instructions stored in internal memory 121, thereby causing electronic device 100 to perform the methods for displaying page elements provided in some embodiments of this application, as well as various applications and data processing. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data created during the use of electronic device 100 (such as photos, contacts, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, processor 110 can execute instructions stored in internal memory 121 and / or instructions stored in memory disposed in processor 110, thereby causing electronic device 100 to perform the methods for displaying page elements provided in embodiments of this application, as well as other applications and data processing. Electronic device 100 can implement audio functions such as music playback and recording through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0056] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0057] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the touch position based on the detection signal from the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0058] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 around three axes (i.e., the X, Y, and Z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0059] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0060] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0061] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0062] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0063] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0064] For example, Figure 2 A software architecture block diagram of the electronic device 100 is shown. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0065] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0066] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0067] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0068] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0069] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0070] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0071] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0072] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0073] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0074] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0075] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0076] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0077] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0078] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0079] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0080] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0081] A 2D graphics engine is a graphics engine for 2D drawing.

[0082] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0083] The second part introduces the example application scenarios disclosed in the embodiments of this application as follows.

[0084] Please see Figure 3 The voice assistant wake-up method provided in this application embodiment can be applied to, for example... Figure 3 The application scenarios shown are as follows. Figure 3 As shown, this application scenario includes multiple electronic devices equipped with voice assistants, all sharing the same wake-up word. In this embodiment, this application scenario can be referred to as a multi-device scenario. In this multi-device scenario, after the user speaks the wake-up word, the method of this embodiment—that is, determining the scene of the electronic device based on multiple sensor data and selecting the most suitable electronic device to wake up the voice assistant based on scene priority—allows the electronic device to better perceive the user's intent, thereby providing the user with a more intelligent product experience.

[0085] In some embodiments, a voice assistant can be installed in an electronic device to enable voice control functionality. The voice assistant is typically in a dormant state. Before using the voice control function of the electronic device, the user needs to wake up the voice assistant using voice commands. The voice data used to wake up the voice assistant can be called a wake-up word (or wake-up voice). This wake-up word can be pre-registered in the electronic device. In this embodiment, waking up the voice assistant means that the electronic device activates the voice assistant in response to the user's spoken wake-up word. The voice control function means that after the voice assistant is activated, the user can trigger the electronic device to automatically execute the event corresponding to that voice command by speaking a voice command (e.g., a piece of voice data).

[0086] Furthermore, the aforementioned voice assistant can be an embedded application within an electronic device (i.e., a system application of the electronic device) or a downloadable application. An embedded application is an application provided as part of the implementation of an electronic device (such as a mobile phone). A downloadable application is an application that can provide its own Internet Protocol Multimedia Subsystem (IMS) connectivity. Downloadable applications can be pre-installed on electronic devices or are third-party applications downloaded and installed by the user.

[0087] For example, such as Figure 4 As shown, the electronic device may include a state awareness module, a communication module, a decision-making module, and a wake-up module. The wake-up module is mainly used to wake up the voice assistant; the state awareness module is mainly used to calculate the state awareness data of the electronic device; the decision-making module is responsible for deciding which electronic device's voice assistant to wake up. This module adopts a distributed decision-making method, whereby each electronic device's decision-making module, after receiving state awareness data from other electronic devices, can determine the target electronic device to wake up according to preset rules and notify the wake-up module; the communication module is responsible for sending and receiving messages between electronic devices.

[0088] Part Three, the scope of protection disclosed in the embodiments of this application is described below. Furthermore, the following embodiments are combined with... Figure 3 Taking a scenario with multiple devices, where all of these electronic devices are equipped with voice assistants and the wake word is "Xiao Bu Xiao Bu", this example will be used for illustration.

[0089] Please see Figure 5 , Figure 5 This application provides a flowchart illustrating a voice assistant wake-up method, applicable to, for example... Figure 3 For any of the electronic devices shown in the figure, the voice assistant wake-up method includes the following operations.

[0090] S510: When user voice data is received, acquire sensor data.

[0091] In this embodiment, for a first electronic device equipped with a voice assistant, when no other software or hardware is using the microphone to collect voice data, the DSP of the first electronic device can monitor in real time whether the user is inputting voice data via the microphone. Generally, when a user wants to use the voice control function of the electronic device, they can speak within the pickup range of the electronic device to input the sound into the microphone. At this time, if no other software or hardware is using the microphone to collect voice data, the DSP of the first electronic device can monitor the corresponding voice data via the microphone and cache it.

[0092] For example, Figure 5a As shown, when a user is sitting on the sofa in the living room and wants to use the voice control function to play music, they can say the wake word "Xiao Bu Xiao Bu". If the phone, tablet, and laptop are all around the user, meaning the user is within their voice pickup range, and no other software or hardware is using the microphone to collect voice data, then the DSP of the phone, tablet, and smartwatch can detect the voice data corresponding to the wake word "Xiao Bu Xiao Bu" through their respective microphones.

[0093] For example, after receiving the aforementioned voice data, the first electronic device can verify the voice data, that is, determine whether the received voice data is a wake-up word registered in the first electronic device. If the verification passes, it indicates that the received voice data is a wake-up word, and sensor data can be acquired. If the verification fails, it indicates that the received voice data is not a wake-up word, and the first electronic device can delete the cached voice data.

[0094] Specifically, when the voice data received by the first electronic device is a wake-up word, the first electronic device can activate the decision-making system service, which is integrated into the operating system as a service. When the operating system detects the wake-up word, the decision-making system service begins registering initialization tasks such as sensor monitoring. That is, the operating system starts the sensors to monitor the state of the first electronic device and transmits the sensor data back through callback functions. Only after all sensor data has been transmitted back is the state awareness module notified to read the sensor data and calculate the state of the first electronic device.

[0095] In this embodiment, transmitting sensor data via a callback function is an asynchronous operation that can improve system efficiency.

[0096] S520. Determine the state perception data based on the sensor data.

[0097] The sensor data includes at least one of the following: acceleration sensor data, angular velocity sensor data, Z-axis acceleration data, distance sensor data, and light sensor data; the state perception data includes: first state data, second state data, third state data, and device identifier.

[0098] For example, the first state data can be the data within the value range of the first electronic device being held or laid flat, the second state data can be the value within the value range of the first electronic device being inverted or not inverted, and the third state data can be the value within the value range of the first electronic device being in a pocket, covered, daytime, or nighttime state.

[0099] Optionally, determining the state perception data based on the sensor data includes: determining the first state data based on the acceleration sensor data and the angular velocity sensor data; determining the second state data based on the Z-axis acceleration data; and determining the third state data based on the distance sensor data and the light sensor data.

[0100] The state perception module can use accelerometers and angular velocity sensors to determine whether the electronic device is in the "holding" or "flat" (non-holding) state and determine whether the electronic device is in the "inverted", "pocket", "obstructed", "night" and "day" states by using accelerometers, proximity sensors and light sensors.

[0101] Optionally, determining the first state data based on the acceleration sensor data and the angular velocity sensor data includes:

[0102] A first included angle is calculated based on the value of the acceleration sensor data. The first included angle is the angle between the first electronic device and the horizontal plane. If the first included angle is greater than or equal to a preset angle, the value of the first state data is set to a first value, which is a value within the range of values ​​to which the first state belongs. If the first included angle is less than the preset angle, it is determined whether the first electronic device is shaking based on the value of the angular velocity sensor data. If the first electronic device is shaking, the value of the first state data is set to the first value. Otherwise, the value of the first state data is set to a second value, which is a value outside the range of values ​​to which the first state belongs.

[0103] In this context, the first state can be a holding state, and non-first states can be flat states; the second state can be an inverted state, and non-second states can be non-inverted states; the third state is a pocket state. The first value can be any value within the range of values ​​belonging to the first state, and the second value can be any value outside the range of values ​​belonging to the first state. For example, a mapping relationship can be constructed between the first included angle and values ​​within the range of values ​​belonging to the first state, and between the first included angle and values ​​outside the range of values ​​belonging to the first state, so that the first or second value can be determined based on the mapping relationship. It is understood that the range of values ​​belonging to the first state and the range of values ​​belonging to the non-first state do not overlap.

[0104] In practical applications, whether the first electronic device is currently being used by a user can be determined by whether it is being held. By using accelerometer data to calculate the first angle, the first state data value can be set to a first value whenever the angle between the first electronic device and the horizontal plane exceeds a preset angle, whether the device is in landscape or portrait mode. This allows the decision module to determine that the first electronic device is being held.

[0105] For example, such as Figure 5b As shown, the holding state includes, but is not limited to, the following scenarios, such as... Figure 5b In each of the scenarios shown, the first electronic device detects that the device is being held.

[0106] For example, the formula for calculating the angle between the mobile phone and the horizontal plane using accelerometer data can be expressed as:

[0107]

[0108] Wherein, acc_x and acc_y are the gravitational acceleration components along the x-axis and y-axis, respectively; g is the gravitational acceleration, with a value of 9.81.

[0109] For example, the preset angle can be configured by the system, such as 15 degrees; it can also be set according to the user's usage habits. Of course, this application does not limit other ways of setting the preset angle.

[0110] Furthermore, in some scenarios, when the first angle is less than the preset angle, the first electronic device may still be in a holding state, for example, when the user uses the first electronic device in the palm of their hand. Therefore, when the first angle is less than the preset angle, the first electronic device can initiate shake detection. Through shake detection, the first electronic device in a holding state but with the first angle less than the preset angle can be detected.

[0111] Optionally, determining whether the first electronic device is shaking based on the value of the angular velocity sensor data includes:

[0112] A first angular velocity is calculated based on the value of the angular velocity sensor data. The first angular velocity is the angular velocity between the first electronic device and the horizontal plane. If the first angular velocity is less than a first preset angular velocity but greater than a second preset angular velocity, the first angular velocity is stored in a cache; otherwise, the cache is cleared. When the number of first angular velocities in the cache is greater than or equal to the statistical window, the average value of the first angular velocities within the statistical window during the sampling period is calculated. If the average value is greater than or equal to a third preset angular velocity, or the first angular velocity is greater than the first preset angular velocity, it is determined that the first electronic device is jittering; otherwise, it is determined that the first electronic device is stationary.

[0113] For example, the first angular velocity is the magnitude of the angular velocity vector of the first electronic device, and the formula for calculating the magnitude of the angular velocity vector can be expressed as:

[0114]

[0115] Wherein, axisX, axisY, and axisZ are the angular velocities of the angular velocity sensor along the x-axis, y-axis, and z-axis, respectively.

[0116] For example, the first preset angular velocity, the second preset angular velocity, the third preset angular velocity, the sampling period, and the statistical window can be set by the system. For example, the first preset angular velocity can be set to 0.030 rad / s, the second preset angular velocity to 0.002 rad / s, the third preset angular velocity to 0.010 rad / s, the sampling period to 4, and the statistical window to 20. Alternatively, the settings can be made according to the specific application scenario. This application embodiment does not limit the settings in this regard.

[0117] In practical applications, angular velocity sensors (such as gyroscopes) report angular velocity data in a very short period (2.5ms per packet). However, fluctuations in the angular velocity data measured by the gyroscope may continue for a period of time. Therefore, this fluctuation is filtered out by increasing the statistical window and using interval sampling.

[0118] Specifically, when the state perception module receives the instantaneous angular velocity value reported by the gyroscope, it calculates the magnitude of the reported angular velocity vector. If the magnitude of the angular velocity vector is greater than the first preset angular velocity v1 or less than the second preset angular velocity v2, the state perception module clears the magnitude of the angular velocity vector stored in the memory buffer. If the magnitude of the angular velocity vector is greater than v1, the first electronic device is determined to be jittering. If the magnitude of the angular velocity vector is less than v2, the first electronic device is determined to be stationary. If the magnitude of the angular velocity vector is greater than the first preset angular velocity v1 and less than the second preset angular velocity v2, the magnitude of the angular velocity vector is input into a buffer for caching. If the number of magnitudes of angular velocity vectors stored in the buffer is less than the statistical window Wstatistics_window, the state awareness module performs the above calculation on the next received instantaneous angular velocity value. If the number of magnitudes of angular velocity vectors stored in the buffer is greater than or equal to Wstatistics_window, the angular velocities in the statistical window are sampled at intervals using a sampling period Isamping_interval. The number of acceleration samples is Wstatistics_window / Isamping_interval. Then, the average angular velocity value Avg of these Wstatistics_window / Isamping_interval angular velocities is calculated to reduce fluctuations in the angular velocity data. If the average angular velocity value Avg is less than the third preset angular velocity, the first electronic device is determined to be stationary; if the average angular velocity value Avg is less than or equal to the third preset angular velocity, the first electronic device is determined to be jittering.

[0119] Optionally, determining the second state data based on the Z-axis acceleration data includes: if the value of the Z-axis acceleration data is less than an acceleration threshold, then setting the value of the second state data to a third value, wherein the third value is a value within the range to which the second state belongs; otherwise, setting the value of the second state data to a fourth value, wherein the fourth value is a value outside the range to which the second state belongs.

[0120] The state perception module sets second state data based on the acceleration along the z-axis reported by the accelerometer to detect whether the first electronic device is in an inverted state. Specifically, if the z-axis acceleration data value is less than an acceleration threshold, the second state data value is set to a third value, allowing the decision module to determine that the first electronic device is in an inverted state; if the z-axis acceleration data value is greater than or equal to the acceleration threshold, the second state data value is set to a fourth value, allowing the decision module to determine that the first electronic device is in a non-inverted state.

[0121] For example, the aforementioned acceleration threshold can be set to -9 m / s². 2When an electronic device is inverted, the gravitational acceleration along the Z-axis is opposite to the direction of gravitational acceleration, and the acceleration along the Z-axis will be less than the acceleration threshold.

[0122] It should be noted that the aforementioned third value can be any value within the range of values ​​described in the second state. For example, the value of the Z-axis acceleration data can also be mapped to a value within the range of values ​​belonging to the second state. For instance, if the range of values ​​for the second state is [-1, -2], and the Z-axis acceleration is -9.1 m / s², then... 2 --9.4m / s 2 When the Z-axis acceleration is within the specified range, the second state data can be set to -1; when the Z-axis acceleration is -9.5 m / s² 2 --9.8m / s 2 When the value is within the specified range, the second state data can be set to -2. The fourth value can be any value outside the range of the second state. For example, the value of the Z-axis acceleration data can also be mapped to values ​​outside the range of the second state. For instance, if the range outside the second state is [1,4], and the Z-axis acceleration is -9.0 m / s², then... 2 --5.0m / s 2 When the Z-axis acceleration is within the specified range, the second state data can be set to 1; when the Z-axis acceleration is -4.9 m / s² 2 --1.0m / s 2 When the Z-axis acceleration is within the specified range, the second state data can be set to 2; when the Z-axis acceleration is -0.9 m / s² 2 -4.0m / s 2 When the Z-axis acceleration is within the specified range, the second state data can be set to 3; when the Z-axis acceleration is 4.1 m / s² 2 -9.8m / s 2 When the value is within the specified range, the second state data can be set to 4. Of course, this application embodiment does not limit other methods of mapping the third value to values ​​within the range of the second state, or the fourth value to values ​​outside the range of the second state.

[0123] Optionally, determining the third state data based on the distance sensor data and the light sensor data includes: if the value of the distance sensor data is less than the occlusion distance threshold and the value of the light sensor data is less than the light intensity threshold, then the value of the third state data is set to a fifth value, where the fifth value is a value within the range to which the third state belongs; otherwise, the value of the third state data is set to a sixth value, where the sixth value is a value outside the range to which the third state belongs.

[0124] The third state is the pocket state. When the first electronic device is in the pocket state, it indicates that the user's willingness to use the first electronic device is very low, meaning the probability of the user waking up the voice assistant on the first electronic device is very small. Since the light in the pocket scene is dim and there is obstruction, the values ​​of the third state data can be set using a distance sensor and a light sensor, allowing the decision module to detect whether the first electronic device is in the pocket state. Specifically, when the data value reported by the distance sensor is less than or equal to the obstruction distance threshold and the data value reported by the light sensor is less than or equal to the light intensity threshold, the value of the third state data is set to the fifth value. The decision module can determine that the first electronic device is in the pocket state based on the fifth value. Otherwise, the value of the third state data is set to the sixth value, and the decision module determines that the first electronic device is not in the pocket state based on the sixth value.

[0125] For example, the occlusion distance threshold can be set to 0, 0.1cm, 0.2cm, etc., and the light intensity threshold can be set to 10 lux.

[0126] For example, the occlusion distance threshold and the light intensity threshold can also be set according to the user's habits or according to the specific actual scenario. This application embodiment does not limit this.

[0127] Optionally, the non-third state includes the fourth, fifth, and sixth states.

[0128] For example, the fourth state can be an occlusion state, the fifth state can be a night state, and the sixth state can be a day state.

[0129] Optionally, setting the value of the third state data to a sixth value includes: if the value of the distance sensor data is less than the occlusion distance threshold and the value of the light sensor data is greater than or equal to the light intensity threshold, then setting the value of the third state data to a seventh value, wherein the seventh value is a value within the value range of the fourth state; if the value of the distance sensor data is greater than or equal to the occlusion distance threshold and the value of the light sensor data is less than the light intensity threshold, then setting the value of the third state data to an eighth value, wherein the eighth value is a value within the value range of the fifth state; if the value of the distance sensor data is greater than or equal to the occlusion distance threshold and the value of the light sensor data is greater than or equal to the light intensity threshold, then setting the value of the third state data to a ninth value, wherein the ninth value is a value within the value range of the sixth state.

[0130] Among them, the numerical range of the non-third state can be the union of the numerical range of the fourth state, the numerical range of the fifth state, and the numerical range of the sixth state. That is, the sixth value can be any value in the numerical range of the fourth state, the numerical range of the fifth state, and the numerical range of the sixth state.

[0131] In practical applications, distance sensors can detect the distance between electronic devices and target objects. Therefore, the data from distance sensors can also be used to detect whether the electronic device is not in a pocket but is obstructed by an object, such as when a phone is held to the ear (for example, when answering a call). Light sensors can detect the ambient light level of the electronic device. Therefore, the data from light sensors can be used to determine whether the electronic device is in daylight or darkness.

[0132] Optionally, the method further includes: sending state-aware data of the first electronic device to the at least one second electronic device.

[0133] When the state perception module in the first electronic device determines the state perception data based on the sensor data, it can send the state perception data to the surrounding second electronic device, so that the second electronic device can make a decision based on the state perception data of the surrounding multiple devices to select the target wake-up device.

[0134] For example, the first electronic device may set a timer in the communication module. When the voice data received by the first electronic device is a wake word, the timer is started, and then the state perception data calculated by the state perception module is broadcast within the timer.

[0135] In this embodiment, the electronic device broadcasts state-aware data to synchronize the states of multiple devices, reducing the time overhead associated with establishing connections. Simultaneously, real-time processing of sensor data eliminates the need for multiple data collections and calculations, further reducing data collection time. This provides users with new functionality while effectively controlling the wake-up time of the voice assistant.

[0136] S530, Receive state-aware data from at least one second electronic device.

[0137] In this embodiment, when the voice data received by the first electronic device is a wake-up word, the first electronic device can use the communication module to listen to the status-aware data sent by the surrounding second electronic devices. For example, the first electronic device can set another timer within the communication module. When the voice data received by the first electronic device is a wake-up word, the timer is started, and then the status-aware data sent by the first electronic device is received within the timer. For example, the first electronic device can set a timer within the communication module to start broadcasting and receiving status-aware data, and the timer automatically stops broadcasting and receiving status-aware data when it expires.

[0138] For example, the timer can be set by the system, such as a 300ms timeout; it can also be set according to the specific actual scenario, and this application embodiment does not limit this.

[0139] In this embodiment, by using broadcasting and receiving to synchronize the status awareness data of multiple devices, and by using real-time sensor data processing, decision latency can be effectively reduced and user experience improved.

[0140] For example, the communication module can encapsulate various near-field communication methods, such as Bluetooth (BT), Bluetooth Low Energy (BLE), Wi-Fi Direct, and Wireless Local Area Network (WLAN). When calling the communication module, the specific communication method can be specified through parameters or a configuration file. Taking BLE as an example, a BLE broadcast packet transmits a series of data including a Universally Unique Identifier (UUID), MAC address, and Bluetooth name. However, there is still some remaining space available for transmitting custom data. Calculations show that each broadcast packet can hold a maximum of 23 bytes of remaining space. By placing the state-aware data calculated by the state-aware module into this remaining space and sending it to the second electronic device, the BLE GATT connection establishment time can be saved, thereby reducing the latency required for wake-up.

[0141] In this embodiment of the application, if a connection has been established between the first electronic device and the second electronic device, the first electronic device and / or the second electronic device can reuse the established connection to directly send and receive data packets. This makes the communication method for the established connection more reliable and secure, and also has lower latency.

[0142] It should be noted that the execution order of S520 and S530 is not limited in the embodiments of this application. That is, the first electronic device can execute S520 and S530 at the same time; it can also execute S530 first and then S520; or it can execute S520 first and then S530.

[0143] S540. Determine the target wake-up device based on the state perception data of the first electronic device and the state perception data of the at least one second electronic device.

[0144] In this embodiment, after the communication module receives the status awareness data sent by the second electronic device, it can send the status awareness data of the second electronic device and the status awareness data of the first electronic device together to the decision module. The decision module determines the target wake-up device based on the status awareness data of multiple devices.

[0145] Optionally, determining the target wake-up device based on the state-aware data of the first electronic device and the state-aware data of the at least one second electronic device includes:

[0146] Based on the second state data, the number of first candidate devices is determined, where the first candidate devices are the first electronic device and / or the second electronic device in the second state; if the number of first candidate devices is 1, then the first candidate device is determined as the target wake-up device; if the number of first candidate devices is greater than 1, then based on the third state data, the number of second candidate devices is determined from the first candidate devices, where the second candidate devices are the first candidate devices in the third state; if the number of second candidate devices is 1, then the second candidate device is determined as the target wake-up device; if the number of second candidate devices is greater than 1, then based on the first state data, the number of third candidate devices is determined from the second candidate devices, where the third candidate devices are the second candidate devices in the first state; if the number of third candidate devices is 1, then the third candidate device is determined as the target wake-up device; if the number of second candidate devices is greater than 1, then the target wake-up device is determined from the third candidate devices based on the device identifier.

[0147] Specifically, the data structure of state-aware data can be represented as {isHoldInHand; isUpsideDown; isInPocket; deviceId}. Here, isHoldInHand is the value of the first state data, isUpsideDown is the value of the second state data, isInPocket is the value of the third state data, and deviceId is the device ID.

[0148] In this embodiment, a chain decision-making method is adopted, that is, selection is performed sequentially based on state-aware data. When a unique target wake-up device is selected, the process immediately exits and does not continue to pass on. If the process is passed to the last rule and no target wake-up device is selected, the target wake-up device is selected by device ID.

[0149] Specifically, firstly, the decision module selects a first and / or second electronic device (i.e., an electronic device in a non-inverted state) from multiple devices based on the second state data in the state-aware data. That is, if the value of the second state data of an electronic device is within the range of values ​​described for the second state, then the electronic device is determined to be in the second state. If only one electronic device is in a non-second state, then that electronic device is identified as the target wake-up device. If multiple electronic devices are in a non-second state, then electronic devices in a non-third state (electronic devices in the fourth, fifth, and sixth states) are selected from among these non-second state electronic devices. Specifically, if the value of the third state data of an electronic device is within the range of values ​​described for a non-third state, then the electronic device is determined to be in a non-third state. If no electronic device is in a non-second state (all are in the inverted state), then an electronic device in a non-third state is selected from among the multiple devices. If only one electronic device is not in the third state, that electronic device is directly identified as the target wake-up device. If multiple electronic devices are not in the third state, the electronic device in the first state is selected from among them. Specifically, if the value of the first state data of the electronic device is within the range of the first state, then the electronic device is determined to be in the first state. If no electronic device is not in the third state (all are in the pocket state), then the electronic device in the first state is selected from among the multiple devices or multiple electronic devices not in the second state. If only one electronic device is in the first state, that electronic device is directly identified as the target wake-up device. If multiple or no electronic devices are in the first state, the target wake-up device is selected based on the device ID.

[0150] Among them, the methods of selecting a unique device by device ID include, but are not limited to, selecting the smallest or largest device ID, or randomly selecting a device ID by a random algorithm.

[0151] In this embodiment, the electronic device determines the various scenarios it is in based on data detected by multiple sensors, and selects the device that responds most appropriately based on scenario priority. This allows the electronic device to better perceive the user's intent, thereby enabling the user to experience a more intelligent product experience.

[0152] S550. If the target wake-up device is the first electronic device, wake up the voice assistant.

[0153] The decision module selects the target wake-up device and passes the acquisition perception data structure to the wake-up module. The wake-up module uses the deviceId to determine whether it is itself. If not, it does not activate the voice assistant; otherwise, it activates the voice assistant to provide a voice response.

[0154] As can be seen, the voice assistant wake-up method proposed in this application involves acquiring sensor data when the first electronic device receives the user's voice data, determining state perception data based on the sensor data, and then receiving state perception data from at least one second electronic device. Based on the state perception data of the first electronic device and the state perception data of the at least one second electronic device, a target wake-up device is determined. If the target wake-up device is the first electronic device, the voice assistant is woken up. This application determines the voice assistant on the electronic device that the user needs to wake up based on the state perception data of multiple electronic devices, allowing the electronic devices to better perceive the user's intent. This enables the user to wake up the voice assistant on the desired electronic device from among multiple electronic devices, thus improving the user experience.

[0155] It is understood that, in order to achieve the above-mentioned functions, electronic devices include hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0156] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0157] When dividing each function into modules according to its corresponding function. Figure 6 A schematic diagram of the voice assistant wake-up device is shown, such as... Figure 6 As shown, the voice assistant wake-up device 600 is applied to an electronic device. The voice assistant wake-up device 600 may include: a state perception module 601, a communication module 602, a decision module 603, and a wake-up module 604.

[0158] The state awareness module 601 can be used to support electronic devices in executing S510, S520, etc., and / or other processes used in the technology described herein.

[0159] The communication module 602 can be used to support electronic devices in performing the above-described S530, etc., and / or other processes used in the techniques described herein.

[0160] The decision module 603 can be used to support electronic devices in performing the above-described S540, and / or other processes used in the techniques described herein.

[0161] The wake-up module 604 can be used to support electronic devices in performing the above-described S550, and / or other processes used in the techniques described herein.

[0162] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0163] The electronic device provided in this embodiment is used to execute the above-described voice assistant wake-up method, and therefore can achieve the same effect as the above-described implementation method.

[0164] When using integrated units, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device; for example, it can support the electronic device in executing the steps performed by the state sensing module 601, communication module 602, decision-making module 603, and wake-up module 604. The storage module can support the electronic device in executing stored program code and data. The communication module can support communication between the electronic device and other devices.

[0165] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0166] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment can be a device having... Figure 1 The device with the structure shown.

[0167] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the voice assistant wake-up method in the above embodiment.

[0168] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the voice assistant wake-up method in the above embodiment.

[0169] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the voice assistant wake-up method in the above method embodiments.

[0170] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0171] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0173] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0175] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for waking up a voice assistant, characterized in that, Applied to a first electronic device, the method includes: When user voice data is received, sensor data is acquired, including accelerometer data, angular velocity sensor data, Z-axis acceleration data, distance sensor data, and light sensor data. Determining state perception data based on the sensor data includes: determining first state data based on the acceleration sensor data and the angular velocity sensor data; determining second state data based on the Z-axis acceleration data; and determining third state data based on the distance sensor data and the light sensor data. The state perception data includes the first state data determined based on the acceleration sensor data and the angular velocity sensor data, the second state data determined based on the Z-axis acceleration data, and the third state data determined based on the distance sensor data and the light sensor data. Receive state-aware data from at least one second electronic device; The target wake-up device is determined based on the state awareness data of the first electronic device and the state awareness data of the at least one second electronic device; If the target wake-up device is the first electronic device, wake up the voice assistant; The state-aware data further includes a device identifier, and determining the target wake-up device based on the state-aware data of the first electronic device and the state-aware data of the at least one second electronic device includes: Based on the second state data, the number of first candidate devices is determined, wherein the first candidate devices are the first electronic device and / or the second electronic device that are in a non-second state; If the number of the first candidate devices is 1, then the first candidate device is determined as the target wake-up device; If the number of the first candidate devices is greater than 1, then the number of the second candidate devices is determined from the first candidate devices according to the third state data, wherein the second candidate devices are the first candidate devices that are not in the third state; If the number of the second candidate devices is 1, then the second candidate device is determined as the target wake-up device. If the number of the second candidate devices is greater than 1, then the number of the third candidate devices is determined from the second candidate devices according to the first state data. The third candidate device is the second candidate device in the first state. If the number of the third candidate devices is 1, then the third candidate device is determined as the target wake-up device; if the number of the third candidate devices is greater than 1, then the target wake-up device is determined from the third candidate devices according to the device identifier. The first state is the holding state, the second state is the inverted state, and the third state is the pocket state.

2. The method according to claim 1, characterized in that, Determining the first state data based on the acceleration sensor data and the angular velocity sensor data includes: The first included angle is calculated based on the value of the acceleration sensor data, and the first included angle is the angle between the first electronic device and the horizontal plane; If the first included angle is greater than or equal to the preset included angle, the value of the first state data is set to a first value, where the first value is a value within the range of values ​​to which the first state belongs; If the first included angle is smaller than the preset included angle, determine whether the first electronic device is shaking based on the value of the angular velocity sensor data; If the first electronic device vibrates, the value of the first state data is set to the first value; otherwise, the value of the first state data is set to the second value, which is a value outside the range of the first state.

3. The method according to claim 2, characterized in that, The step of determining whether the first electronic device is shaking based on the value of the angular velocity sensor data includes: The first angular velocity is calculated based on the value of the angular velocity sensor data, and the first angular velocity is the angular velocity of the first electronic device relative to the horizontal plane; If the first angular velocity is less than the first preset angular velocity but greater than the second preset angular velocity, the first angular velocity is stored in the cache; otherwise, the cache is cleared. When the number of first angular velocities in the buffer is greater than or equal to the statistical window, the average value of the first angular velocities within the statistical window during the sampling period is calculated. If the average value is greater than or equal to the third preset angular velocity, or the first angular velocity is greater than the first preset angular velocity, the first electronic device is determined to be shaking; otherwise, the first electronic device is determined to be stationary.

4. The method according to claim 1, characterized in that, Determining the second state data based on the Z-axis acceleration data includes: If the value of the Z-axis acceleration data is less than the acceleration threshold, the value of the second state data is set to a third value, which is a value within the range of values ​​to which the second state belongs; otherwise, the value of the second state data is set to a fourth value, which is a value outside the range of values ​​to which the second state belongs.

5. The method according to claim 1, characterized in that, Determining the third state data based on the distance sensor data and the light sensor data includes: If the value of the distance sensor data is less than the occlusion distance threshold and the value of the light sensor data is less than the light intensity threshold, then the value of the third state data is set to the fifth value, which is a value within the value range of the third state; otherwise, the value of the third state data is set to the sixth value, which is a value outside the value range of the third state.

6. The method according to claim 5, characterized in that, The non-third state includes the fourth state, the fifth state, and the sixth state; Setting the value of the third state data to the sixth value includes: If the value of the distance sensor data is less than the occlusion distance threshold, and the value of the light sensor data is greater than or equal to the light intensity threshold, then the value of the third state data is set to the seventh value, which is a value within the range of the fourth state. If the value of the distance sensor data is greater than or equal to the occlusion distance threshold, and the value of the light sensor data is less than the light intensity threshold, then the value of the third state data is set to the eighth value, which is a value within the range of the fifth state. If the value of the distance sensor data is greater than or equal to the occlusion distance threshold, and the value of the light sensor data is greater than or equal to the light intensity threshold, then the value of the third state data is set to the ninth value, which is a value within the range of the sixth state.

7. The method according to claim 1, characterized in that, The method further includes: The state awareness data of the first electronic device is sent to the at least one second electronic device.

8. A voice assistant wake-up device, characterized in that, Applied to a first electronic device, the device includes: The state perception module is used to acquire sensor data when it receives user voice data, wherein the sensor data includes accelerometer data, angular velocity sensor data, Z-axis acceleration data, distance sensor data and light sensor data; The state perception module is further configured to determine state perception data based on the sensor data, wherein the state perception module determines first state data based on the acceleration sensor data and the angular velocity sensor data, and determines second state data based on the Z-axis acceleration data; and determines third state data based on the distance sensor data and the light sensor data; wherein the state perception data includes the first state data determined based on the acceleration sensor data and the angular velocity sensor data, the second state data determined based on the Z-axis acceleration data, and the third state data determined based on the distance sensor data and the light sensor data; A communication module for receiving status-aware data from at least one second electronic device; The decision module is used to determine the target wake-up device based on the state perception data of the first electronic device and the state perception data of the at least one second electronic device; The wake-up module is used to wake up the voice assistant if the target wake-up device is the first electronic device; The status awareness data further includes a device identifier, and the decision module is specifically used for: Based on the second state data, the number of first candidate devices is determined, wherein the first candidate devices are the first electronic device and / or the second electronic device that are in a non-second state; If the number of the first candidate devices is 1, then the first candidate device is determined as the target wake-up device; If the number of the first candidate devices is greater than 1, then the number of the second candidate devices is determined from the first candidate devices according to the third state data, wherein the second candidate devices are the first candidate devices that are not in the third state; If the number of the second candidate devices is 1, then the second candidate device is determined as the target wake-up device. If the number of the second candidate devices is greater than 1, then the number of the third candidate devices is determined from the second candidate devices according to the first state data. The third candidate device is the second candidate device in the first state. If the number of the third candidate devices is 1, then the third candidate device is determined as the target wake-up device; if the number of the third candidate devices is greater than 1, then the target wake-up device is determined from the third candidate devices according to the device identifier. The first state is the holding state, the second state is the inverted state, and the third state is the pocket state.

9. An electronic device, characterized in that, The method includes a processor, a memory, a communication interface, and one or more programs, said one or more programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange, wherein the computer program causes a computer to perform the method as described in any one of claims 1-7.

Images