Voice wake-up control method, device, equipment and medium based on multi-device collaboration

Abstract

The present invention relates to a voice wake-up control method, apparatus, device, and medium based on multi-device collaboration, and relates to the field of intelligent device interaction technology. The method comprises: obtaining wake-up requests sent by multiple first intelligent devices, wherein the wake-up requests include audio energy values ​​collected by the first intelligent devices and corresponding wake-up modes, and the wake-up modes include normal wake-up and abnormal wake-up; waking up the first intelligent device with the largest audio energy value as the target device, and obtaining environmental information of at least one second intelligent device, where the second intelligent device is an intelligent device that has not triggered the wake-up request or has not been awakened; if the wake-up mode of the target device is abnormal wake-up, waking up at least one second intelligent device that meets preset conditions based on the environmental information of the second intelligent device. The method implements intelligent decision-making for abnormal wake-up, dynamic call channels, and enhanced accompanying mode, thereby achieving the effects of reducing false wake-up rates, seamless cross-device interaction, and rapid response to emergency scenarios.

Description

Technical Field

[0001] The present invention relates to the field of intelligent device interaction technology, and in particular to a voice wake-up control method, device, equipment and medium based on multi-device collaboration. Background Art

[0002] Existing smart devices (such as smart speakers and cameras) usually respond to wake-up commands independently, which has the following defects:

[0003] False wakeup interference: Environmental noise (such as dropped objects or pet activity) can easily trigger the device to wake up abnormally, but it cannot distinguish whether manual intervention is required;

[0004] Scenario fragmentation: Users need to manually switch devices when moving, and cannot dynamically establish cross-device interaction channels;

[0005] Insufficient monitoring of special groups: Abnormal behaviors (such as falls and sudden illnesses) of groups such as elderly people living alone are difficult to respond to in a timely manner through multiple devices. Summary of the Invention

[0006] The present invention provides a voice wake-up control method based on multi-device collaboration. Through the multi-device collaborative wake-up mechanism, combined with audio energy analysis, environmental information fusion and dynamic device switching, it realizes the functions of abnormal wake-up intelligent decision-making, dynamic call channels and enhanced accompanying mode, thereby achieving the effects of reducing the false wake-up rate, seamless interaction across devices, and rapid response to emergency scenarios.

[0007] In a first aspect, the present invention provides a voice wake-up control method based on multi-device collaboration, the method comprising:

[0008] Obtaining wake-up requests sent by multiple first smart devices, wherein the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, and the wake-up modes include normal wake-up and abnormal wake-up;

[0009] Waking up the first smart device with the largest audio energy value as the target device, and obtaining environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered a wake-up request or has not been awakened;

[0010] If the wake-up mode of the target device is abnormal wake-up, at least one second smart device that meets a preset condition is woken up according to the environment information of the second smart device.

[0011] In a second aspect, the present invention provides a voice wake-up control device based on multi-device collaboration, comprising:

[0012] an acquiring unit, configured to acquire wake-up requests sent by a plurality of first smart devices, wherein the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, wherein the wake-up modes include normal wake-up and abnormal wake-up;

[0013] A first awakening unit is configured to awaken a first smart device with a maximum audio energy value as a target device, and obtain environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered an awakening request or has not been awakened;

[0014] The second awakening unit is configured to awaken at least one second smart device that meets a preset condition according to the environment information of the second smart device if the awakening mode of the target device is abnormal awakening.

[0015] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0016] Memory for storing computer programs;

[0017] The processor is used to implement the steps of the voice wake-up control method based on multi-device collaboration described in any embodiment of the first aspect when executing the program stored in the memory.

[0018] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the voice wake-up control method based on multi-device collaboration as described in any embodiment of the first aspect are implemented.

[0019] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0020] The necessity of abnormal wake-up is verified through environmental information; dynamic switching logic improves user experience; audio enhancement and alarm linkage in escort mode ensure user safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1A flowchart of a multi-device collaboration-based voice wake-up control method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic structural diagram of a multi-device collaboration-based voice wake-up control device provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a process for normalizing voice collection energy of multiple electronic devices provided by an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of a process for dynamically adjusting parameters between electronic devices provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0029] The multiple smart devices of the present invention interact with cloud servers or other devices through a network, and the voice wake-up control method based on multi-device collaboration is applied to the server side. In various embodiments, the aforementioned network can be a wired network or a wireless network, and can have many different configurations. The aforementioned network can include various interconnected data paths such as a local area network (LAN), a wide area network (WAN) (for example, the Internet). The network can also be coupled to or include part of a telecommunications network for sending data in a variety of different communication protocols. In some embodiments, the network can also include a Bluetooth communication network, Wi-Fi or a cellular communication network for sending and receiving data.

[0030] Example

[0031] Figure 1 The embodiment of the present invention provides a flow chart of a voice wake-up control method based on multi-device collaboration. The embodiment of the present invention provides a voice wake-up control method based on multi-device collaboration. Specifically, see Figure 1 The voice wake-up control method based on multi-device collaboration includes the following steps S101-S103.

[0032] S101: Acquire wake-up requests sent by multiple first smart devices, where the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, and the wake-up modes include normal wake-up and abnormal wake-up.

[0033] In this embodiment, the first smart device refers to a smart device that collects the audio of the wake-up word uttered by the user.

[0034] Specifically, the smart device includes a microphone that collects ambient audio. When the user issues a wake-up word, the smart device preprocesses the collected wake-up word audio and calculates the energy value of the wake-up word audio, i.e., the audio energy value. The wake-up mode is determined based on the semantics and pitch of the wake-up word. The distance between the smart device and the sound source determines the size of the audio energy value. Generally, the farther the smart device is from the sound source, the smaller the audio energy value, and the closer the distance, the larger the audio energy value.

[0035] After collecting the audio signal, the smart device performs signal preprocessing. In this embodiment, the preprocessing includes the following steps S1110-S1120:

[0036] S1110, Multi-channel Adaptive Echo Cancellation (MAEC): uses an improved NLMS algorithm and adds an echo path sharing mechanism between devices.

[0037] S1120, dual-mode noise reduction.

[0038] Preferably, the above step S1120 includes steps S1121-S1123:

[0039] S1121, frequency domain Wiener filtering (traditional signal processing);

[0040] S1122, lightweight CNN noise suppression (deep learning model <5MB);

[0041] S1123, Dynamic Weight Fusion: Automatically adjusts the mixing ratio of the two algorithms according to the signal-to-noise ratio.

[0042] In one embodiment, since energy values ​​cannot be directly compared due to differences in microphone sensitivity and frequency response curves between different devices, the smart device is provided with a hardware difference compensation algorithm. Specifically, the hardware difference compensation algorithm includes the following steps S1210-S1230:

[0043] S1210, Pre-calibration stage: The response curve of the 100Hz-8kHz sweep frequency signal is recorded when the device leaves the factory.

[0044] S1220, online calibration stage.

[0045] Preferably, the above step S1220 includes steps S1221-S1223:

[0046] S1221, shares the ambient noise floor spectrum (20-200Hz) for low-frequency gain compensation;

[0047] S1222, high-frequency phase alignment using ultrasonic handshake signals (18kHz) between devices;

[0048] S1223, dynamically swapping 1kHz reference tones to achieve real-time gain synchronization.

[0049] S1230 , generating a device fingerprint matrix: saving it as a two-dimensional compensation table of [frequency x gain].

[0050] In one embodiment, due to the difference in energy attenuation caused by the different distances and orientations between the device and the sound source, the smart device is provided with a spatial positioning compensation algorithm. The formula of the spatial positioning compensation algorithm is:

[0051] ;

[0052] Among them, Ecomp is the compensated energy, which is the sound signal energy after distance and direction compensation (final output value); E raw is the original energy, the energy of the original sound signal received by the microphone array (uncompensated); α is the distance compensation factor, the attenuation coefficient of sound wave propagation, which is related to the characteristics of the medium (such as air) and can be determined through TDOA-IMU fusion positioning; β is the direction compensation factor, the directional attenuation coefficient of the microphone array, which is related to the array geometry and the direction of the sound source and can be determined based on the beam pattern database of the microphone array; d is the sound source distance, the straight-line distance from the sound source to the microphone, calculated through TDOA-IMU fusion positioning; θ is the sound source azimuth, the deviation angle of the sound source relative to the main axis of the microphone array (horizontally). Specifically, the spatial positioning compensation algorithm includes the following steps S1310-S1320:

[0053] S1310, distance compensation factor α: positioning through TDOA-IMU fusion.

[0054] Preferably, the above step S1310 includes steps S1311-S1313:

[0055] S1311, the master device broadcasts a timestamp after detecting voice;

[0056] S1312, calculate the arrival time difference from the device, combined with the orientation data of the built-in IMU;

[0057] S1313, solve the three-dimensional coordinates of the sound source using the least squares method.

[0058] S1320, Directional compensation factor β: beam pattern database based on microphone array.

[0059] Preferably, the above step S1320 includes steps S1321-S1322:

[0060] S1321, pre-store the directional modes of various types of equipment;

[0061] S1322: Calculate directional attenuation based on the sound source orientation interpolation.

[0062] In one embodiment, the smart device is provided with a whole array of enhancement processing algorithms, the formula is:

[0063] ;

[0064] Among them, ω is the weight vector, which represents the complex weight vector to be optimized (dimension N mic ×1); R n is the noise covariance matrix (dimension N mic ×N mic , usually a positive Hermitian matrix); a(θ0) is the steering vector, which represents the array response at the beam pointing angle θ0; H is the complex conjugate transpose (Hermitian transpose); ω H R n ω represents minimizing the output noise power (i.e., suppressing noise interference). Specifically, the whole column enhancement processing algorithm includes the following steps S1310-S1320:

[0065] S1410: Generate a standardized beam template to ensure that beam widths of different array devices are consistent.

[0066] S1420, the output energy is multiplied by the array gain compensation coefficient. The formula of the array gain compensation coefficient (AGC) is:

[0067] , ;

[0068] Among them, N mic is the actual number of microphones, indicating the number of microphones actually used in the array; N ref is the number of reference microphones (default 4), indicating the number of benchmark microphones; E beam is the beamforming energy, which represents the signal energy output by the normalized beam template (without compensating for array gain differences); E out is the compensated energy, which represents the final output energy after array gain compensation.

[0069] In one embodiment, due to environmental noise, reverberation, and electromagnetic interference that cause energy measurement distortion, the smart device is equipped with an environmental dynamic compensation algorithm. Specifically, the environmental dynamic compensation algorithm includes the following steps S1510-S1530:

[0070] S1510, input: Mel spectrum + modulation spectrum features (128 dimensions in total).

[0071] S1520, three-layer decision tree classification.

[0072] Preferably, the above step S1520 includes steps S1521-S1523:

[0073] S1521, noise type (steady state / transient);

[0074] S1522, reverberation level (T60 estimate);

[0075] S1523, electromagnetic interference index (based on 50Hz harmonic analysis).

[0076] S1530, dynamically adjust parameters of each compensation module.

[0077] Preferably, the above step S1530 includes steps S1531-S1532:

[0078] S1531, enhances the weight of direct sound in highly reverberant scenes;

[0079] S1532: Enable band-stop filtering in electromagnetic interference scenarios.

[0080] In one embodiment, the electronic device obtains the audio energy value through energy normalization processing. Specifically, the energy normalization processing includes the following steps S1610-S1630:

[0081] S1610, time domain framing (default 20ms / frame), the formula is:

[0082] ,

[0083] Among them, 0≤n≤N-1, x m [n] is the frame signal, which represents the time domain signal of the mth frame (a total of M frames); N is the frame length, which represents the number of sampling points in a single frame and is determined by the 20ms duration; L is the frame shift, which represents the interval between the starting points of adjacent frames (if there is no overlap, L=N).

[0084] S1620, frequency domain weighting, the formula is:

[0085] ;

[0086] Among them, E normis the normalized energy, which represents the frequency domain energy after auditory perception weighting (reflecting the sensitivity of the human ear); E(k) is the subband energy, which represents the energy of the kth cochlear filter subband (calculated by STFT); A(k) is the auditory perception weight (based on equal loudness curves), which represents the subband weighting coefficient based on equal loudness curves (such as the ISO 226 standard), simulating the sensitivity of the human ear to different frequencies; K is the number of non-uniform subbands (cochlear filters), which represents the number of non-uniformly divided frequency bands (cochlear frequency resolution, default K=40).

[0087] S1630, time integral, the formula is:

[0088] ;

[0089] Among them, E total (t) is the total energy, which represents the time-varying energy after smoothing (taking into account both real-time performance and historical data attenuation); T is the integration window length, which represents the time range of energy accumulation (affecting response speed and stability); E norm (t) is the instantaneous value of the normalized energy after frequency domain weighting at time point τ. It is calculated through the frequency domain weighting step and represents the "perceived energy" of the signal after auditory characteristics correction at time point τ. By weighting, it suppresses frequency bands to which the human ear is insensitive (such as very low frequency or high frequency noise) and highlights the frequency bands where speech or key acoustic events are located. λ is the attenuation coefficient, which represents the control of the historical energy attenuation rate. The default value is λ=0.2, taking into account both real-time and stability. t is the current time. τ is the integral variable.

[0090] In one embodiment, the determination condition for normal awakening is that a voice instruction that matches a preset wake-up word is collected, and the determination condition for abnormal awakening is that an abnormal sound signal that is not a voice instruction is collected.

[0091] Specifically, user A falls at home, and the smart bracelet (the first smart device) detects the impact sound (audio energy value 80dB, abnormal wake-up), triggering a wake-up request. User B calls "voice assistant", and the smart speaker (the first smart device) detects the call word (audio energy value 65dB, normal wake-up), triggering a wake-up request.

[0092] S102: Wake up a first smart device with the largest audio energy value as a target device, and obtain environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered a wake-up request or has not been awakened.

[0093] Specifically, after waking up the target device, the system continuously collects environmental information from all second smart devices and dynamically updates the list of awakened second smart devices based on the environmental information. Waking up the target device includes activating a display and / or speaker to interact with the user via the display / speaker, and activating a microphone to collect user voice within a preset time. The second smart device then activates a collection module to collect environmental information within the preset time.

[0094] Furthermore, the environmental information includes at least one of the following: environmental image information collected by a camera; biometric information collected by an infrared sensor; and continuous audio information collected by a microphone.

[0095] In one embodiment, when the audio energy values ​​of multiple first smart devices are the same, a target device is selected based on device priority or geographic location information.

[0096] In one embodiment, when waking up the device, the speaker and display are activated in a preset order. The speaker is used to play audio, and the display is used to display information. For example, when a user speaks a wake-up word and wakes up a smart TV, the smart TV announces "I'm here" through the speaker and / or displays a voice interaction interface on the display. Activating the speaker (prioritized) and display in a preset order prevents instantaneous power overload.

[0097] S103: If the wake-up mode of the target device is abnormal wake-up, wake up at least one second smart device that meets a preset condition according to the environment information of the second smart device.

[0098] Specifically, when the target device wakes up abnormally, it means that an abnormal situation has occurred near the target device, and the user needs to be notified to go to the vicinity of the target device to check.

[0099] In one embodiment, the above step S103 includes: determining whether there is an object that meets the preset target in the environment where the second smart device is located based on the environmental information, and if so, waking up the corresponding second smart device, where the preset target includes an adult or an object with specific biological signs.

[0100] Specifically, the environmental information collected by the second smart device is used to analyze whether there are objects that meet the preset targets near the second smart device. For example, after the smart bracelet (first smart device) triggers an abnormal wake-up, the smart TV (second smart device) uses the camera and infrared sensor to determine whether there is an adult nearby. If so, a notification is made through the speaker and display.

[0101] In one embodiment, the above step S103 further includes: after waking up the at least one second smart device, establishing a two-way call channel between the target device and the second smart device, so that: the audio data received by the target device is transmitted to the second smart device in real time for playback; and the audio data received by the second smart device is transmitted to the target device in real time for playback.

[0102] Specifically, the first smart device establishes a two-way communication channel with the second smart device respectively, so that the first smart device and the second smart device can communicate with each other, that is, cross-device broadcasting. Usually, the number of second smart devices awakened is one.

[0103] In one embodiment, during a two-way call, the distance between the user and the second smart device is monitored in real time. If it is detected that the user moves within a preset range of another third smart device, the following operations are performed:

[0104] Disconnect the communication channel of the second smart device; wake up the third smart device and establish a two-way communication channel between the target device and the third smart device.

[0105] Specifically, if a user is near a speaker (a second smart device) and moves toward the first smart device, and a third smart device is located between the first and second smart devices, as the user approaches the third smart device (while moving away from the second smart device), the call between the second smart device and the first smart device switches to a call between the third smart device and the first smart device. This allows the user to adapt to the changes in their location and switch to the nearest wake-up device, improving the user's voice interaction comfort.

[0106] In one embodiment, the monitoring of the user distance is achieved by at least one of the following methods:

[0107] The user's position is identified based on the camera or infrared sensor of the second smart device; the sound source positioning result is calculated based on the difference in audio energy values ​​of multiple smart devices; and the calculation is based on the correlation between the positioning tag carried by the user and the signal strength of the smart device.

[0108] Furthermore, the sound source localization based on audio energy value differences includes:

[0109] The audio energy value change curves collected by multiple smart devices within a preset time window are obtained, and the change curves are synchronously drawn through a coordinate system with time as the horizontal axis and audio energy value as the vertical axis; if the audio energy value change curves of the awakened device and the unaware device have an intersection in the coordinate system, it is determined that the user has moved to the coverage area of ​​the unaware device, triggering a device switching operation.

[0110] Specifically, at the beginning, the user interacts with device C, and then the user moves to a position close to device A to continue interacting. The second energy curve of device A gradually rises, and the second energy curve of device C gradually decreases. The second energy curves of device A and device C intersect at an intersection, and the awakening state of device C switches to the awakening state of device A.

[0111] Furthermore, the intersection determination condition includes at least one of the following:

[0112] The audio energy value of the unaware device exceeds the audio energy value of the awakened device at the intersection point, and the difference is greater than a preset threshold; the audio energy value change curve of the unaware device maintains an upward trend during the duration after the intersection point; the frequency of occurrence of the intersection point reaches a preset number of times per unit time.

[0113] In one embodiment, establishing the two-way communication channel includes: exclusively controlling the microphones and speakers of the target device and the second smart device, and suppressing the audio collection and playback functions of non-communication devices.

[0114] Suppress the audio collection and playback functions of non-call devices to prevent interference with two-way calls.

[0115] In one embodiment, the sound source localization based on audio energy value differences further includes:

[0116] See also Figure 4-Figure 5 , the audio of a sound source is collected through multiple smart devices; each smart device performs signal preprocessing, hardware difference compensation, spatial positioning compensation, whole column enhancement processing, environmental dynamic compensation, and energy normalization processing on the collected audio to obtain the sound energy of the audio; the smart device calculates and broadcasts the sound energy value of the audio; the smart device also receives the sound energy values ​​calculated by other devices. If the sound energy value broadcast by the target device is greater than all the received sound energy values, the target device is determined to be the device closest to the sound source.

[0117] In one embodiment, when the target device is in the accompanying mode, regardless of whether the wake-up mode is normal wake-up or abnormal wake-up, the following operations are performed:

[0118] After waking up the target device, at least one second smart device is simultaneously woken up; and a two-way communication channel is established between the target device and the second smart device.

[0119] Specifically, when waking up a device in the accompanying mode, a device in a non-accompanying mode is also woken up at the same time, so that other users can care about the user of the accompanying mode device in real time.

[0120] Furthermore, the device in the accompanying mode preferentially collects and identifies audio features of preset objects, which include elderly people or specific guardianship objects, and performs enhancement processing on the collected audio data of the preset objects, wherein the enhancement processing includes at least one of the following operations:

[0121] Improve the target object's speech signal-to-noise ratio based on voiceprint matching; suppress the speech signal of non-target objects; and mark keywords in the speech in real time.

[0122] Furthermore, the audio feature recognition of the preset object is achieved by:

[0123] The fundamental frequency, formant distribution, and speech rate characteristics of the audio signal are extracted and matched with the preset acoustic feature library for the elderly. The age range of the audio is predicted in real time through the trained age classification model, and enhancement processing is triggered if the prediction is for the elderly group.

[0124] Specifically, the enhancement processing includes amplifying the audio signal with an amplification factor of between 1.5 times and 3 times. When the electronic device is set to the accompanying mode, the collected voice interaction audio is subjected to timbre recognition. If it is determined to be an elderly voice feature, the audio signal is enhanced by a pre-processed audio signal amplification method, and then the second sound energy value of the enhanced audio signal is calculated. It should be noted that the principle of the enhancement processing method can be specifically determined by reference to existing data, and the present invention does not make specific limitations. For example: after identifying the voice of an elderly user, adaptive gain control is started to amplify the audio signal by 2 times. The pre-processing stage uses deep learning to identify the elderly voice features and improve sensitivity; special optimization for elderly users improves the success rate of voice interaction.

[0125] In one embodiment, the accompanying mode is further associated with at least one of the following functions:

[0126] When it is detected that the target object has not moved for a long time or makes abnormal physiological sounds, an emergency alarm is automatically triggered; during a two-way call, if the target object does not respond, the alarm level will be gradually increased and more devices will be linked to broadcast alarm information.

[0127] Specifically, the alarm level is increased and more devices are linked to broadcast alarm information to attract the attention of users near multiple devices.

[0128] The embodiments of the present invention can achieve the following advantages:

[0129] The necessity of abnormal wake-up is verified through environmental information; dynamic switching logic improves user experience; audio enhancement and alarm linkage in escort mode ensure user safety.

[0130] The following examples illustrate usage scenarios based on the present application:

[0131] Scenario 1: User A falls at home. The smart bracelet (first device) detects the impact sound (audio energy value 80dB, abnormal wakeup), triggering a wake-up request. After waking up the bracelet, the system uses the camera (second device) to recognize that user A has fallen and no other adults are present. It automatically activates the smart speaker to broadcast a call for help and dial the emergency contact.

[0132] Scenario 2: User B walks from the living room to the bedroom. The audio energy curves of the living room smart screen (target device) and the bedroom smart light (third device) intersect in the coordinate system (the bedroom device's energy value exceeds that of the living room device by 10dB and continues to rise). The system disconnects the living room device and establishes a two-way communication channel with the bedroom device.

[0133] Scenario 3: An elderly person living alone, User C, has their smart TV set to caregiver mode. When User C wakes up the TV, the system simultaneously wakes up their smart wristband and establishes a two-way call. The wristband monitors User C's heart rate in real time. If it detects abnormal heart rate fluctuations, it triggers a flashing alarm on the TV screen and sends a notification to the caregiver's phone.

[0134] See also Figure 2 An embodiment of the present invention further provides a voice wake-up control apparatus 400 based on multi-device collaboration, which includes an acquisition unit 401 , a first wake-up unit 402 , and a second wake-up unit 403 .

[0135] The acquisition unit 401 is configured to acquire wake-up requests sent by multiple first smart devices, wherein the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, and the wake-up modes include normal wake-up and abnormal wake-up.

[0136] In one embodiment, the determination condition for normal awakening is that a voice instruction that matches a preset wake-up word is collected, and the determination condition for abnormal awakening is that an abnormal sound signal that is not a voice instruction is collected.

[0137] The first awakening unit 402 is configured to awaken the first smart device with the largest audio energy value as a target device, and obtain environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered an awakening request or has not been awakened.

[0138] In one embodiment, when the audio energy values ​​of multiple first smart devices are the same, a target device is selected based on device priority or geographic location information.

[0139] The second awakening unit 403 is configured to awaken at least one second smart device that meets a preset condition according to the environment information of the second smart device if the awakening mode of the target device is abnormal awakening.

[0140] In one embodiment, it is determined based on the environmental information whether there is an object that meets a preset target in the environment where the second smart device is located. If so, the corresponding second smart device is awakened, where the preset target includes an adult or an object with specific biological signs.

[0141] In one embodiment, after waking up the at least one second smart device, a two-way communication channel is established between the target device and the second smart device, so that: the audio data received by the target device is transmitted to the second smart device in real time for playback; and the audio data received by the second smart device is transmitted to the target device in real time for playback.

[0142] In one embodiment, during a two-way call, the distance between the user and the second smart device is monitored in real time. If it is detected that the user moves within a preset range of another third smart device, the following operations are performed:

[0143] Disconnect the communication channel of the second smart device; wake up the third smart device and establish a two-way communication channel between the target device and the third smart device.

[0144] In one embodiment, the monitoring of the user distance is achieved by at least one of the following methods:

[0145] The user's position is identified based on the camera or infrared sensor of the second smart device; the sound source positioning result is calculated based on the difference in audio energy values ​​of multiple smart devices; and the calculation is based on the correlation between the positioning tag carried by the user and the signal strength of the smart device.

[0146] In one embodiment, establishing the two-way communication channel includes: exclusively controlling the microphones and speakers of the target device and the second smart device, and suppressing the audio collection and playback functions of non-communication devices.

[0147] In one embodiment, when the target device is in the accompanying mode, regardless of whether the wake-up mode is normal wake-up or abnormal wake-up, the following operations are performed:

[0148] After waking up the target device, at least one second smart device is simultaneously woken up; and a two-way communication channel is established between the target device and the second smart device.

[0149] like Figure 3 As shown, Figure 3 This is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 can be a terminal or a server. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, personal digital assistant, wearable device, or other electronic device with communication capabilities. The server can be a standalone server or a server cluster consisting of multiple servers.

[0150] The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0151] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 may execute a voice wake-up control method based on multi-device collaboration.

[0152] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0153] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a voice wake-up control method based on multi-device collaboration.

[0154] The network interface 505 is used to communicate with other devices over the network. Those skilled in the art will appreciate that the above structure is merely a block diagram of a portion of the structure related to the present invention and does not limit the computer device 500 to which the present invention is applied. A specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0155] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0156] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0157] Therefore, the present invention further provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program.

[0158] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, a removable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk, among other physical storage media capable of storing program code. The computer-readable storage medium may be either non-volatile or volatile.

[0159] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0160] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0161] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0162] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (such as a personal computer, terminal, or network device) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0163] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, to the extent such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to encompass such changes and modifications.

[0165] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A voice wake-up control method based on multi-device collaboration, characterized in that: The method comprises: Obtaining wake-up requests sent by multiple first smart devices, wherein the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, wherein the wake-up modes include normal wake-up and abnormal wake-up, wherein the determination condition for the normal wake-up is the collection of a voice command that matches a preset wake-up word, and the determination condition for the abnormal wake-up is the collection of an abnormal sound signal that is not a voice command; Waking up the first smart device with the largest audio energy value as the target device, and obtaining environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered a wake-up request or has not been awakened; If the wake-up mode of the target device is abnormal wake-up, at least one second smart device that meets a preset condition is woken up according to the environment information of the second smart device.

2. The method according to claim 1, characterized in that The waking up at least one second smart device that meets a preset condition includes: Based on the environmental information, it is determined whether there is an object that meets the preset target in the environment where the second smart device is located. If so, the corresponding second smart device is awakened, where the preset target includes an adult or an object with preset biological signs.

3. The method according to claim 2, characterized in that After waking up the at least one second smart device, a two-way communication channel is established between the target device and the second smart device, such that: The audio data received by the target device is transmitted to the second smart device in real time for playback; The audio data received by the second smart device is transmitted to the target device in real time for playback.

4. The method according to claim 2, characterized in that During a two-way call, the distance between the user and the second smart device is monitored in real time. If the user is detected to be within a preset range of another third smart device, the following operations are performed: Disconnecting the call channel of the second smart device; The third smart device is awakened and a two-way communication channel is established between the target device and the third smart device.

5. The method according to claim 1, wherein When the target device is in the accompanying mode, no matter the wake-up mode is normal wake-up or abnormal wake-up, the following operations are performed: After waking up the target device, synchronously waking up at least one second smart device; A two-way communication channel is established between the target device and the second smart device.

6. The method according to claim 5, characterized in that The accompanying mode is also associated with at least one of the following functions: When it is detected that the target object has not moved for a long time or makes abnormal physiological sounds, an emergency alarm is automatically triggered; During a two-way call, if the target party does not respond, the alarm level will be gradually increased and more devices will be linked to broadcast alarm information.

7. A voice wake-up control device based on multi-device collaboration, characterized in that: include: an acquisition unit, configured to acquire wake-up requests sent by multiple first smart devices, wherein the wake-up requests include audio energy values ​​collected by the first smart devices and corresponding wake-up modes, wherein the wake-up modes include normal wake-up and abnormal wake-up, wherein the determination condition for the normal wake-up is the collection of a voice command that meets a preset wake-up word, and the determination condition for the abnormal wake-up is the collection of an abnormal sound signal that is not a voice command; A first awakening unit is configured to awaken a first smart device with a maximum audio energy value as a target device, and obtain environmental information of at least one second smart device, where the second smart device is a smart device that has not triggered an awakening request or has not been awakened; The second awakening unit is configured to awaken at least one second smart device that meets a preset condition according to the environment information of the second smart device if the awakening mode of the target device is abnormal awakening.

8. A computer device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the steps of the method according to any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images