Remote voice control method, device and system

Abstract

The present application discloses a remote voice control method, device and system for performing identity verification on the voiceprint of voice commands to improve the security of remote voice control. The present application provides a remote voice control method, comprising: when a user successfully logs in to a local server through a terminal, receiving a user voice command sent by the terminal; performing voiceprint verification on the voice command, and when the verification passes, remotely controlling the smart home device through the voice command.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and in particular to a remote voice control method, device and system. Background Art

[0002] With the development of science and technology, voice remote control has also begun to appear slowly. Users use their mobile phones to issue voice commands to remotely control smart home devices.

[0003] However, the existing remote voice control only transmits voice commands without performing voice command identity verification, which leads to the problem of insecurity of voice commands. Summary of the invention

[0004] The embodiments of the present application provide a remote voice control method, device and system for performing identity verification on the voiceprint of voice commands to improve the security of remote voice control.

[0005] A remote voice control method provided in an embodiment of the present application includes:

[0006] When the user successfully logs in to the local server through the terminal, receiving the user voice command sent by the terminal;

[0007] The voice command is subjected to voiceprint verification, and when the verification passes, remote control of the smart home device is achieved through the voice command.

[0008] Through this method, when a user successfully logs in to a local server through a terminal, a user voice command sent by the terminal is received; a voiceprint verification is performed on the voice command, and when the verification passes, remote control of the smart home device is achieved through the voice command, thereby achieving identity verification of the voiceprint of the voice command, thereby improving the security of remote voice control.

[0009] Optionally, the method further comprises:

[0010] Receive facial images and voice information collected by users through terminals;

[0011] Extracting facial features of the facial image and voiceprint features of the voice information;

[0012] The voiceprint feature is fused with the facial feature, and based on the fusion result, it is determined whether the user can log in to the local server.

[0013] In the embodiment of the present application, the user's face verification alone can be cracked by photos or masks, and the voiceprint has recognition loopholes for recorded and cloned voices. In the embodiment of the present application, facial features and voiceprint features are integrated to increase the reliability of user login verification information.

[0014] Optionally, fusing the voiceprint feature with the facial feature, and judging whether the user can log in to the local server based on the fusion result, specifically includes:

[0015] Constructing a fusion feature vector based on the vector of the facial feature and the vector of the voiceprint feature;

[0016] Determining a weight matrix of the fused feature vector;

[0017] Each characteristic weight in the weight matrix is ​​summed up to finally obtain a value. If the value belongs to a preset value range, it is determined that the user has successfully logged into the local server through the terminal.

[0018] Optionally, remote control of smart home devices is achieved through the voice command, specifically including:

[0019] The voice command is transmitted to the coordinator and the speaker module respectively, the power switch of the smart home device is controlled by the coordinator, and the command classification and transmission are performed by the speaker module according to the keywords of the smart home device contained in the voice command, so that the voice command is transmitted to the speaker corresponding to the smart home device corresponding to the keyword, and the voice command is played to the smart home device corresponding to the keyword through the speaker.

[0020] In order to solve the problem that commands for remote voice control of home appliances cannot be fed back in a timely manner, the embodiments of the present application classify user commands and implement two-way communication between the user and the local server, and between the local server and the smart home appliances, thereby achieving timely feedback and optimization of remote control home appliance information.

[0021] Optionally, the method further comprises:

[0022] When the voice command cannot be executed by the corresponding smart home device, error command prompt information is fed back to the user terminal.

[0023] Optionally, when the verification passes, the method further includes:

[0024] If the voice command cannot be processed offline on the local server, the cloud server is called to process the voice command.

[0025] A remote voice control device provided in an embodiment of the present application includes:

[0026] A memory for storing program instructions;

[0027] The processor is used to call the program instructions stored in the memory and execute any of the above methods according to the obtained program.

[0028] A remote voice control system provided by an embodiment of the present application comprises the remote voice control device, a coordinator and a speaker module respectively connected to the remote voice control device, and at least one speaker connected to the speaker module; wherein:

[0029] The coordinator is used to receive the voice command sent by the remote voice control device and control the power switch of the smart home device based on the voice command;

[0030] The speaker module is used to determine the keywords of the smart home device contained in the voice command sent by the remote voice control device, send the voice command to the speaker corresponding to the smart home device corresponding to the keyword, and play the voice command to the smart home device corresponding to the keyword through the speaker.

[0031] Optionally, the system also includes a smart home device arranged corresponding to each of the speakers.

[0032] Another embodiment of the present application provides a computing device, which includes a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory, and execute any of the above methods according to the obtained program.

[0033] Another embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to enable the computer to execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of the overall system framework provided for an embodiment of the present application;

[0036] Figure 2 A schematic diagram of voiceprint feature collection and recognition provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of facial feature collection and recognition provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of the fusion of face and voiceprint features provided in an embodiment of the present application;

[0039] Figure 5A schematic diagram of the user identity registration and identity login process provided in the embodiment of the present application;

[0040] Figure 6 A schematic diagram of a voice signal remote transmission and voice command parsing framework provided in an embodiment of the present application;

[0041] Figure 7 A schematic diagram of a remote voice control smart home device framework provided in an embodiment of the present application;

[0042] Figure 8 A flowchart of a remote voice control method provided in an embodiment of the present application;

[0043] Fig. 9 A schematic diagram of the structure of a remote voice control device provided in an embodiment of the present application;

[0044] Fig.10 A schematic diagram of the structure of another remote voice control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The embodiments of the present application provide a remote voice control method and device for verifying the identity of the person who issues a user voice command, increasing the reliability of user login verification information, and improving the security of remote voice control.

[0047] Among them, the method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0048] The following is a detailed description of each embodiment of the present application in conjunction with the accompanying drawings. It should be noted that the display order of the embodiments of the present application only represents the order of the embodiments, and does not represent the advantages and disadvantages of the technical solutions provided by the embodiments.

[0049] The embodiment of the present application provides a remote voice-controlled smart home system that a user can trust, specifically including:

[0050] 1. For the existing remote voice control, only voice commands are transmitted without voice command identity verification, which leads to the problem of insecurity of voice commands. In the embodiment of the present application, the voice signal is divided into a voiceprint signal and a voice command, and the identity of each voice step of the user is verified;

[0051] Second, the user's face verification alone can be cracked by photos or masks, and the voiceprint has recognition vulnerabilities for recorded and cloned voices. The embodiment of the present application adopts the fusion of face features and voiceprint features to increase the reliability of user login verification information;

[0052] 3. To address the problem that remote voice control of home appliance commands cannot provide timely feedback, the embodiments of the present application classify user commands and implement two-way communication between users and local servers, and between local servers and smart home appliances, thereby achieving timely feedback and optimization of remote control home appliance information.

[0053] In summary, the embodiments of the present application have the following advantages:

[0054] 1. The voiceprint and face fusion features are used to build user registration and login pages, and the user login verification function of the fusion features is realized, which avoids the loopholes of face verification or voiceprint verification alone and increases the security of the login system;

[0055] 2. After the user successfully logs in, each voice is analyzed and the voiceprint is verified again to ensure that each message is verified, so as to realize real-time monitoring of the legitimacy of user behavior and further ensure the security of remote voice control;

[0056] 3. Build a smart home hardware system to play user information to related devices through the speaker function of the smart speaker. The related devices can implement related voice operations and can feedback information to users in a timely manner, realizing the realization and feedback of remote control functions.

[0057] The embodiment of the present application mainly relates to a remote control system based on real-time monitoring. It can provide users with a new remote command recognition method to protect the privacy and property safety of users. The system framework provided by the embodiment of the present application is as follows: Figure 1 As shown, its main contents include:

[0058] (1) Collect user voiceprint features and build a voiceprint database.

[0059] (2) Collect user facial features and build a face database.

[0060] (3) Fusion of voiceprint features with facial features.

[0061] (4) Build a user identity registration system and identity login system.

[0062] (5) Voice signal remote transmission and voice command analysis system.

[0063] (6) Design of remote voice control framework for smart home devices.

[0064] Regarding the above content, the specific embodiments are described as follows:

[0065] (1) Collect user voiceprint features and build a voiceprint database:

[0066] The present application embodiment is about voiceprint feature collection and recognition. Figure 2 As shown in the figure, it is divided into five stages: speech input, preprocessing, speech feature extraction, classifier and construction of voiceprint database.

[0067] Voice input: Specify relevant text through the mobile terminal, and collect user voice by reading the text.

[0068] Preprocessing: The audio segments in the original speech are obtained through time domain analysis, and the audio recognition effect is improved through high-pass filter pre-emphasis processing. The audio frames are obtained through window function framing and post-processing. The short-time energy and short-time zero-crossing rate of the audio frames are detected, and the starting and ending points of the speech signal are determined by setting the threshold.

[0069] The short-time energy refers to the energy of each frame of speech signal.

[0070] The short-time zero-crossing rate indicates the number of times the waveform of each frame of speech signal crosses the zero axis.

[0071] About setting the threshold: Using the average short-time zero-crossing rate and short-time energy as the threshold for speech signal endpoint detection is a basic double-threshold endpoint detection algorithm. Among them, the endpoint detection effect is best when the high threshold of short-time energy is set to 4 to 5 times the low threshold and the high threshold of short-time zero-crossing rate is set to about 2 times the low threshold.

[0072] Speech feature extraction: Mel-frequency cepstral coefficients (MFCC) and first-order difference coefficients are used to extract speech features. The preprocessed speech signal is then processed into a feature vector through a Gaussian mixture model plus a background model. The feature vector is then reduced in dimension to form a low-dimensional and discriminative voiceprint feature.

[0073] The Mel-Frequency Cepstral Coefficient (MFCC) is a linear frequency coefficient determined based on the human ear's sensory judgment of equidistant pitch changes.

[0074] The mixed Gaussian model, i.e., using the Gaussian probability density function to accurately quantify things, is a model that decomposes things into several Gaussian probability density functions.

[0075] The background model refers to the background sound of speech, and the background sound can be removed by subtracting two adjacent frames of the speech signal to avoid interference.

[0076] The first-order difference coefficient: between two consecutive frames, the difference between the feature parameters of the latter frame and the previous frame is made, reflecting the connection between the current speech frame and the previous frame.

[0077] The feature vector is a vector composed of multiple parameters that describe the characteristics of the speech signal.

[0078] The dimension reduction is to reduce the high-dimensional feature vector to a low dimension through an algorithm (such as PCA).

[0079] The classifier: selects the voice database VoxForge, uses the extracted voice feature vector, and constructs a classifier for voiceprint identity recognition through support vector product (SVM).

[0080] The speech database VoxForge: VOxForge is an open source speech corpus and acoustic model library, which is commonly used in academia and has strong robustness for testing speech models in different intonation and stress environments.

[0081] The support vector machine (SVM) is a classifier that performs binary classification on data in a supervised learning manner.

[0082] The voiceprint database is constructed by the classifier of the voiceprint recognition in the previous step. Then the voice is input again, and after signal processing, feature extraction and recognition, the user's voiceprint information is found in the voiceprint database to complete the matching.

[0083] (2) Collect user facial features and build a face database:

[0084] In this embodiment of the present application, facial feature collection and recognition are as follows: Figure 3 As shown, it is divided into five stages: face acquisition, image preprocessing, image feature extraction, classifier, and building a face database.

[0085] Face collection: collect face images through mobile camera;

[0086] Image preprocessing: The collected face image data is grayed out, then filtered to reduce noise, and then histogram equalized to enhance the image features.

[0087] Image feature extraction: Use the Histogram of Oriented Gradients (HOG) to describe the features of the face image. Divide the face image into 3×3 sub-blocks, extract HOG features from each sub-block, and then use the Principal Component Analysis (PCA) to reduce the dimensionality of the high-dimensional HOG features.

[0088] Classifier: Select the face database CAS-PEAL, use the image features extracted in the previous step as the classification basis, and construct a face recognition classifier through the support vector machine (SVM);

[0089] Build a face recognition database: Construct a face database through the face recognition classifier obtained in the previous step, and input the face image again. After image preprocessing, feature extraction, and recognition, find the user's voiceprint information in the face database to complete the match.

[0090] (3) Fusion of voiceprint features with facial features.

[0091] In this embodiment of the present application, facial features and voiceprint features are integrated as follows: Figure 4 As shown, the voiceprint features and facial features are fused at the feature level, which is divided into three parts: constructing a fused feature vector, setting weights for each dimension, and recognition.

[0092] Construct fusion feature vector: Use F = (f1,f2,f3,…,f m ) represents the feature vector of the face, where f1,f2,f3,…,f m Represents the eigenvalue of each feature describing the face. Use V = (v1, v2, v3, ..., v n ) represents the feature vector of the voiceprint, where v1,v2,v3,…,v n Represents the eigenvalue of each feature describing the voiceprint. After dimension normalization, F and V are fused into a new eigenvector S = (s1, s2, s3, ..., s m+n ), where s1,s2,s3,…,s m+n Represents the feature value of each feature describing the fusion features of face and voiceprint;

[0093] Weight determination: Select two sets of fused feature vectors S1 = (s 11 ,s 12 ,s 13 ,…,s 1(m+n) )、S2=(s 21 ,s 22 ,s 23 ,…,s 2(m+n) ), where s 11 ,s 12 ,s 13 ,…,s 1(m+n) Represents the eigenvalue of each feature of the first group describing the fusion features of face and voiceprint, s 21 ,s 22 ,s 23 ,…,s 2(m+n Represents the feature value of each feature of the second group describing the fusion features of face and voiceprint. The average distance between S1 and S2 is calculated according to the following formula:

[0094]

[0095] The weight of the feature vector of each face and voiceprint fusion feature is calculated according to the following formula:

[0096]

[0097] Get the weight matrix W = (w1,w2,w3,…,w m+n ), repeat the above calculation process, after multiple calculations, multiple different weight matrices are obtained, and then the average value is taken as the final weight;

[0098] Recognition: By inputting face and voice information, a fused feature vector is obtained. After step (3), its weight matrix is ​​obtained. Each element of the weight matrix represents a feature weight. The feature weights are added together to finally obtain a value. Based on the numerical range of this value, the recognition result is obtained. For example, if the match is 80%, the recognition is considered successful.

[0099] Specifically, for example, the fused feature vector value is S = (0.8, 0.7, 0.5), and the weight matrix of each feature is W = (0.5, 0.4, 0.1). The feature weights are added together to calculate the matching degree P = 0.8*0.5 + 0.7*0.4 + 0.5*0.1 = 0.73. If the setting standard is that P is greater than or equal to 0.8 for a successful match, then the recognition result 0.73 is less than 0.8, which is unsuccessful, that is, the recognition fails.

[0100] (4) Build a user identity registration system and identity login system:

[0101] User identity registration and identity login system Figure 5 As shown, the embodiment of the present application will construct a user identity registration system and an identity login system, which is divided into two parts: training of voiceprint and facial features, and identity login verification.

[0102] Voiceprint and face feature training: After the voiceprint and face information of the user's family members are collected, they are transmitted to the family's local server database on an individual basis. The voiceprint and face features are then extracted and trained on the local server after feature fusion. After the training is completed, the training results are saved to the local hard disk.

[0103] Identity login verification: The user logs in on the mobile terminal, and the mobile terminal inputs the user's login face and voiceprint information into the home local server, and retrieves the personal face and voiceprint data from the local server hard disk for matching. If the match is successful, the user logs in successfully, otherwise the login fails;

[0104] The embodiment of the present application is to realize user information registration. When the user wants to remotely control home appliances, he needs to log in to the mobile terminal App first. The mobile terminal App login requires simultaneous verification of voiceprint and face, ensuring the reliability of the user's login identity.

[0105] (5) Voice signal remote transmission and voice command analysis system:

[0106] The voice signal remote transmission and voice command parsing framework in the embodiment of the present application is as follows: Figure 6 As shown, it is divided into three parts: remote transmission of voice signals, analysis and identity authentication of voice signal data, and transmission of voice commands.

[0107] Remote transmission of voice signals: Input voice signals on the mobile terminal, and transmit the voice signals to the local server;

[0108] Voice signal data analysis and identity authentication: After receiving the voice signal, the voice data is divided into two parts: voiceprint data and voice command data. The voiceprint data is first transmitted to the local server for voiceprint database verification. If the voiceprint exists in the local database, the verification is successful, otherwise the verification fails;

[0109] Voice command transmission: After the voiceprint information is verified, the voice command is transmitted to the relevant smart home devices through the local server to achieve remote control by voice.

[0110] In the embodiment of the present application, after the user successfully logs in, each instruction of the user is further verified to ensure that the issuer of each instruction of the user is a registrant of the user's family member, thereby ensuring that each instruction of the user's remote control of the smart home is safe and reliable.

[0111] (6) Design of remote voice control framework for smart home devices:

[0112] The remote voice control smart home device framework designed in the embodiment of the present application is as follows: Figure 7 As shown, the dotted box represents the smart home appliances and the speaker devices that act as speakers for them. For example, speaker 1 serves as the speaker of the indoor environment acquisition module, and the indoor environment acquisition module includes a humidifier, an air purifier, etc.; speaker 2 serves as the speaker of the home equipment module, and the home equipment module includes an air conditioner, a refrigerator, a television, etc.; speaker 3 serves as the speaker of the basic module, and the basic module includes a lamp, a curtain, etc. Figure 7 The security module in here only reflects the integrity of home equipment and is not necessarily within the scope of remote control.

[0113] The remote voice control smart home device framework designed in the embodiment of the present application is that the user inputs voice information to the local server at the terminal. The local server processes the information that can be processed offline locally, and for the information that cannot be processed locally, it is necessary to call the relevant services in the cloud.

[0114] Among them, the services that need to be obtained in the cloud include, for example:

[0115] Internet resources, such as songs, crosstalk, etc.;

[0116] For speech recognition, the local server computing power is insufficient, and it is necessary to utilize the computing power of the cloud server, such as voice cloning and voice synthesis.

[0117] The local server can realize two-way communication. The voice module in the local server processes the voice signal input by the user (for example, the user inputs "open the curtains", the local server executes the operation of opening the curtains, and then transmits the result of whether the curtains are opened to the user) through the control module in the local server to the coordinator and the speaker module (the speaker module can be used as an intelligent speaker independently to realize basic intelligent speaker operations, and can also transmit information of the user's remote control of home appliances). The coordinator is responsible for turning on the power switch of the smart home related equipment to power on the equipment (if it is already powered on, this operation is not required); the speaker module adopts a distributed speaker (deployed in multiple rooms and can communicate with each other). The speaker module receives relevant instructions and classifies and transmits the instructions according to the keywords of the home appliances contained in the relevant instructions (for example, "open the curtains", including "curtains", the instruction will be classified into the basic module), and transmits it to the speaker close to the corresponding device, that is, different types of voice instructions are sent to different speakers for playback. At this time, speakers 1, 2, and 3 act as speakers at this time. The speaker plays the relevant voice instructions, and the voice modules of the relevant home appliances receive the instructions to realize voice control.

[0118] When the voice module in the local server detects that the voice command sent by the user terminal is not a command of the smart home device, it will feedback to the user terminal and prompt the user terminal that the user command failed.

[0119] In the embodiment of the present application, the remote voice control smart home is divided into three modules according to the type of received instructions:

[0120] Basic module: including lights and curtains, only accepts opening and closing related instructions, and other instructions (except opening and closing instructions) will be returned to the local server, and the local server will then feedback to the user, prompting the user that the instruction failed; the other instructions, that is, the instructions that the basic module cannot recognize, are illegal instructions; the other instructions are returned to the local server, and the instructions that the basic module cannot execute will return an unexecutable signal, which will be returned to the local server;

[0121] Indoor environment acquisition module: including humidifier and air purifier, receiving on, off and mode instructions, returning other instructions to the local server and feeding back to the user; the other instructions are instructions that the indoor environment acquisition module cannot recognize; the other instructions returned to the local server, that is, the instructions that the indoor environment acquisition module cannot execute, will return an unexecutable signal, which will be returned to the local server;

[0122] Home equipment module: including air conditioners, televisions, refrigerators, etc., in addition to instructions such as on, off, mode, etc., it also involves other complex instructions. If the relevant instructions do not exist, they will also be fed back to the user through the local server; if the relevant instructions do not exist, they will be fed back to the user through the local server, that is, the instructions that the home equipment module cannot execute will return a signal that cannot be executed. This signal will be returned to the local server and fed back to the user through the local server.

[0123] The above classification is only an example, and the technical solutions provided in the embodiments of the present application are not limited to this classification method.

[0124] In summary, in order to solve the problem that the local server cannot process relevant voice commands, the embodiment of the present application protects the privacy of the user to a certain extent. The local server transmits some commands to the cloud and ensures the implementation of relevant commands. For example, when the user needs voice cloning, the voice cloning model training process with high requirements on server computing power is implemented by the cloud, and the voice model obtained by the voice cloning training in the cloud is placed on the local server.

[0125] The embodiment of the present application also solves the problem of powering on related home appliances. Through the coordinator, the related touch switch is turned on to power on the related equipment.

[0126] The embodiment of the present application utilizes distributed speakers as loudspeakers, which solves the problem of home appliances being distributed in different rooms receiving instructions; and classifies smart home appliances to facilitate timely feedback of user erroneous instructions to users.

[0127] See also Figure 8 On the local server side, a remote voice control method provided by an embodiment of the present application includes:

[0128] S101. When a user successfully logs in to a local server through a terminal, receiving a user voice command sent by the terminal;

[0129] S102: Perform voiceprint verification on the voice command, and when the verification passes, remote control of the smart home device is achieved through the voice command.

[0130] In an embodiment of the present application, after the user successfully logs in to the local server through the terminal, each voice command can be further verified by voiceprint, and subsequent operations can be performed only after the verification passes, thereby improving the security of remote voice control.

[0131] Optionally, the method further comprises:

[0132] Receive facial images and voice information collected by users through terminals;

[0133] Extracting facial features of the facial image and voiceprint features of the voice information;

[0134] The voiceprint feature is fused with the facial feature, and based on the fusion result, it is determined whether the user can log in to the local server.

[0135] In the embodiment of the present application, the user's face verification alone can be cracked by photos or masks, and the voiceprint has recognition loopholes for recorded and cloned voices. In the embodiment of the present application, facial features and voiceprint features are integrated to increase the reliability of user login verification information.

[0136] Optionally, fusing the voiceprint feature with the facial feature, and judging whether the user can log in to the local server based on the fusion result, specifically includes:

[0137] Based on the vector of the face feature and the vector of the voiceprint feature, a fusion feature vector is constructed; for example, F = (f1, f2, f3, ..., f m ) and V=(v1,v2,v3,…,v n ), we get the above S=(s1,s2,s3,…,s m+n );

[0138] Determine the weight matrix of the fused feature vector; for example, the above W = (w1, w2, w3, ..., w m+n );

[0139] Each feature weight in the weight matrix is ​​added up to finally obtain a numerical value. If the numerical value belongs to the preset numerical range, it is determined that the user has successfully logged into the local server through the terminal. For example, the fused feature vector value obtained is S = (0.8, 0.7, 0.5), and the weight matrix of each feature is W = (0.5, 0.4, 0.1). The feature weights are added up to obtain a numerical value, that is, the matching degree P = 0.8*0.5+0.7*0.4+0.5*0.1 = 0.73 is calculated. If the set standard is that P is greater than or equal to 0.8 for a successful match, then this recognition result 0.73 is less than 0.8, which is unsuccessful, that is, the recognition fails.

[0140] Optionally, remote control of smart home devices is achieved through the voice command, specifically including:

[0141] The voice command is transmitted to the coordinator and the speaker module respectively, the power switch of the smart home device is controlled by the coordinator, and the command classification and transmission are performed by the speaker module according to the keywords of the smart home device contained in the voice command, so that the voice command is transmitted to the speaker corresponding to the smart home device corresponding to the keyword, and the voice command is played to the smart home device corresponding to the keyword through the speaker.

[0142] In order to solve the problem that commands for remote voice control of home appliances cannot be fed back in a timely manner, the embodiments of the present application classify user commands and implement two-way communication between the user and the local server, and between the local server and the smart home appliances, thereby achieving timely feedback and optimization of remote control home appliance information.

[0143] Optionally, the method further comprises:

[0144] When the voice command cannot be executed by the corresponding smart home device, error command prompt information is fed back to the user terminal.

[0145] For example, if a signal is received indicating that the basic module cannot execute an instruction, error instruction prompt information is fed back to the user terminal.

[0146] Optionally, when the verification passes, the method further includes:

[0147] If the voice command cannot be processed offline on the local server, the cloud server is called to process the voice command.

[0148] For example, the local server processes the information that can be processed offline locally, and for the information that cannot be processed locally, it needs to call the cloud server for related services, such as:

[0149] Obtain Internet resources from cloud servers, such as songs, crosstalk, etc.;

[0150] When performing speech recognition through cloud servers, the local server's computing power is insufficient, and it is necessary to utilize the computing power of the cloud server to perform processes such as voice cloning and voice synthesis.

[0151] See also Fig. 9 , a remote voice control device provided in an embodiment of the present application includes:

[0152] Memory 520, used to store program instructions;

[0153] The processor 500 is configured to call the program instructions stored in the memory and execute according to the obtained program:

[0154] When the user successfully logs in to the local server through the terminal, receiving the user voice command sent by the terminal;

[0155] The voice command is subjected to voiceprint verification, and when the verification passes, remote control of the smart home device is achieved through the voice command.

[0156] Optionally, the processor 500 is further configured to call a program instruction stored in the memory, and execute according to the obtained program:

[0157] Receive facial images and voice information collected by users through terminals;

[0158] Extracting facial features of the facial image and voiceprint features of the voice information;

[0159] The voiceprint feature is fused with the facial feature, and based on the fusion result, it is determined whether the user can log in to the local server.

[0160] Optionally, fusing the voiceprint feature with the facial feature, and judging whether the user can log in to the local server based on the fusion result, specifically includes:

[0161] Constructing a fusion feature vector based on the vector of the facial feature and the vector of the voiceprint feature;

[0162] Determining a weight matrix of the fused feature vector;

[0163] Each characteristic weight in the weight matrix is ​​summed up to finally obtain a value. If the value belongs to a preset value range, it is determined that the user has successfully logged into the local server through the terminal.

[0164] Optionally, remote control of smart home devices is achieved through the voice command, specifically including:

[0165] The voice command is transmitted to the coordinator and the speaker module respectively, the power switch of the smart home device is controlled by the coordinator, and the command classification and transmission are performed by the speaker module according to the keywords of the smart home device contained in the voice command, so that the voice command is transmitted to the speaker corresponding to the smart home device corresponding to the keyword, and the voice command is played to the smart home device corresponding to the keyword through the speaker.

[0166] Optionally, the processor 500 is further configured to call a program instruction stored in the memory, and execute according to the obtained program:

[0167] When the voice command cannot be executed by the corresponding smart home device, error command prompt information is fed back to the user terminal.

[0168] Optionally, when the verification passes, the processor 500 is further configured to call the program instructions stored in the memory and execute according to the obtained program:

[0169] If the voice command cannot be processed offline on the local server, the cloud server is called to process the voice command.

[0170] The transceiver 510 is configured to receive and send data under the control of the processor 500 .

[0171] Among them, Fig. 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 500 and memory represented by memory 520. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 510 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 500 when performing operations.

[0172] The processor 500 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD).

[0173] A remote voice control system provided in the embodiment of the present application can be found in Figure 7 (but not limited to Figure 7 The structure shown in the figure) comprises the remote voice control device (i.e., the local server), a coordinator and a speaker module respectively connected to the remote voice control device, and at least one speaker connected to the speaker module; wherein,

[0174] The coordinator is used to receive the voice command sent by the remote voice control device and control the power switch of the smart home device based on the voice command;

[0175] The speaker module is used to determine the keywords of the smart home device contained in the voice command sent by the remote voice control device, send the voice command to the speaker corresponding to the smart home device corresponding to the keyword, and play the voice command to the smart home device corresponding to the keyword through the speaker.

[0176] Optionally, the system further includes a smart home device corresponding to each of the speakers, such as Figure 7 The speaker 1 corresponds to the indoor environment collection module, the speaker 2 corresponds to the home equipment module, and the speaker 3 corresponds to the basic module.

[0177] It should be noted that there may be no connection between the above-mentioned speakers and the corresponding smart home devices, and they are only used as speakers, that is, playing voice commands to the corresponding smart home devices, and the smart home devices perform corresponding operations after receiving the voice commands.

[0178] See also Fig.10 Another remote voice control device provided in an embodiment of the present application includes:

[0179] The first unit 11 is used to receive a user voice command sent by the terminal when the user successfully logs in to the local server through the terminal;

[0180] The second unit 12 is used to perform voiceprint verification on the voice command, and when the verification passes, remote control of the smart home device is achieved through the voice command.

[0181] Optionally, the first unit 11 is further used for:

[0182] Receive facial images and voice information collected by users through terminals;

[0183] Extracting facial features of the facial image and voiceprint features of the voice information;

[0184] The voiceprint feature is fused with the facial feature, and based on the fusion result, it is determined whether the user can log in to the local server.

[0185] Optionally, fusing the voiceprint feature with the facial feature, and judging whether the user can log in to the local server based on the fusion result, specifically includes:

[0186] Constructing a fusion feature vector based on the vector of the facial feature and the vector of the voiceprint feature;

[0187] Determining a weight matrix of the fused feature vector;

[0188] Each characteristic weight in the weight matrix is ​​summed up to finally obtain a value. If the value belongs to a preset value range, it is determined that the user has successfully logged into the local server through the terminal.

[0189] Optionally, remote control of smart home devices is achieved through the voice command, specifically including:

[0190] The voice command is transmitted to the coordinator and the speaker module respectively, the power switch of the smart home device is controlled by the coordinator, and the command classification and transmission are performed by the speaker module according to the keywords of the smart home device contained in the voice command, so that the voice command is transmitted to the speaker corresponding to the smart home device corresponding to the keyword, and the voice command is played to the smart home device corresponding to the keyword through the speaker.

[0191] Optionally, the second unit 12 is further configured to:

[0192] When the voice command cannot be executed by the corresponding smart home device, error command prompt information is fed back to the user terminal.

[0193] Optionally, when the verification passes, the first unit 11 is further configured to:

[0194] If the voice command cannot be processed offline on the local server, the cloud server is called to process the voice command.

[0195] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0197] The embodiment of the present application provides a computing device, which may be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, an input / output device, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0198] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In an embodiment of the present application, the memory may be used to store the program of any of the methods provided in the embodiments of the present application.

[0199] The processor calls the program instructions stored in the memory, and the processor is used to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions.

[0200] An embodiment of the present application provides a computer storage medium for storing computer program instructions used by the apparatus provided in the above-mentioned embodiment of the present application, which includes a program for executing any of the methods provided in the above-mentioned embodiment of the present application.

[0201] The computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drives (SSDs)), etc.

[0202] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0203] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0204] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0206] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A remote voice control method, characterized in that: The method includes: When the user successfully logs in to the local server through the terminal, a user voice command sent by the terminal is received, wherein when the user successfully logs in to the local server through the terminal, based on the fused feature vector and weight matrix obtained based on the face image and voice information collected by the user through the terminal, the value after the sum of the determined feature weights belongs to a preset value range, each weight in the weight matrix is ​​the average value of the weights in the weight matrix of multiple groups of fused feature vector pairs of the user, and the weight matrix of each group of fused feature vector pairs is the average distance between the first fused feature vector S1 and the second fused feature vector S2 in the fused feature vector pair and (1+e -|S1i-S2i| ), i is the element number; Performing voiceprint verification on the voice command, and when the verification passes, transmitting the voice command to the coordinator and the speaker module respectively, wherein the speaker module adopts distributed speakers deployed in multiple rooms; The coordinator controls the power switch of the smart home device, and the speaker module classifies and transmits the command according to the keywords of the smart home device contained in the voice command, so that the voice command is transmitted to the speaker corresponding to the smart home device corresponding to the keyword; The voice command is played through the speaker to the smart home device corresponding to the keyword, so that the smart home device executes the action corresponding to the voice command.

2. The method according to claim 1, characterized in that The method further includes: Receive facial images and voice information collected by users through terminals; Extracting facial features of the facial image and voiceprint features of the voice information; The voiceprint feature is fused with the facial feature, and based on the fusion result, it is determined whether the user can log in to the local server.

3. The method according to claim 2, characterized in that The voiceprint feature is integrated with the face feature, and based on the integrated result, it is determined whether the user can log in to the local server, specifically including: Constructing a fusion feature vector based on the vector of the facial feature and the vector of the voiceprint feature; Determining a weight matrix of the fused feature vector; Each characteristic weight in the weight matrix is ​​summed up to finally obtain a value. If the value belongs to a preset value range, it is determined that the user has successfully logged into the local server through the terminal.

4. The method according to claim 1, characterized in that: The method further includes: When the voice command cannot be executed by the corresponding smart home device, error command prompt information is fed back to the user terminal.

5. The method according to claim 1, characterized in that When the verification passes, the method further includes: If the voice command cannot be processed offline on the local server, the cloud server is called to process the voice command.

6. A remote voice control device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program.

7. A remote voice control system, characterized in that: The device comprises the device of claim 6, a coordinator and a speaker module respectively connected to the device, and at least one speaker connected to the speaker module, wherein the speaker module is deployed in multiple rooms using distributed speakers; wherein: The coordinator is used to receive the voice command sent by the device and control the power switch of the smart home device based on the voice command; The speaker module is used to determine the keywords of the smart home device contained in the voice command sent by the device, send the voice command to the speaker corresponding to the smart home device corresponding to the keyword, and play the voice command to the smart home device corresponding to the keyword through the speaker, so that the smart home device executes the action corresponding to the voice command.

8. The system according to claim 7, characterized in that The system also includes a smart home device arranged corresponding to each of the speakers.

9. A computer storage medium, characterized in that The computer storage medium stores computer executable instructions, and the computer executable instructions are used to enable the computer to execute the method according to any one of claims 1 to 5.

Images