Household appliance and voiceprint control method thereof, server, and readable storage medium

Abstract

The present invention discloses a method for voiceprint control of household appliances, comprising: receiving voice control information for voice control of household appliances; determining user information corresponding to the voice control information based on a preset voiceprint library, wherein the preset voiceprint library includes at least one voiceprint model, wherein the voiceprint model is obtained by self-training after voiceprint clustering of voice control information received within a period of time; determining operation habit information corresponding to the user information; and controlling the household appliances to operate according to the operation habit information. The present invention also discloses a household appliance, a server, and a readable storage medium, which solves the problem that the registration process in the prior art is very cumbersome, is particularly not conducive to voiceprint registration for the elderly and children, and cannot meet the voiceprint control needs of more users.

Description

Technical Field

[0001] The present invention relates to the field of household appliances, and in particular to household appliances and voiceprint control methods, servers, and readable storage media thereof. Background Art

[0002] With the development of speech recognition technology, voiceprint recognition is gradually applied to the control of home appliances. However, in order to realize the voiceprint control of home appliances, voiceprint registration must be performed on the home appliances first. The user needs to enter multiple predetermined registration voice information according to the registration prompt information, and then determine the user's registered voiceprint characteristics based on the registration voice information and the preset acoustic model. Finally, the corresponding relationship between the user's registration information (user name, etc.) and its registered voiceprint characteristics is established, and the user registration is completed. The registration process in the prior art is very cumbersome, which is particularly not conducive to voiceprint registration for the elderly and children. Summary of the invention

[0003] The embodiments of the present application provide a home appliance and a voiceprint control method, a server, and a readable storage medium to solve the problem that the registration process is very complicated, which is particularly inconvenient for the elderly and children to register voiceprints and cannot meet the voiceprint control needs of more users.

[0004] The present application provides a method for controlling a voiceprint of a household appliance, including:

[0005] Receiving voice control information for voice control of home appliances;

[0006] Determining user information corresponding to the voice control information based on a preset voiceprint library, wherein the preset voiceprint library includes at least one voiceprint model, and the voiceprint model is obtained by self-training after voiceprint clustering of voice control information received within a period of time;

[0007] Determining operation habit information corresponding to the user information;

[0008] Control the home appliance to operate according to the operation habit information.

[0009] Optionally,

[0010] The step of performing voiceprint clustering and self-training on the voice control information received within a period of time comprises:

[0011] Extracting the voiceprint features of the voice control information received in the first time period of the period;

[0012] Calculate the similarity between every two voice control information in the voiceprint features of all voice control information to obtain a similarity matrix;

[0013] Performing cluster analysis on all voice control signals according to the similarity matrix to obtain a cluster category for each voice control signal;

[0014] Register the voiceprint of the voice control information corresponding to the same cluster category to obtain the voiceprint template of the registered user;

[0015] The voice control information received in the second time period of the period is used to self-train the registered voiceprint template to obtain the voiceprint model.

[0016] Optionally, the step of extracting the voiceprint features of the voice control information received in the first time period of the period includes:

[0017] Extracting audio features of the voice control information received in the first time period of the period;

[0018] The extracted audio features are input into a preset classifier, and a neural network forward calculation is performed to obtain the voiceprint features of each voice control information; the preset classifier is a deep neural network formed by training based on a database of multiple audio samples.

[0019] Optionally, the step of registering the voiceprint of the voice control information corresponding to the same cluster category to obtain the voiceprint template of the registered user includes:

[0020] Determining whether the number of voice control information corresponding to the same cluster category is greater than or equal to a first preset number;

[0021] When the number of voice control information corresponding to the same cluster category is greater than or equal to a first preset number, the voiceprint features of the voice control information corresponding to the cluster category are averaged to obtain the cluster center of the cluster category;

[0022] According to the distance between the voiceprint feature of each voice control information and the class center, the voice control information within a preset range from the class center is screened out, and the voiceprint of the screened voice control information is registered to obtain the voiceprint template of the registered user.

[0023] Optionally, the voiceprint control method further includes:

[0024] When the preset time arrives or the number of newly received voice control information reaches a second preset number, the newly received voice control information is clustered and self-trained in combination with historical voiceprint clustering data to obtain a new voiceprint model.

[0025] Optionally, the operation habit information corresponding to the user information is obtained by analyzing all voice control information corresponding to the user information after determining the user information corresponding to the voice control information based on a preset voiceprint library, and is stored corresponding to the user information.

[0026] The embodiment of the present application also provides a voiceprint control method for household appliances, including:

[0027] receiving voice control information for voice controlling the home appliance;

[0028] The received voice control information is sent to the server, and the operating habit information returned by the server is received, and the operation is performed according to the operating habit information; the server performs voiceprint clustering on the voice control signal within a preset time period and self-trains to obtain a voiceprint model, and determines the user information corresponding to the voice control information based on the voiceprint model, and determines the operating habit information corresponding to the user information.

[0029] Optionally, after the step of receiving voice control information for voice controlling the home appliance, the method further comprises:

[0030] Determine whether a voiceprint model exists locally;

[0031] When a voiceprint model exists, the user information corresponding to the voice control information is determined according to the voiceprint model, and the operation habit information corresponding to the user information is determined, and the operation is performed according to the operation habit information.

[0032] An embodiment of the present application also provides a server, wherein the household appliance comprises a processor, a memory, and a voiceprint control program stored in the memory, and when the voiceprint control program is called and executed by the processor, the steps of the voiceprint control method as described in any of the above implementation schemes are implemented.

[0033] An embodiment of the present application also provides a household appliance, which includes a processor, a memory, and a voiceprint control program stored in the memory. When the voiceprint control program is called and executed by the processor, the steps of the voiceprint control method described in any of the above embodiments are implemented.

[0034] An embodiment of the present application also provides a computer storage medium, wherein the readable storage medium stores a computer program for performing voiceprint control on a household appliance, and the computer program is executed by a processor to implement the steps of the voiceprint control method described in any of the above implementation schemes.

[0035] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0036] 1. In the implementation mode of the present invention, the voiceprint model of the user is obtained by clustering the voice control information issued by the user to the home appliance within a period of time and then self-training, so as to be used for voiceprint recognition, thereby eliminating the tedious voiceprint registration process. In the process of the user's daily voice control of the home appliance, the voiceprint model is obtained to realize the voiceprint control of the home appliance.

[0037] 2. Through the voice control information received daily, the existing voiceprint model can be iterated multiple times, so that it can better match the voiceprint characteristics of the current user, achieve better and better voiceprint recognition effects, and enhance the user's voiceprint control experience.

[0038] 3. After the user information is recognized based on the voiceprint model, the user's operating habit information can also be formed according to the voice control information corresponding to the user information. Therefore, when the user's identity is recognized by the voiceprint, the user's personalized mode / preference mode can be obtained according to the user's current usage habits, and then the corresponding preference mode can be recommended for the current user, making the control of home appliances more intelligent and convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a diagram showing an example of the system architecture of a voiceprint control system for a household appliance according to an embodiment of the present invention;

[0040] Figure 2 It is a flow chart of an embodiment of a method for controlling a voiceprint of a household appliance of the present invention;

[0041] Figure 3 It is a schematic diagram of the process of obtaining a voiceprint model of a preset voiceprint library in the voiceprint control method of household appliances of the present invention;

[0042] Figure 4 yes Figure 1 Detailed flow chart of step S11;

[0043] Figure 5 It is a flow chart of another embodiment of the voiceprint control method for household appliances of the present invention. DETAILED DESCRIPTION

[0044] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] like Figure 1 As shown, Figure 1 1 is a schematic diagram of the system architecture of the voiceprint control system of the household appliance in the embodiment of the present invention. The voiceprint control system includes a household appliance 100, a voiceprint server 200 and a household appliance control terminal 300.

[0046] Among them, the household appliance 100 includes, for example, an air conditioner, a refrigerator, a television, a water dispenser, a fan, an air purifier, an electric rice cooker, a water heater, a speaker device, and the like. In addition to the functional components for realizing the corresponding functions, the household appliance also includes a communication module, and functional components such as a voice module. The communication module is used to realize the communication between the household appliance and the external device, such as a wifi module, a bluetooth module, an infrared module, a ZigBee module, and the like. The voice module is used to realize the sound collection in the surrounding environment, so that when the voice control instruction is collected and recognized, the household appliance executes the voice control instruction. Of course, the voice module may not be set, and the voice control instruction sent by the external device (such as the household appliance control terminal 300) is received through the communication module.

[0047] The home appliance control terminal 300 includes a mobile device such as a mobile phone, iPad, or a wearable device such as a smart watch. Through the control APP on the home appliance control terminal 300, it can be bound to the home appliance to control the operation of the home appliance. For example, the home appliance control terminal 300 includes functional components such as a communication module and a voice module. The communication module is used to communicate with the home appliance or server, and the voice module is used to collect sound information in the surrounding environment and recognize the voice control command, and then send the voice control command to the target home appliance to control the operation of the target home appliance.

[0048] The voiceprint server 200 is used to enable the user to register the voiceprint through the home appliance 100, or to register the voiceprint through the home appliance control terminal 300. The voiceprint server 200 is used for the voiceprint registration scheme proposed in the technical solution of the present invention. Of course, the voiceprint server 200 can also be used for traditional voiceprint registration. For example, the user starts the voiceprint registration through the home appliance 100, or starts the voiceprint registration through the home appliance control terminal 300, the home appliance 100 or the home appliance control terminal 300 sends a voiceprint registration request to the voiceprint server 200, and the voiceprint server 200 returns the registration prompt information according to the voiceprint registration request; the home appliance 100 or the home appliance control terminal 400 outputs the registration prompt information, and collects multiple voice information input by the user according to the registration prompt information, and then sends the collected voice information to the voiceprint server 200. The voiceprint server 200 extracts and analyzes the voice information, obtains the registered voiceprint features of the user, and thus forms a voiceprint template corresponding to the user, which is used for voiceprint recognition.

[0049] Furthermore, in addition to the voiceprint registration module for realizing the voiceprint registration function, the voiceprint server 200 may also include a voiceprint recognition module and a voiceprint deregistration module. The voiceprint recognition module is used to identify whether the user's voiceprint information is a registered user. If so, the recognition is successful and the home appliance executes the voice control command; otherwise, the recognition fails and the home appliance does not respond. When the user initiates a voiceprint deregistration request through the home appliance 100 or initiates a voiceprint deregistration request through the home appliance control terminal 300, the voiceprint deregistration module deletes the voiceprint template corresponding to the user.

[0050] like Figure 2 As shown, the present invention proposes a process step of an embodiment of a voiceprint control method for household appliances. The voiceprint control method of this embodiment includes the following steps:

[0051] S11, receiving voice control information for voice controlling a home appliance;

[0052] S12, determining user information corresponding to the voice control information based on a preset voiceprint library, wherein the preset voiceprint library includes at least one voiceprint model, and the voiceprint model is obtained by self-training after voiceprint clustering of voice control information received within a preset time period;

[0053] S13, determining the operation habit information corresponding to the user information;

[0054] S14, controlling the household appliance to operate according to the operation habit information.

[0055] Specifically, in step S12, a period of time can be a set default fixed value, or it can be flexibly adjusted according to the reception of voice control information. For example, in one embodiment, by setting a default fixed value, when the set time arrives, the received voice control information is clustered and then self-trained, so as to obtain the voiceprint model of the user. In another embodiment, when the amount of received voice control information reaches the amount of information required for voiceprint clustering, the received voice control information is clustered and then self-trained, so as to ensure the accurate acquisition of the voiceprint model and realize accurate voiceprint control of home appliances. By clustering the voice control information received within a period of time and then self-training, a voiceprint model required for voiceprint recognition can be formed. When the received voice control information is all issued by the same user, a voiceprint model is formed; when the received voice control information is issued by multiple users, multiple voiceprint models are formed.

[0056] Based on the preset voiceprint library, the received voice control information is subjected to voiceprint recognition to determine the user information corresponding to the voice control information, and then the operation habit information corresponding to the user information is determined, and the home appliance is operated according to the operation habit information. The operation habit information includes, for example, the control parameters of the home appliance, the mode preferred by the user, the operation time of the home appliance, etc. Taking the home appliance as an air conditioner, user A returns home at 7 pm and is used to turning on the air conditioner in the living room. The target temperature when turning on is 22°C, the wind speed is medium wind and sweeps up and down, and the target temperature is adjusted to 26°C after running for half an hour. User A turns on the air conditioner in the bedroom at 10 pm, and the target temperature when turning on is 26°C. After running for 1 hour, the operation mode is adjusted to sleep mode. The operation habit information is obtained by performing data analysis and annotation on the voice control information of the same user after performing voiceprint recognition on the voice control information to determine the corresponding user information.

[0057] In the embodiment of the present invention, during the user's daily voice control of home appliances, the voice control information within a period of time is clustered and then self-trained to obtain the user's voiceprint model, thereby realizing the user's voiceprint recognition and recommending the user's exclusive personalized mode. Therefore, the embodiment of the present invention eliminates the cumbersome voiceprint registration process, and during the user's daily voice control of home appliances, the voiceprint model is obtained, the voiceprint control of home appliances is realized, and the user's exclusive personalized mode is intelligently recommended, thereby meeting the user's personalized requirements and greatly improving the user's experience.

[0058] Furthermore, in the voiceprint control method of the above embodiment, obtaining the voiceprint model in the preset voiceprint library can be implemented on the server, for example Figure 1 The voiceprint server 200 in the embodiment of the present invention can also be implemented on the home appliance 100.

[0059] When the home appliance 100 or the home appliance control terminal 300 collects voice control information, the collected voice control information is sent to the server. The server performs voiceprint recognition on the voice control information based on the generated voiceprint model, and determines the user identity corresponding to the voice control information (for example, a registered user), and then returns the recognition result, so that the home appliance 100 operates according to the collected voice control information or the home appliance control terminal 300 controls the home appliance to operate according to the collected voice control information. Of course, after determining the user information corresponding to the voice control information, the server can also obtain the operation habit information corresponding to the user information, and make intelligent recommendations to the home appliance 100 or the home appliance control terminal 300.

[0060] In another embodiment, after the server obtains the voiceprint model, it can also send the obtained voiceprint model to the home appliance 100, so that the home appliance 100 can perform voiceprint recognition on the voice control instructions collected from the surrounding based on the voiceprint model, and execute the voice control instructions after the voiceprint recognition is successful; or send it to the home appliance control terminal 300, so that the home appliance control terminal 300 can perform voiceprint recognition on the collected voice control instructions based on the voiceprint model, and after the voiceprint recognition is successful, send the voice control instructions to the home appliance 100 to control the home appliance 100 to execute the voice control instructions. Furthermore, after the home appliance 100 and the home appliance control terminal 300 determine the user information corresponding to the voice control instruction based on the voiceprint model, they can also perform statistical analysis on the voice control instructions of the same user to obtain the user's habitual operation information, thereby realizing intelligent recommendation of the user's exclusive personalized mode, thereby meeting the user's personalized requirements and greatly improving the user's usage experience.

[0061] Furthermore, if Figure 3 As shown, in step S12 of the above embodiment, the step of performing self-training after voiceprint clustering on the voice control information received within a period of time includes:

[0062] S101, extracting the voiceprint features of the voice control information received in the first time period of the period;

[0063] S102, calculating the similarity between every two pieces of voice control information in the voiceprint features of all voice control information to obtain a similarity matrix;

[0064] S103, performing cluster analysis on all voice control signals according to the similarity matrix to obtain a cluster category of each voice control signal;

[0065] S104, registering the voiceprint of the voice control signal corresponding to the same cluster category to obtain the voiceprint template of the registered user;

[0066] S105, self-training the registered voiceprint template with the voice control information received in the second time period of the period to obtain the voiceprint model.

[0067] The above time period includes a first time period and a second time period, which can be divided according to the time when the voice control information is received, or according to the amount of voice control information received. The voice control information received in the first time period is used to perform voiceprint clustering and obtain the user's voiceprint template, and the voice control information received in the second time period is used to perform self-training on the obtained voiceprint template to obtain a voiceprint model.

[0068] In the process of forming the above-mentioned voiceprint model, the voiceprint features of each voice control information received in the first time period are first extracted, and then each two voice control information are cross-scored, that is, the voiceprint similarity of each two voice control information is calculated, so as to obtain a similarity matrix. For example, if N voice control information is received in the first time period, an N*N similarity matrix is ​​obtained. Then, based on the calculated similarity matrix, a cluster analysis is performed on each voice control information, for example, the voice control information is classified according to the calculated similarity, and the similarity of the same type of voice control information is higher than the similarity of other types of voice control information. Through cluster analysis, one or more cluster categories and the voice control information corresponding to each cluster category, that is, cluster data, will be obtained. With the cluster data, the voiceprint of the same type of voice control information can be registered to obtain the voiceprint template of the class (that is, the user).

[0069] With the voiceprint template, the registered voiceprint template is self-trained based on the voice control information received in the second time period, so as to obtain a voiceprint model. Specifically, the voice control information in the second time period is classified based on the registered voiceprint template, that is, the voice control information matching the voiceprint template is obtained, and then the matching voice control information is self-trained on the voiceprint template to obtain a voiceprint model.

[0070] The current voiceprint clustering is mainly aimed at segmenting and clustering a segment of audio with multiple people speaking, which is used to solve the problem of who is speaking and when in this segment of audio. In this embodiment, the voiceprint features of each voice control information are first extracted, and the clustering method based on the cross-similarity matrix is ​​adopted, which can well represent the similarity of the speakers between the voice control information, thereby ensuring the accuracy of clustering. Moreover, after clustering, the voiceprint template obtained by clustering is self-trained by continuing to receive voice control information, so that the voiceprint model is more compatible with the user's voiceprint characteristics, thereby improving the voiceprint recognition effect.

[0071] Furthermore, the above step S101 includes:

[0072] The audio features of the voice control information received in the first time period of the period are extracted; the extracted audio features are input into a preset classifier, and a neural network forward calculation is performed to obtain the voiceprint features of each voice control information.

[0073] The above-mentioned preset classifier is a deep neural network formed by training based on a database of multiple audio samples. For example, the audio sample is an audio database of 10,000 people, and a deep neural network is formed by training the audio database. After the audio feature extraction of the voice control information received in the first time period is performed, it is input into the preset classifier, and after the neural network completes the forward calculation, the voiceprint feature of the audio is output.

[0074] The embodiment of the present invention further improves the accuracy of clustering by using a voiceprint extraction model of a preset classifier and combining it with similarity calculation between voice control information.

[0075] Furthermore, the above step S104 includes:

[0076] Determining whether the number of voice control information corresponding to the same cluster category is greater than a first preset number;

[0077] When the number of voice control information corresponding to the same cluster category is greater than a first preset number, the voiceprint features of the voice control information corresponding to the cluster category are averaged to obtain the cluster center of the cluster category;

[0078] According to the cosine distance between the voiceprint features of each voice control information and the class center, the voice control information within a preset range from the class center is screened out, and the voiceprint of the screened voice control information is registered to obtain the voiceprint template of the registered user.

[0079] In this embodiment, by traversing each cluster category, the voice control information under each cluster category is judged, so as to screen out the cluster category that meets the voiceprint template registration conditions. Specifically, before voiceprint registration is performed on the voice control information of a cluster category, it is first determined whether the number of voice control information under the category is greater than or equal to a first preset number, such as n; when the number of voice control information under the category is greater than or equal to the first preset number n, the voice control information of the category is registered, otherwise the registration of the voice control information of the category is abandoned. By filtering the number of voice control information, the influence of clustering errors caused by small data volume on the final registered voiceprint template effect can be reduced, thereby ensuring the accuracy of the registered voiceprint template.

[0080] When the number of voice control information under the same cluster category is greater than or equal to the first preset number, the voice control information of this category is registered. Specifically, the voiceprint features of the voice control information corresponding to the cluster category are first averaged to obtain the class center of the cluster category, and the distance between the voiceprint features of each voice control information and the class center is calculated, such as the cosine distance. Then, based on the calculated distance, the voice control information within the preset distance range from the class center is filtered out, such as the voice control information whose distance from the class center is within the L value. Of course, the m voice control information closest to the class center can also be filtered out based on the calculated distance. Finally, the filtered voice control information is voiceprint registered to obtain the voiceprint template of the registered user.

[0081] The embodiment of the present invention further ensures the accuracy of the registered voiceprint template by screening the voice control information under the cluster category.

[0082] Furthermore, the voiceprint control method of the above embodiment also includes:

[0083] When the preset time arrives or the number of newly received voice control information reaches a second preset number, the newly received voice control information is clustered and self-trained in combination with historical voiceprint clustering data to obtain a new voiceprint model.

[0084] In this embodiment, the existing voiceprint library can be iteratively upgraded. As the user's daily voice control data increases, multiple iterations of the voiceprint library can better match the current user's voiceprint characteristics and achieve better and better voiceprint recognition effects, that is, the more it is used, the more accurate it is, and the better it is used.

[0085] Specifically, by setting a cycle or setting the number of voice control information, when the preset time arrives or the number of newly received voice control information reaches a second preset number, the iterative upgrade process is entered. First, the newly received voice control information is clustered to obtain the current clustering data. The voiceprint clustering step is implemented with reference to steps S101-103 in the previous embodiment. Then, the voice control information of the clustering category is registered with the historical voiceprint clustering data and the current clustering data, the voiceprint template of the registered user is obtained, and the registered voiceprint template is self-trained to obtain a new voiceprint model.

[0086] The embodiments of the present invention achieve better and better voiceprint recognition effects by iterating the existing voiceprint model multiple times, thereby improving the user's voiceprint control experience.

[0087] Furthermore, when performing voiceprint recognition based on a preset voiceprint library, the embodiment of the present invention can not only identify the basic voiceprint information of the user, but also identify the additional voiceprint information of the user, such as gender (male / female), approximate age (child / adult / elderly), approximate emotions (normal, sad, angry, happy, etc.), etc. For children, the voice will change as they grow up, especially for children in the voice change period, a single voiceprint template cannot meet their voiceprint control requirements. Therefore, through the iterative extraction and iterative training of daily voice control data by the voiceprint clustering and self-training method of this embodiment, the changes in the iterative model age recognition results can be continuously recorded, and through the retrospective comparison of the historical model recognition age data, it can be more accurately determined whether the current user is a child, thereby outputting an age judgment result with a higher confidence level.

[0088] The iterative continuity of the voiceprint model can be updated as the child grows up. When the child grows up to 20 years old, the voiceprint model will also be updated to form a voiceprint additional information recognition result (gender, age, etc.) that matches the actual situation. The age recognition and tracing records of children in the voice change period in the embodiment of the present invention are as follows (assuming that the actual child's gender = male, age = 15 years old):

[0089]

[0090] It can be seen from the above table that with the increase in the number of iterations, the historical age recognition ratio and confidence of the current voiceprint model are gradually improved, that is, the accuracy of identifying the current child's identity is also gradually improved, thereby improving the accuracy of voiceprint recognition.

[0091] Furthermore, if Figure 4 As shown, a detailed process step of step S11 in the above embodiment is proposed. The above step S11 includes:

[0092] S111, receiving a voice control instruction sent by the home appliance or the home appliance control terminal;

[0093] S112, extracting effective voice information from the voice control instruction and storing it.

[0094] Specifically, when the home appliance or the home appliance control terminal collects the surrounding voice information, it first determines whether it is the control information for the home appliance. If so, the voice control information is uploaded to the server. The server receives the voice control information sent by the home appliance or the home appliance control terminal, extracts valid voice information from the voice control information, such as performing denoising on the voice control information.

[0095] Furthermore, when the user controls the home appliance or the home appliance control terminal, the home appliance or the home appliance control terminal may also use other feature collection components to collect the user's feature data, such as face recognition, body recognition, etc., and upload the collected feature data to the server together with the voice control information. After the server extracts the valid voice information from the voice control information, it stores it accordingly according to the feature data.

[0096] Furthermore, the operation habit information corresponding to the above user information is obtained by analyzing all voice control information corresponding to the user information after determining the user information corresponding to the voice control information based on the preset voiceprint library, and is stored corresponding to the user information. That is, after identifying the user information corresponding to the voice control information based on the preset voiceprint library, the voice control information can be stored corresponding to the user information. As the user's daily voice control data increases, the voice control information corresponding to the user information can be analyzed to form the user's portrait information, and based on the portrait information, the operation habit information corresponding to the user can be obtained to achieve intelligent recommendation of user-specific modes.

[0097] Specifically, during the daily use of home appliances by users, the server can collect voice control information from users during voice control, and conduct statistical analysis on the collected voice control information to gradually form complete user portrait information. User portrait information, also known as user role, is a labeled user model abstracted from information such as user social attributes, living habits, and consumption behaviors. Constructing a user portrait is to "label" the user, and the label is a highly refined feature identifier obtained by the server through analysis of user behavior data. By analyzing and labeling the voice control data of the same user, the user's usage habits for home appliances can be obtained, thereby forming user portrait information. Due to the uniqueness of voiceprint information, the behavior data of a single or multiple users in a household during long-term use will also be intelligently recorded in the cloud server by home appliances or home appliance control terminals. The usage habits of a "certain" user form a one-to-one correspondence with the voiceprint information of the user.

[0098] In this embodiment, through the formation of user portrait information, when the user information is recognized by voiceprint, the recorded personalized mode / preference mode can be called according to the user's current usage habits, and the corresponding preference mode can be recommended for the current user, thereby making the control of home appliances more intelligent and convenient.

[0099] Furthermore, if Figure 5 As shown, the process steps of another embodiment of the voiceprint control method of the home appliance of the present invention are proposed. The voiceprint control method of the home appliance of this embodiment can be run on the home appliance or the home appliance control terminal, and includes the following steps:

[0100] S21, receiving voice control information for voice controlling the home appliance;

[0101] S22, sending the received voice control information to the server, and receiving the operating habit information returned by the server, and operating according to the operating habit information; the server performs voiceprint clustering on the voice control signals within a preset time period and self-trains to obtain a voiceprint model, and determines the user information corresponding to the voice control information based on the voiceprint model, and determines the operating habit information corresponding to the user information.

[0102] Specifically, the home appliance or the home appliance control terminal collects voice information sent by surrounding users and determines whether the voice information is voice control information. In one embodiment, the user can first wake up the voice function of the home appliance or the home appliance control terminal through the wake-up word, and then the home appliance or the home appliance control terminal whose voice function is awakened collects the surrounding voice information and determines whether the voice information is voice control information. Then, the home appliance or the home appliance control terminal sends the voice control information to the server, so that the server determines the user information corresponding to the voice control information based on the preset voiceprint library, and determines the operation habit information corresponding to the user information, so that the home appliance operates according to the operation habit information.

[0103] In one embodiment, the server may also send the obtained voiceprint model to the home appliance or to the home appliance control terminal. After the home appliance or the home appliance control terminal receives the voiceprint model sent by the server, the voiceprint model is stored. Therefore, when the home appliance receives the voice control information, it first determines whether there is a voiceprint model locally. When there is a voiceprint model, it can retrieve the locally stored voiceprint model, identify the user, and execute the voice control information. Or, the user information corresponding to the voice control information is determined according to the voiceprint model, and the operation habit information corresponding to the user information is determined, and the operation is performed according to the operation habit information. When the home appliance control terminal receives the voice control information, it can call the stored voiceprint model, identify the user, and then send the received voice control information to the home appliance to control the home appliance to operate according to the target control parameters in the voice control information. Or, the user information corresponding to the voice control information is determined according to the voiceprint model, and the operation habit information corresponding to the user information is determined, and the home appliance is controlled to operate according to the operation habit information.

[0104] The server may also send the operation habit information corresponding to the user information to the home appliance or the home appliance control terminal. Of course, the home appliance or the home appliance control terminal may also analyze the received voice control information based on the voiceprint model sent by the server during the user's daily use to form the operation habit information corresponding to the user information.

[0105] Furthermore, after obtaining the voiceprint model, the above-mentioned home appliance or home appliance control terminal can also cluster the voiceprints according to the voice control information of the home appliance, and obtain a new registered voiceprint template in combination with the historical clustering data. Then, the newly registered voiceprint template is self-trained through the newly collected voice control information to obtain a new voiceprint model, thereby realizing the iterative upgrade of the voiceprint model. The new voiceprint model that is iteratively upgraded improves the accuracy of voiceprint recognition, especially for children in the voice change period. With the increase of daily data and the increase of iteration times, the recognition accuracy gradually improves.

[0106] In the embodiment of the present invention, a new voiceprint model is obtained by clustering the voiceprints of new voice control information of home appliances and then self-training, which not only avoids the tedious and complicated voiceprint registration process but also improves the accuracy of voiceprint recognition.

[0107] Furthermore, the present invention proposes a household appliance / household appliance control terminal and a server for running the voiceprint control method of the above embodiment. The household appliance / household appliance control terminal and the server are both provided with a control circuit board for voiceprint control, and the control circuit board is provided with a processor, a memory and a voiceprint control program stored in the memory, and when the voiceprint control program is called and executed by the processor, the voiceprint control method of the above embodiment is implemented.

[0108] The memory may include a storage module integrated on the control circuit board, or may be a storage device electrically connected to the control circuit board.

[0109] Furthermore, as a computer-readable storage medium, an operating system and a voiceprint control program may be stored. Of course, the memory may also be provided with various other application programs, such as a control program for home appliances. After the computer storage medium is connected to the processor, the processor may be used to call the voiceprint control program stored in the storage medium to implement the steps of the voiceprint control method in the above embodiments of the present invention.

[0110] Furthermore, the present invention also proposes a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the voiceprint control method in the above embodiments of the present invention are implemented. It should be noted that the computer program product can be a physical product including a hardware structure, or it can only include a virtual carrier of a computer program, such as a cloud server.

[0111] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0116] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0117] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A voiceprint control method for household appliances, characterized in that: The voiceprint control method of the household appliance comprises: Receiving voice control information for voice control of home appliances; Determine the user information corresponding to the voice control information based on a preset voiceprint library, wherein the preset voiceprint library includes at least one voiceprint model, and the voiceprint model is obtained by self-training after voiceprint clustering of voice control information received within a period of time during the user's daily voice control of the home appliance; Determining operation habit information corresponding to the user information; Controlling the home appliance to operate according to the operation habit information; The step of clustering voiceprints and then self-training the voice control information received during a period of time during the user's daily voice control of the home appliance includes: Extracting the voiceprint features of the voice control information received in the first time period of the period; Calculate the similarity between every two voice control information in the voiceprint features of all voice control information to obtain a similarity matrix; Performing cluster analysis on all voice control signals according to the similarity matrix to obtain a cluster category for each voice control signal; Register the voiceprint of the voice control information corresponding to the same cluster category to obtain the voiceprint template of the registered user; The voice control information received in the second time period of the period is used to self-train the registered voiceprint template to obtain the voiceprint model.

2. The voiceprint control method of household appliances according to claim 1, characterized in that: The step of extracting the voiceprint features of the voice control information received in the first time period of the period includes: Extracting audio features of the voice control information received in the first time period of the period; The extracted audio features are input into a preset classifier, and a neural network forward calculation is performed to obtain the voiceprint features of each voice control information; the preset classifier is a deep neural network formed by training based on a database of multiple audio samples.

3. The voiceprint control method of household appliances according to claim 1, characterized in that: The step of registering the voiceprint of the voice control information corresponding to the same cluster category to obtain the voiceprint template of the registered user includes: Determining whether the number of voice control information corresponding to the same cluster category is greater than or equal to a first preset number; When the number of voice control information corresponding to the same cluster category is greater than or equal to a first preset number, the voiceprint features of the voice control information corresponding to the cluster category are averaged to obtain the cluster center of the cluster category; According to the distance between the voiceprint feature of each voice control information and the class center, the voice control information within a preset range from the class center is screened out, and the voiceprint of the screened voice control information is registered to obtain the voiceprint template of the registered user.

4. The voiceprint control method for household appliances according to any one of claims 1 to 3, characterized in that: The voiceprint control method further includes: When the preset time arrives or the number of newly received voice control information reaches a second preset number, the newly received voice control information is clustered and self-trained in combination with historical voiceprint clustering data to obtain a new voiceprint model.

5. The voiceprint control method of household appliances according to claim 1, characterized in that: The operation habit information corresponding to the user information is obtained by analyzing all voice control information corresponding to the user information after determining the user information corresponding to the voice control information based on a preset voiceprint library, and is stored corresponding to the user information.

6. A voiceprint control method for household appliances, characterized in that: The voiceprint control method of the household appliance comprises: receiving voice control information for voice controlling the home appliance; The received voice control information is sent to the server, and the operation habit information returned by the server is received, and the operation is performed according to the operation habit information; the server clusters the voice control signals within a preset time period during the user's daily voice control of the home appliance and then self-trains to obtain a voiceprint model, and determines the user information corresponding to the voice control information based on the voiceprint model, and determines the operation habit information corresponding to the user information. The server extracts the voiceprint features of the voice control information received in the first time period within the preset time period; calculates the similarity between every two voice control information in the voiceprint features of all voice control information to obtain a similarity matrix; performs cluster analysis on all voice control signals according to the similarity matrix to obtain a clustering category for each voice control signal; performs voiceprint registration on the voice control information corresponding to the same clustering category to obtain a voiceprint template of the registered user; and performs self-training on the registered voiceprint template with the voice control information received in the second time period within the preset time period to obtain the voiceprint model.

7. The voiceprint control method of household appliances according to claim 6, characterized in that: After the step of receiving the voice control information for voice controlling the home appliance, the method further comprises: Determine whether a voiceprint model exists locally; When a voiceprint model exists, the user information corresponding to the voice control information is determined according to the voiceprint model, and the operation habit information corresponding to the user information is determined, and the operation is performed according to the operation habit information.

8. A server, characterized in that: The invention comprises a processor, a memory and a voiceprint control program stored in the memory, wherein the voiceprint control program, when called and executed by the processor, implements the steps of the voiceprint control method according to any one of claims 1 to 5.

9. A household appliance, characterized in that: The household appliance comprises a processor, a memory and a voiceprint control program stored in the memory, and the voiceprint control program implements the steps of the voiceprint control method according to claim 6 or 7 when called and executed by the processor.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program for performing voiceprint control on a household appliance, and the computer program is executed by a processor to implement the steps of the voiceprint control method according to any one of claims 1 to 7.

Images