Processing method and device based on smart home system, equipment, medium and product

Abstract

The embodiment of the invention provides a processing method and device based on a smart home system, equipment, a medium and a product. The method comprises the following steps: acquiring a pre-trained network model for a specified user; acquiring real-time voice data and real-time context information of the specified user; performing feature extraction on the real-time voice data to obtain real-time voice features; and calling the network model, and generating a personalized response strategy according to the real-time voice features and the real-time context information, so as to control the smart home system to respond according to the personalized response strategy. According to the embodiment of the invention, personalized voice recognition and response of the smart home system for the specified user are realized, so that the smart home system can be more suitable for the specified user such as the old, and more accurate and convenient smart home service is provided for the specified user.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to processing methods, devices, equipment, media and products based on smart home systems. Background Art

[0002] With the development of smart home technology, more and more smart home devices have entered family life, bringing great convenience to users, and users' requirements for smart home devices are also getting higher and higher.

[0003] In the existing technology, smart home systems usually adopt a fixed mode for all users, but there will be usage problems for some special groups of users under this fixed mode. For example, for elderly users, because their language habits and preferences are different from those of young people, it is difficult for the smart home system to accurately understand the instructions and needs of the elderly, resulting in difficulties for elderly users to use the smart home system. Summary of the invention

[0004] In view of the above problems, a processing method, device, equipment, medium and product based on a smart home system are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A processing method based on a smart home system, the method comprising:

[0006] Get a pre-trained network model for a specified user;

[0007] Acquire real-time voice data and real-time context information of the specified user;

[0008] Extracting features from the real-time speech data to obtain real-time speech features;

[0009] The network model is called to generate a personalized response strategy according to the real-time voice features and the real-time context information, so as to control the smart home system to respond according to the personalized response strategy.

[0010] Optionally, the calling of the network model to generate a personalized response strategy according to the real-time voice features and the real-time context information includes:

[0011] Calling the network model to generate a real-time speech recognition result according to the real-time speech feature;

[0012] The behavior pattern of the designated user pre-learned by the network model and the real-time context information are combined to generate a personalized response strategy for the real-time speech recognition result.

[0013] Optionally, after obtaining the pre-trained network model for the specified user, the method further includes:

[0014] Obtaining historical interaction data of the specified user;

[0015] According to the historical interaction data, the network model is individually adjusted so that the network model learns the behavior pattern of the designated user.

[0016] Optionally, the generating a personalized response strategy for the real-time speech recognition result by combining the behavior pattern of the designated user and the real-time context information pre-learned by the network model includes:

[0017] A personalized response strategy for the real-time speech recognition result is generated by combining the behavior pattern of the designated user pre-learned by the network model, the real-time context information, and the personalized needs of the designated user set for the network model.

[0018] Optionally, after obtaining the pre-trained network model for the specified user, the method further includes:

[0019] Providing a personalized setting option to the user, and obtaining the personalized needs of the designated user in response to the user's operation on the personalized setting option;

[0020] According to the personalized needs, the network model is personalized adjusted so that the personalized needs of the designated user are set for the network model.

[0021] Optionally, obtaining a pre-trained network model for a specified user includes:

[0022] Get sample data collected for a specified user;

[0023] Annotating the sample data to obtain an annotation result of the sample data;

[0024] The preset network model is trained in combination with the sample data and the annotation result to obtain a network model for a specified user.

[0025] Optionally, the sample data includes sample voice data, and after acquiring the sample data collected for the designated user, the method further includes:

[0026] The sample speech data is subjected to noise reduction processing and / or speech enhancement processing.

[0027] Optionally, the designated user is an elderly user.

[0028] A processing device based on a smart home system, the device comprising:

[0029] A network model acquisition module is used to obtain a pre-trained network model for a specified user;

[0030] A real-time data acquisition module, used to acquire the real-time voice data and real-time context information of the specified user;

[0031] A real-time feature extraction module, used to extract features from the real-time voice data to obtain real-time voice features;

[0032] The personalized response strategy generation and response module is used to call the network model and generate a personalized response strategy according to the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy.

[0033] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0035] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described above.

[0036] The embodiments of the present invention have the following advantages:

[0037] In an embodiment of the present invention, by obtaining a pre-trained network model for a designated user, obtaining real-time voice data and real-time context information of the designated user, performing feature extraction on the real-time voice data to obtain real-time voice features, calling the network model, and generating a personalized response strategy based on the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy, the smart home system is implemented to perform personalized voice recognition and response for the designated user, so that the smart home system can be more suitable for designated users such as the elderly, and provide more accurate and convenient smart home services for the designated users. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0039] Figure 1is a flowchart of a processing method based on a smart home system provided by some embodiments of the present invention;

[0040] Figure 2 is a flowchart of another processing method based on a smart home system provided by some embodiments of the present invention;

[0041] Figure 3 is a flowchart of another processing method based on a smart home system provided by some embodiments of the present invention;

[0042] Figure 4 is a flowchart of another processing method based on a smart home system provided by some embodiments of the present invention;

[0043] Figure 5 It is a structural block diagram of another processing device based on the smart home system provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Reference Figure 1 , shows a flowchart of a processing method based on a smart home system provided by some embodiments of the present invention, which may specifically include the following steps:

[0046] Step 101: Obtain a pre-trained network model for a specified user.

[0047] In order to enable the smart home system to perform personalized voice recognition and response for designated users and make the smart home system more adaptable to designated users such as the elderly, a network model for designated users can be pre-trained, such as the designated user can be an elderly user.

[0048] In some embodiments of the present invention, obtaining a pre-trained network model for a specified user includes: obtaining sample data collected for the specified user; annotating the sample data to obtain an annotation result of the sample data; and training a preset network model in combination with the sample data and the annotation result to obtain a network model for the specified user.

[0049] In practical applications, sample data collected for a specified user can be obtained, and the sample data can include sample voice data. For example, in places where the elderly often move, such as bedrooms, living rooms, etc., intelligent voice collection devices, such as smart speakers with microphones, are installed. Through these devices, the daily voices of the elderly can be continuously collected, including conversations with family members, instructions for smart home devices, questions asked, etc. For example, in a month's time, approximately 10,000 voice data are collected from the families of 100 elderly people. Another example is to cooperate with the community to organize the elderly to participate in voice collection activities, set up a special collection area, and let the elderly answer some specific questions or have simple conversations. For example, ask the elderly about the weather today, whether to turn on the TV, etc., and 2,000 additional voice data are collected. Another example is to cooperate with institutions such as nursing homes to collect the voice data of the elderly in a collective living environment to increase the diversity of data. Suppose 3,000 voice data are collected from 5 nursing homes.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, user voice information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0051] After obtaining the sample data, a professional data annotation team can be formed to annotate the collected voice data. The annotation content includes the text content of the voice, semantic understanding results, emotional tendencies, etc. For example, for a voice "Turn up the TV volume a bit", the text content is annotated as "Turn up the TV volume a bit", the semantic understanding result is "Adjust the TV volume to increase", and the emotional tendency is neutral.

[0052] In some examples, key annotations are made for the common dialects, accents, and special expressions of the elderly. For example, for the voice of an elderly person with a local accent "Make the light brighter", the same semantic understanding result as the standard expression "Turn up the light a bit" is annotated.

[0053] In the embodiments of the present invention, the voice data of the elderly are collected through multiple channels and are specially annotated and preprocessed to adapt to the possible characteristics of the elderly, such as accents, speech rate changes, special expressions, etc., and to improve the adaptability of the model to the voices of the elderly.

[0054] After obtaining the sample data and annotation results, the preset network model can be trained by combining the sample data and annotation results to obtain a network model for the specified user.

[0055] In some embodiments of the present invention, the sample data includes sample voice data, and after acquiring the sample data collected for the designated user, the following further includes:

[0056] The sample speech data is subjected to noise reduction processing and / or speech enhancement processing.

[0057] In some examples, after obtaining the sample voice data, the sample voice data can be subjected to noise reduction processing to remove environmental noise and interference, such as using audio processing software to reduce background noise by 30% to improve voice clarity. In some examples, the sample voice data can also be subjected to voice enhancement processing to highlight the voice characteristics of the elderly, such as by adjusting the frequency range of the audio to make the voice of the elderly more prominent.

[0058] In some examples, after obtaining the sample speech data, the speech signal can be converted into a digital feature vector to obtain the sample speech features, such as using Mel Frequency Cepstrum Coefficient (MFCC) to extract the speech features. In some examples, a piece of speech data can be converted into a feature vector with a length of 128.

[0059] In some examples, the network model may be a deep Q network model, and a deep neural network consisting of multiple fully connected layers may be constructed as a Q network. For example, three hidden layers are used, each hidden layer contains 256 neurons, the activation function is ReLU (Linear Rectification Function), and the last layer outputs the Q value corresponding to each action, with a dimension of 20.

[0060] For the network model, state representation: the preprocessed speech feature vector is combined with the context information as the state representation of the network model (deep Q network). The context information can include time, location, previous voice command, etc. For example, when the elderly issue a voice command in the bedroom at 8 pm, the state can be represented as the feature vector of the current speech plus the information that the time is 8 pm and the location is the bedroom. Assume that the dimension of the state vector is 256, of which the speech feature vector occupies 128 dimensions and the context information occupies 128 dimensions.

[0061] For the network model, action space definition: Based on the possible semantic understanding results, define the action space of the network model (deep Q network). For example, actions can include operating instructions for smart home devices such as turning on the TV, turning off the lights, adjusting the temperature, as well as answering questions, providing suggestions, etc.

[0062] In the embodiment of the present invention, a unique state representation method is designed according to the specific needs and habits of designated users such as the elderly, and the speech features of designated users such as the elderly are converted into state vectors suitable for processing by the network model (deep Q network). At the same time, the action space is carefully defined to make it more in line with the voice instructions and demand categories of designated users such as the elderly.

[0063] During the model training process, an experience replay buffer can be created to store the interaction experience between the model and the specified user. Each time the model receives a voice sample and makes an action, the current state, action, reward, and next state are stored in the experience replay buffer. Assume that the size of the experience replay buffer is 10,000, and a batch of experience samples are randomly drawn from the experience replay buffer to update the parameters of the network model (deep Q network), such as drawing 64 samples for training each time.

[0064] In some examples, the current network model (deep Q network) is copied as the target network model (deep Q network), such as copying the parameters of the network model (deep Q network) to the target network model (deep Q network) every certain number of training steps, such as updating once every 100 steps.

[0065] In some examples, mean squared error can be used as a loss function to measure the difference between the Q value estimated by the deep Q network and the target Q value. For example, for a sample, the square of the difference between the current Q value and the target Q value is calculated, and then the losses of all samples are summed and averaged.

[0066] In some examples, the Adam optimization algorithm is used to optimize the parameters of the network model (deep Q network), and the learning rate is adjusted to 0.001. By continuously reducing the loss function, the network's estimation is made more accurate.

[0067] In an embodiment of the present invention, through continuous learning and optimization mechanisms, the model can continuously learn new voice data and behavior patterns of designated users such as the elderly in actual applications, and continuously optimize its own performance through mechanisms such as experience playback and regular updating of network models (deep Q networks), providing more accurate and considerate services for designated users.

[0068] The following combination Figure 2 The following is an example of the model training process:

[0069] 1. Data collection: Collect voice samples of the corresponding elderly groups through mobile phones, smart voice collection devices in the room, smart speakers, etc.;

[0070] 2. Annotate the collected voice data: Annotate in three directions, including text content, semantic understanding, and emotional tendency;

[0071] 3. Build the model: determine the structure and parameters of the deep Q network, such as the number of network layers, number of neurons, activation function, etc.;

[0072] 4. Define the action space: Determine the possible semantic understanding results as different actions according to the elderly groups;

[0073] 5. Model training phase: Use the training set to train the deep Q network and optimize the model parameters through experience replay and target Q network update mechanism;

[0074] 6. Personalized adjustment: Record the voice interaction history and feedback information of the elderly group, and continuously adjust the model's state identification and action space according to the elderly's language habits and preferences.

[0075] Step 102: Acquire real-time voice data and real-time context information of the designated user.

[0076] When a designated user needs to interact with the smart home system, such as an elderly person asking about today's weather or controlling the opening of curtains, the real-time voice data collected by the smart home device can be obtained, and the real-time context information of the real-time voice data, such as time, location, and the previous voice command, can be obtained.

[0077] Step 103: extract features from the real-time speech data to obtain real-time speech features.

[0078] After obtaining real-time speech data, the speech signal can be converted into a digital feature vector to obtain real-time speech features, which can then be used for subsequent speech recognition, such as using Mel-frequency cepstral coefficients to extract speech features. For example, a piece of speech data can be converted into a feature vector of length 128.

[0079] In some examples, before extracting voice features, real-time voice data can be processed for noise reduction to remove environmental noise and interference, such as using audio processing software to reduce background noise by 30% to improve voice clarity. In some examples, real-time voice data can also be processed for voice enhancement to highlight the voice features of the elderly, such as by adjusting the frequency range of the audio to make the voice of the elderly more prominent.

[0080] Step 104 , calling the network model, generating a personalized response strategy according to the real-time voice features and the real-time context information, so as to control the smart home system to respond according to the personalized response strategy.

[0081] After obtaining real-time speech features and real-time context information, the pre-trained network model can be used for speech recognition and generate corresponding personalized response strategies. Then, the personalized response strategy can be used to control the smart home system to respond, such as voice broadcasting of weather messages and controlling curtains to close.

[0082] In some examples, an input state can be generated based on real-time speech features and real-time context information. After obtaining the input state, the network model performs speech recognition based on the input state, and selects a corresponding action from a predefined action space based on the speech recognition result as a personalized response strategy.

[0083] For example, when an elderly person issues a voice command in the bedroom at 8 pm, the input state can be represented as the feature vector of the current voice plus the context information that the time is 8 pm and the location is the bedroom. Assuming that the dimension of the state vector is 256, the voice feature vector occupies 128 dimensions and the context information occupies 128 dimensions.

[0084] In the embodiment of the present invention, a unique state representation method is designed according to the specific needs and habits of designated users such as the elderly, and the speech features of designated users such as the elderly are converted into state vectors suitable for processing by the network model (deep Q network). At the same time, the action space is carefully defined to make it more in line with the voice instructions and demand categories of designated users such as the elderly.

[0085] In some embodiments of the present invention, after obtaining a pre-trained network model for a specified user, it also includes: obtaining historical interaction data of the specified user; and personalizing the network model based on the historical interaction data so that the network model learns the behavior pattern of the specified user.

[0086] In some embodiments of the present invention, the calling of the network model to generate a personalized response strategy according to the real-time voice features and the real-time context information includes:

[0087] The network model is called to generate a real-time speech recognition result according to the real-time speech feature; and a personalized response strategy for the real-time speech recognition result is generated by combining the behavior pattern of the designated user and the real-time context information pre-learned by the network model.

[0088] During the use of the model, the historical interaction data of the specified user is continuously recorded, such as the voice commands and operating habits frequently used by the elderly at different times and in different scenarios. Then, the parameters and behaviors of the model can be dynamically adjusted according to the historical interaction data, and the network model learns the behavior patterns of the specified user. Since the behavior patterns of the specified user are learned in advance, the network model can combine the pre-learned behavior patterns when performing voice recognition and response to make personalized responses.

[0089] For example, an elderly person often asks about the weather and opens the curtains in the morning. The model will learn this behavior pattern and dynamically adjust the model's parameters and behavior based on the elderly person's behavior changes. If the elderly person's behavior pattern changes, such as starting to ask about the weather at night, the model will adjust its semantic understanding strategy for the elderly person accordingly.

[0090] In some embodiments of the present invention, after obtaining a pre-trained network model for a specified user, it also includes: providing a personalized setting option to the user, and obtaining the personalized needs of the specified user in response to the user's operation of the personalized setting option; and according to the personalized needs, personalizing the network model to set the personalized needs of the specified user for the network model.

[0091] In some embodiments of the present invention, the method of generating a personalized response strategy for the real-time speech recognition result by combining the behavior pattern of the designated user and the real-time context information pre-learned by the network model includes: generating a personalized response strategy for the real-time speech recognition result by combining the behavior pattern of the designated user and the real-time context information pre-learned by the network model, and the personalized needs of the designated user set for the network model.

[0092] In practical applications, personalized setting options can be provided. By selecting the personalized setting options, the personalized needs of the specified user can be obtained, and the behavior of the model can be adjusted according to the needs and preferences of the specified user. Since the personalized needs of the specified user are learned in advance, the network model can combine the pre-learned personalized needs to make personalized responses when performing speech recognition and response.

[0093] For example, the elderly can choose different voice response styles, such as friendly, formal, etc., or adjust the speed and volume of the response. Another example is that family members or caregivers can be allowed to make personalized settings for the elderly, such as setting reminders to take medicine on time, etc. The model will provide personalized services based on these settings.

[0094] In an embodiment of the present invention, by obtaining a pre-trained network model for a designated user, obtaining real-time voice data and real-time context information of the designated user, performing feature extraction on the real-time voice data to obtain real-time voice features, calling the network model, and generating a personalized response strategy based on the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy, the smart home system is implemented to perform personalized voice recognition and response for the designated user, so that the smart home system can be more suitable for designated users such as the elderly, and provide more accurate and convenient smart home services for the designated users.

[0095] Reference Figure 3 , shows a flowchart of another processing method based on a smart home system provided by some embodiments of the present invention, which may specifically include the following steps:

[0096] Step 301, obtaining a pre-trained network model for a specified user.

[0097] In order to enable the smart home system to perform personalized voice recognition and response for designated users and make the smart home system more adaptable to designated users such as the elderly, a network model for designated users can be pre-trained, such as the designated user can be an elderly user.

[0098] Step 302: Acquire historical interaction data of the designated user.

[0099] Step 303: Perform personalized adjustment on the network model according to the historical interaction data, so that the network model can learn the behavior pattern of the designated user.

[0100] During the use of the model, the historical interaction data of the specified user is continuously recorded, such as the voice commands and operating habits frequently used by the elderly at different times and in different scenarios. Then, the parameters and behaviors of the model can be dynamically adjusted according to the historical interaction data, and the network model learns the behavior patterns of the specified user. Since the behavior patterns of the specified user are learned in advance, the network model can combine the pre-learned behavior patterns when performing voice recognition and response to make personalized responses.

[0101] For example, an elderly person often asks about the weather and opens the curtains in the morning. The model will learn this behavior pattern and dynamically adjust the model's parameters and behavior based on the elderly person's behavior changes. If the elderly person's behavior pattern changes, such as starting to ask about the weather at night, the model will adjust its semantic understanding strategy for the elderly person accordingly.

[0102] Step 304: Acquire the real-time voice data and real-time context information of the designated user.

[0103] When a designated user needs to interact with the smart home system, such as an elderly person asking about today's weather or controlling the opening of curtains, the real-time voice data collected by the smart home device can be obtained, and the real-time context information of the real-time voice data, such as time, location, and the previous voice command, can be obtained.

[0104] Step 305: extract features from the real-time speech data to obtain real-time speech features.

[0105] After obtaining real-time speech data, the speech signal can be converted into a digital feature vector to obtain real-time speech features, which can then be used for subsequent speech recognition, such as using Mel-frequency cepstral coefficients to extract speech features. For example, a piece of speech data can be converted into a feature vector of length 128.

[0106] In some examples, before extracting voice features, real-time voice data can be processed for noise reduction to remove environmental noise and interference, such as using audio processing software to reduce background noise by 30% to improve voice clarity. In some examples, real-time voice data can also be processed for voice enhancement to highlight the voice features of the elderly, such as by adjusting the frequency range of the audio to make the voice of the elderly more prominent.

[0107] Step 306, calling the network model to generate a real-time speech recognition result according to the real-time speech feature.

[0108] Step 307, combining the behavior pattern of the designated user pre-learned by the network model and the real-time context information, generates a personalized response strategy for the real-time speech recognition result, so as to control the smart home system to respond according to the personalized response strategy.

[0109] After obtaining real-time speech features and real-time context information, the pre-trained network model can be used for speech recognition and generate corresponding personalized response strategies. Then, the personalized response strategy can be used to control the smart home system to respond, such as voice broadcasting of weather messages and controlling curtains to close.

[0110] In some examples, an input state can be generated based on real-time speech features and real-time context information. After obtaining the input state, the network model performs speech recognition based on the input state, and selects a corresponding action from a predefined action space based on the speech recognition result as a personalized response strategy.

[0111] For example, when an elderly person issues a voice command in the bedroom at 8 pm, the input state can be represented as the feature vector of the current voice plus the context information that the time is 8 pm and the location is the bedroom. Assuming that the dimension of the state vector is 256, the voice feature vector occupies 128 dimensions and the context information occupies 128 dimensions.

[0112] In the embodiment of the present invention, a unique state representation method is designed according to the specific needs and habits of designated users such as the elderly, and the speech features of designated users such as the elderly are converted into state vectors suitable for processing by the network model (deep Q network). At the same time, the action space is carefully defined to make it more in line with the voice instructions and demand categories of designated users such as the elderly.

[0113] In an embodiment of the present invention, by obtaining a pre-trained network model for a designated user, obtaining real-time voice data and real-time context information of the designated user, performing feature extraction on the real-time voice data to obtain real-time voice features, calling the network model, and generating a personalized response strategy based on the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy, the smart home system is implemented to perform personalized voice recognition and response for the designated user, so that the smart home system can be more suitable for designated users such as the elderly, and provide more accurate and convenient smart home services for the designated users.

[0114] Reference Figure 4 , shows a flowchart of another processing method based on a smart home system provided by some embodiments of the present invention, which may specifically include the following steps:

[0115] Step 401, obtaining a pre-trained network model for a specified user.

[0116] In order to enable the smart home system to perform personalized voice recognition and response for designated users and make the smart home system more adaptable to designated users such as the elderly, a network model for designated users can be pre-trained, such as the designated user can be an elderly user.

[0117] Step 402: Acquire historical interaction data of the designated user.

[0118] Step 403: Perform personalized adjustment on the network model according to the historical interaction data, so that the network model can learn the behavior pattern of the designated user.

[0119] During the use of the model, the historical interaction data of the specified user is continuously recorded, such as the voice commands and operating habits frequently used by the elderly at different times and in different scenarios. Then, the parameters and behaviors of the model can be dynamically adjusted according to the historical interaction data, and the network model learns the behavior patterns of the specified user. Since the behavior patterns of the specified user are learned in advance, the network model can combine the pre-learned behavior patterns when performing voice recognition and response to make personalized responses.

[0120] For example, an elderly person often asks about the weather and opens the curtains in the morning. The model will learn this behavior pattern and dynamically adjust the model's parameters and behavior based on the elderly person's behavior changes. If the elderly person's behavior pattern changes, such as starting to ask about the weather at night, the model will adjust its semantic understanding strategy for the elderly person accordingly.

[0121] Step 404, providing a personalized setting option to the user, and obtaining the personalized needs of the designated user in response to the user's operation on the personalized setting option;

[0122] Step 405: According to the personalized needs, the network model is personalized adjusted to set the personalized needs of the designated user for the network model.

[0123] In practical applications, personalized setting options can be provided. By selecting the personalized setting options, the personalized needs of the specified user can be obtained, and the behavior of the model can be adjusted according to the needs and preferences of the specified user. Since the personalized needs of the specified user are learned in advance, the network model can combine the pre-learned personalized needs to make personalized responses when performing speech recognition and response.

[0124] For example, the elderly can choose different voice response styles, such as friendly, formal, etc., or adjust the speed and volume of the response. Another example is that family members or caregivers can be allowed to make personalized settings for the elderly, such as setting reminders to take medicine on time, etc. The model will provide personalized services based on these settings.

[0125] Step 406: Acquire the real-time voice data and real-time context information of the designated user.

[0126] When a designated user needs to interact with the smart home system, such as an elderly person asking about today's weather or controlling the opening of curtains, the real-time voice data collected by the smart home device can be obtained, and the real-time context information of the real-time voice data, such as time, location, and the previous voice command, can be obtained.

[0127] Step 407: extract features from the real-time speech data to obtain real-time speech features.

[0128] After obtaining real-time speech data, the speech signal can be converted into a digital feature vector to obtain real-time speech features, which can then be used for subsequent speech recognition, such as using Mel-frequency cepstral coefficients to extract speech features. For example, a piece of speech data can be converted into a feature vector of length 128.

[0129] In some examples, before extracting voice features, real-time voice data can be processed for noise reduction to remove environmental noise and interference, such as using audio processing software to reduce background noise by 30% to improve voice clarity. In some examples, real-time voice data can also be processed for voice enhancement to highlight the voice features of the elderly, such as by adjusting the frequency range of the audio to make the voice of the elderly more prominent.

[0130] Step 408: calling the network model to generate a real-time speech recognition result according to the real-time speech feature.

[0131] Step 409, combining the behavior pattern of the designated user pre-learned by the network model, the real-time context information, and the personalized needs of the designated user set for the network model, generates a personalized response strategy for the real-time speech recognition result to control the smart home system to respond according to the personalized response strategy.

[0132] After obtaining real-time speech features and real-time context information, the pre-trained network model can be used for speech recognition and generate corresponding personalized response strategies. Then, the personalized response strategy can be used to control the smart home system to respond, such as voice broadcasting of weather messages and controlling curtains to close.

[0133] In some examples, an input state can be generated based on real-time speech features and real-time context information. After obtaining the input state, the network model performs speech recognition based on the input state, and selects a corresponding action from a predefined action space based on the speech recognition result as a personalized response strategy.

[0134] For example, when an elderly person issues a voice command in the bedroom at 8 pm, the input state can be represented as the feature vector of the current voice plus the context information that the time is 8 pm and the location is the bedroom. Assuming that the dimension of the state vector is 256, the voice feature vector occupies 128 dimensions and the context information occupies 128 dimensions.

[0135] In the embodiment of the present invention, a unique state representation method is designed according to the specific needs and habits of designated users such as the elderly, and the speech features of designated users such as the elderly are converted into state vectors suitable for processing by the network model (deep Q network). At the same time, the action space is carefully defined to make it more in line with the voice instructions and demand categories of designated users such as the elderly.

[0136] In an embodiment of the present invention, by obtaining a pre-trained network model for a designated user, obtaining real-time voice data and real-time context information of the designated user, performing feature extraction on the real-time voice data to obtain real-time voice features, calling the network model, and generating a personalized response strategy based on the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy, the smart home system is implemented to perform personalized voice recognition and response for the designated user, so that the smart home system can be more suitable for designated users such as the elderly, and provide more accurate and convenient smart home services for the designated users.

[0137] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0138] Reference Figure 5 , shows a schematic diagram of the structure of a processing device based on a smart home system provided by some embodiments of the present invention, which may specifically include the following modules:

[0139] A network model acquisition module 501 is used to acquire a pre-trained network model for a specified user;

[0140] A real-time data acquisition module 502 is used to acquire the real-time voice data and real-time context information of the designated user;

[0141] A real-time feature extraction module 503 is used to extract features from the real-time speech data to obtain real-time speech features;

[0142] The personalized response strategy generation and response module 504 is used to call the network model and generate a personalized response strategy according to the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy.

[0143] In some embodiments of the present invention, the calling of the network model to generate a personalized response strategy according to the real-time voice features and the real-time context information includes:

[0144] Calling the network model to generate a real-time speech recognition result according to the real-time speech feature;

[0145] The behavior pattern of the designated user pre-learned by the network model and the real-time context information are combined to generate a personalized response strategy for the real-time speech recognition result.

[0146] In some embodiments of the present invention, after obtaining the pre-trained network model for the specified user, the method further includes:

[0147] Obtaining historical interaction data of the specified user;

[0148] According to the historical interaction data, the network model is individually adjusted so that the network model learns the behavior pattern of the designated user.

[0149] In some embodiments of the present invention, the step of combining the behavior pattern of the designated user pre-learned by the network model and the real-time context information to generate a personalized response strategy for the real-time speech recognition result includes:

[0150] A personalized response strategy for the real-time speech recognition result is generated by combining the behavior pattern of the designated user pre-learned by the network model, the real-time context information, and the personalized needs of the designated user set for the network model.

[0151] In some embodiments of the present invention, after obtaining the pre-trained network model for the specified user, the method further includes:

[0152] Providing a personalized setting option to the user, and obtaining the personalized needs of the designated user in response to the user's operation on the personalized setting option;

[0153] According to the personalized needs, the network model is personalized adjusted so that the personalized needs of the designated user are set for the network model.

[0154] In some embodiments of the present invention, obtaining a pre-trained network model for a specified user includes:

[0155] Get sample data collected for a specified user;

[0156] Annotating the sample data to obtain an annotation result of the sample data;

[0157] The preset network model is trained in combination with the sample data and the annotation result to obtain a network model for a specified user.

[0158] In some embodiments of the present invention, the sample data includes sample voice data, and after acquiring the sample data collected for the designated user, the following further includes:

[0159] The sample speech data is subjected to noise reduction processing and / or speech enhancement processing.

[0160] In some embodiments of the present invention, the designated user is an elderly user.

[0161] In an embodiment of the present invention, by obtaining a pre-trained network model for a designated user, obtaining real-time voice data and real-time context information of the designated user, performing feature extraction on the real-time voice data to obtain real-time voice features, calling the network model, and generating a personalized response strategy based on the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy, the smart home system is implemented to perform personalized voice recognition and response for the designated user, so that the smart home system can be more suitable for designated users such as the elderly, and provide more accurate and convenient smart home services for the designated users.

[0162] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0163] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0164] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0165] A processing method based on a smart home system, the method comprising:

[0166] Get a pre-trained network model for a specified user;

[0167] Acquire real-time voice data and real-time context information of the specified user;

[0168] Extracting features from the real-time speech data to obtain real-time speech features;

[0169] The network model is called to generate a personalized response strategy according to the real-time voice features and the real-time context information, so as to control the smart home system to respond according to the personalized response strategy.

[0170] In some embodiments of the present invention, the calling of the network model to generate a personalized response strategy according to the real-time voice features and the real-time context information includes:

[0171] Calling the network model to generate a real-time speech recognition result according to the real-time speech feature;

[0172] The behavior pattern of the designated user pre-learned by the network model and the real-time context information are combined to generate a personalized response strategy for the real-time speech recognition result.

[0173] In some embodiments of the present invention, after obtaining the pre-trained network model for the specified user, the method further includes:

[0174] Obtaining historical interaction data of the specified user;

[0175] According to the historical interaction data, the network model is individually adjusted so that the network model learns the behavior pattern of the designated user.

[0176] In some embodiments of the present invention, the step of combining the behavior pattern of the designated user pre-learned by the network model and the real-time context information to generate a personalized response strategy for the real-time speech recognition result includes:

[0177] A personalized response strategy for the real-time speech recognition result is generated by combining the behavior pattern of the designated user pre-learned by the network model, the real-time context information, and the personalized needs of the designated user set for the network model.

[0178] In some embodiments of the present invention, after obtaining the pre-trained network model for the specified user, the method further includes:

[0179] Providing a personalized setting option to the user, and obtaining the personalized needs of the designated user in response to the user's operation on the personalized setting option;

[0180] According to the personalized needs, the network model is personalized adjusted so that the personalized needs of the designated user are set for the network model.

[0181] In some embodiments of the present invention, obtaining a pre-trained network model for a specified user includes:

[0182] Get sample data collected for a specified user;

[0183] Annotating the sample data to obtain an annotation result of the sample data;

[0184] The preset network model is trained in combination with the sample data and the annotation result to obtain a network model for a specified user.

[0185] In some embodiments of the present invention, the sample data includes sample voice data, and after acquiring the sample data collected for the designated user, the following further includes:

[0186] The sample speech data is subjected to noise reduction processing and / or speech enhancement processing.

[0187] In some embodiments of the present invention, the designated user is an elderly user.

[0188] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0189] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0190] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0191] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0192] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0195] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0196] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0197] The processing methods, devices, equipment, media and products based on the smart home system are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A processing method based on a smart home system, characterized in that: The method comprises: Get a pre-trained network model for a specified user; Acquire real-time voice data and real-time context information of the specified user; Extracting features from the real-time speech data to obtain real-time speech features; The network model is called to generate a personalized response strategy according to the real-time voice features and the real-time context information, so as to control the smart home system to respond according to the personalized response strategy.

2. The method according to claim 1, characterized in that The calling of the network model to generate a personalized response strategy according to the real-time voice features and the real-time context information includes: Calling the network model to generate a real-time speech recognition result according to the real-time speech feature; The behavior pattern of the designated user pre-learned by the network model and the real-time context information are combined to generate a personalized response strategy for the real-time speech recognition result.

3. The method according to claim 2, characterized in that After obtaining the pre-trained network model for the specified user, the method further includes: Obtaining historical interaction data of the specified user; According to the historical interaction data, the network model is individually adjusted so that the network model learns the behavior pattern of the designated user.

4. The method according to claim 2, characterized in that: The step of combining the behavior pattern of the designated user pre-learned by the network model and the real-time context information to generate a personalized response strategy for the real-time speech recognition result includes: A personalized response strategy for the real-time speech recognition result is generated by combining the behavior pattern of the designated user pre-learned by the network model, the real-time context information, and the personalized needs of the designated user set for the network model.

5. The method according to claim 4, characterized in that After obtaining the pre-trained network model for the specified user, the method further includes: Providing a personalized setting option to the user, and obtaining the personalized needs of the designated user in response to the user's operation on the personalized setting option; According to the personalized needs, the network model is personalized adjusted so that the personalized needs of the designated user are set for the network model.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of a pre-trained network model for a specified user includes: Get sample data collected for a specified user; Annotating the sample data to obtain an annotation result of the sample data; The preset network model is trained in combination with the sample data and the annotation result to obtain a network model for a specified user.

7. The method according to claim 6, characterized in that The sample data includes sample voice data, and after acquiring the sample data collected for the designated user, the method further includes: The sample speech data is subjected to noise reduction processing and / or speech enhancement processing.

8. The method according to claim 1, characterized in that: The designated user is an elderly user.

9. A processing device based on a smart home system, characterized in that: The device comprises: A network model acquisition module is used to obtain a pre-trained network model for a specified user; A real-time data acquisition module, used to acquire the real-time voice data and real-time context information of the specified user; A real-time feature extraction module, used to extract features from the real-time voice data to obtain real-time voice features; The personalized response strategy generation and response module is used to call the network model and generate a personalized response strategy according to the real-time voice features and the real-time context information to control the smart home system to respond according to the personalized response strategy.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 8 when executed by the processor.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images