Smart home control method based on voice recognition and context awareness

Abstract

The invention discloses a smart home control method based on voice recognition and context awareness. The method comprises the following steps: S1, collecting voice signals and environment state information; s2, performing preprocessing on the voice signal; s3, inputting the speech feature sequence into a speech recognition model based on a recurrent neural network transduction structure for recognition processing, and outputting a semantic text sequence; s4, constructing a two-stage trigger matching strategy, and if a matching result exists, determining that the request is a voice wake-up request; s5, executing situation classification, and outputting a current situation type label; and S6, according to the current situation type label, generating different instructions. According to the invention, by fusing voice recognition and environment perception, high-accuracy smart home control suitable for an aged scene is realized, and the system has timely response and personalized interaction capabilities.

Description

Technical Field

[0001] This invention relates to the field of big data processing and intelligent analysis technology, and in particular to a smart home control method based on voice recognition and context awareness. Background Technology

[0002] Currently, smart home control technology is gradually integrating into daily life scenarios. Voice recognition, as an important means of intelligent interaction, is widely used in home automation control tasks such as lighting control, environmental adjustment, and security alarms. Traditional voice control systems typically employ keyword-triggered and static command parsing methods, driving preset actions by recognizing fixed voice commands. However, these methods often lack a comprehensive understanding of the user's current context, making it difficult to dynamically adjust control strategies according to actual needs, resulting in limited interactive experience and insufficient adaptability.

[0003] Current speech recognition technologies are mostly based on connection-time classification (CTC) or end-to-end attention models for speech-to-text conversion. While these technologies have made some progress in the accuracy of general speech recognition, their stability remains challenging in a home environment due to background noise, differences in speaking habits, and sudden commands. Furthermore, most systems lack multi-stage semantic judgment for wake-up requests, failing to effectively address semantic ambiguity and sentence structure variations, making them prone to false or missed wake-ups. Current systems also exhibit relatively weak context awareness, often relying solely on voice input signals and ignoring external environmental factors such as lighting conditions, user behavior, and time information, making it difficult to achieve truly personalized and dynamically responsive home control logic.

[0004] Especially in special application scenarios such as age-friendly design and assisted care, existing solutions are clearly insufficient. Elderly users' language expression is often unpredictable, and some of their behavioral patterns pose potential risks. For example, if situations such as falls or getting up at night cannot be recognized and responded to in a timely manner, it could lead to serious consequences. Therefore, how to integrate multi-source environmental states with semantic wake-up content to build a voice control system with contextual understanding capabilities has become a key issue in current smart home technology.

[0005] Existing technologies still have significant shortcomings in terms of robustness of speech recognition, accuracy of semantic triggering, and intelligence of context awareness. There is an urgent need for a smart home control method that integrates multimodal data and has semantic awareness and environmental understanding capabilities to improve user experience and system security. Summary of the Invention

[0006] One objective of this invention is to propose an age-friendly intelligent wake-up and context recognition method for voice-controlled scenarios. This invention fully integrates a semantic trigger matching strategy, a context-aware mechanism, and an attention-guided multilayer perceptron model. It details the key processes and implementation methods in wake-up request recognition, semantic text parsing, and multi-source environmental state understanding, possessing advantages such as high semantic understanding accuracy, strong environmental perception robustness, and strong adaptability to complex home scenarios. This method effectively identifies semantic ambiguity that may occur in the expressions of elderly users through a two-stage trigger matching strategy, enhances temporal modeling capabilities by combining a predictive network and a gated contextual attention mechanism, and introduces joint modeling of environmental state vectors and semantic text to complete the context classification task. While improving the system's accuracy and practicality, it significantly enhances the intelligent interaction effect in highly sensitive scenarios such as assisted care and anomaly detection.

[0007] A smart home control method based on voice recognition and context awareness according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect voice signals and environmental status information in the user's living environment;

[0009] S2. Perform preprocessing on the speech signal;

[0010] S3. Input the speech feature sequence into a speech recognition model based on a recurrent neural network transduction structure for recognition processing. The speech recognition model includes an encoder, a prediction network and a joint network. The prediction network introduces a gated perception-enhanced prediction network and outputs a semantic text sequence by fusing the input vector sequence and the predicted state features.

[0011] S4. Compare the semantic text sequence with the preset semantic trigger word list to construct a two-stage trigger matching strategy. If a matching result exists, it is determined to be a voice wake-up request.

[0012] S5. Input the environmental state vector and the voice wake-up request into the attention-guided multilayer perceptron model, perform context classification, and output the current context type label.

[0013] S6. Generate different instructions based on the current context type label.

[0014] Optionally, the voice signal is acquired by a microphone array, and the environmental status information includes light intensity, user action status, bed pressure status, and current time data.

[0015] Optionally, the preprocessing includes performing spectral subtraction speech enhancement processing to obtain a denoised audio signal, and extracting Mel filter bank energy features to generate a standardized speech feature sequence.

[0016] Optionally, S3 specifically includes:

[0017] S31. The standardized speech feature sequence is input into the encoder part of the speech recognition model based on the recurrent neural network transduction structure. The encoder is constructed using a multi-layer stacked bidirectional recurrent neural network. At each time step, it processes the forward and backward information flows of speech features simultaneously, generates forward hidden state and reverse hidden state respectively, and constructs a hidden state sequence that integrates the context through splicing operation.

[0018] S32. Record the output label history corresponding to each time step of the encoder, and map each label in the label history to a fixed-dimensional label embedding vector by looking up a table; synchronously collect the current environmental state information, and encode the environmental state information into an environmental state vector by normalization and linear transformation operations; at each prediction time step, concatenate the current label embedding vector with the environmental state vector to generate a fused input vector sequence.

[0019] S33. The fused input vector sequence is input into a prediction network that incorporates a gated perception mechanism. The prediction network is an improved version of the original LSTM structure, constructed as a gated perception enhanced prediction network, specifically including the following processing steps:

[0020] The fused input vector at each time step is fed into an LSTM structure for temporal modeling to obtain the initial predicted hidden state at the current time step. Simultaneously, a set of context state caches is maintained to record the sequence of predicted hidden state information at each time step. Based on the current initial predicted hidden state as the query vector, a gated context attention mechanism is executed with the context state cache. This gated context attention mechanism dynamically adjusts the association strength between historical semantic states and the current prediction task by constructing a gated attention weight matrix, outputting a context-aware attention feature representation.

[0021] The initial predicted hidden state at the current time step is concatenated with the attention feature representation and input into the multi-scale feature fusion unit. In the multi-scale feature fusion unit, the concatenated features are transformed and fused through multiple linear mapping channels of different scales to extract the predicted state features at different semantic granularities.

[0022] S34. The fused input vector sequence generated by the encoder is paired with the predicted state features output by the prediction network, and then input into the joint network. The joint network consists of fully connected layers and nonlinear activation functions, which are used to perform nonlinear mapping on each pair of states and output the corresponding label category probability distribution sequence.

[0023] S35 performs a forward search in the dual-axis combination space formed by the fused input vector sequence generated by the encoder and the predicted state features output by the prediction network, enumerating all possible alignment paths; for each candidate path, it performs adjacent duplicate label compression and whitespace removal operations, and finally decodes to obtain the semantic text sequence.

[0024] Optionally, the gated perception-enhanced prediction network is an improved prediction network built on the original LSTM structure. Its core lies in introducing contextual modeling operations for historical predicted hidden states on the basis of performing temporal modeling.

[0025] The prediction network first receives a fused input vector sequence obtained by concatenating the current label embedding vector and the environment state vector. At each time step, this fused input vector is fed into an LSTM structure for temporal modeling, outputting the initial predicted hidden state for the current time step. Simultaneously, the prediction network maintains a set of context state caches to record the predicted hidden state information from the previous few time steps. Using the current initial predicted hidden state as the query vector, a gated attention weight matrix is ​​constructed in the context state cache, and a context-aware attention feature representation is output accordingly.

[0026] After completing the attention modeling, the initial predicted hidden state and the generated attention feature representation are concatenated and input into the multi-scale feature fusion unit; the predicted state features that can represent the current semantic prediction task are extracted; the predicted state features output by the prediction network are paired with the fused input vector sequence output by the encoder and used as the basic input for subsequent joint network to perform label category probabilistic inference.

[0027] Optionally, S4 specifically includes:

[0028] S41. Perform word segmentation on the semantic text sequence according to the natural language processing flow to generate a semantic fragment set composed of multiple semantic fragments;

[0029] S42. Construct a semantic trigger word list for age-appropriate wake-up recognition scenarios. The semantic trigger word list contains a set of standard phrases and equivalent fuzzy phrases representing requests for lighting, seeking help, and abnormal status feedback.

[0030] S43. In the first stage of the two-stage trigger matching strategy, the first round of trigger detection operation is performed on each semantic segment in the semantic segment set. The string similarity between the semantic segment and any word in the semantic trigger word list is judged based on the edit distance algorithm. If the similarity is higher than the set threshold, it is marked as a candidate matching item.

[0031] S44. In the second stage of the two-stage trigger matching strategy, for semantic segments that have been marked as candidate matches, extract the context segment sequence and perform the second round of trigger detection operation; perform semantic fusion between the current semantic segment and the context segment sequence to generate a context-aware semantic representation; call the standard word corresponding to the current candidate match from the semantic trigger vocabulary and extract the context representation template corresponding to the standard word in the training set; calculate the semantic similarity score between the context-aware semantic representation and the context representation template. If the semantic similarity score is higher than the set semantic trigger threshold, the current semantic segment is determined to be a valid trigger and the semantic trigger matching is confirmed to be successful.

[0032] S45. When there is at least one semantic segment in the set of semantic segments that triggers a successful match, the current semantic text sequence is determined to be a wake-up request, a matching identifier signal is output, and the context determination process is entered; if none of the semantic segments meet the matching conditions, it is determined to be non-wake-up voice information and subsequent control logic is not triggered.

[0033] Optionally, S5 specifically includes:

[0034] S51. When the current semantic text sequence is a wake-up request, collect the environmental state information at the current time point;

[0035] S52. Perform linear normalization on the light intensity data; convert the user action status identifier into a one-hot encoded form; perform discrete mapping encoding on the bed pressure status code; split the current time data into two numerical features according to hours and minutes; and concatenate the above light intensity data, user action status, bed pressure status code and time data in a fixed order to form an environmental status vector.

[0036] S53. Construct a multilayer perceptron model based on an attention-guided mechanism as a context classification network. The multilayer perceptron model consists of an input layer, two hidden layers, and an output layer. An attention weighting unit is inserted between each hidden layer. The attention weighting unit constructs an attention score based on the input environment state vector and the recognized semantic text sequence, and adjusts the contribution of the hidden layer feature mapping through a weighting operation.

[0037] S54. The environmental state vector and the semantic text sequence are concatenated as input and fed into the attention-guided multilayer perceptron model. The predicted probability is generated in the output layer by sequentially passing through the hidden layer calculation, attention weighting and nonlinear activation.

[0038] S55. Select the scenario type corresponding to the highest predicted probability from the output classification probability vector as the final classification result, and output the current scenario type label.

[0039] Optionally, S6 specifically includes:

[0040] S61. Receive the context type label output by the attention-guided multilayer perceptron model. When the context type label is "waking up at night with insufficient light," enter the lighting control process; when the context type label is "falling,"...

[0041] When the system is "falling down", "abnormally stationary", or "continuously calling", the emergency response procedure will be initiated.

[0042] S62. In the lighting control process, a lighting control command is generated, which includes a predefined command identifier "LIGHT_ON" and a target light intensity value parameter; the lighting control command is encapsulated into a control message via a local area network protocol and sent to the intelligent lighting device;

[0043] S63. In the emergency response process, perform the following operations in sequence:

[0044] Generate a voice broadcast command, wherein the voice broadcast command includes a pre-recorded distress call text;

[0045] Send the voice broadcast command to the voice broadcast device to drive the voice broadcast device to play the distress call prompt;

[0046] Generate an alarm notification message, the alarm notification message including a scenario type label, user location information and a timestamp;

[0047] Remote alarms are achieved by sending alarm notification messages to preset care terminals through a remote communication interface.

[0048] Generate a voice broadcast command, wherein the voice broadcast command includes a pre-recorded distress call text;

[0049] Optionally, the speech recognition model based on the recurrent neural network transduction structure includes an encoder, a prediction network, and a joint network:

[0050] The encoder receives a standardized speech feature sequence as input and uses a multi-layer stacked bidirectional recurrent neural network to generate a forward hidden state and a backward hidden state at each time step. The sequence of hidden states with fused context is formed by concatenation operation.

[0051] The concatenation result of the label embedding vector and the environment state vector received by the prediction network is used as the fusion input vector sequence. The LSTM structure with gated perception mechanism is used for temporal modeling to generate the initial prediction hidden state. A context state cache is constructed and an attention feature representation is generated through a gated context attention mechanism. The initial prediction hidden state and the attention feature representation are concatenated and input to the multi-scale feature fusion unit to extract the prediction state features.

[0052] The joint network receives the fused input vector sequence output by the encoder and the predicted state features output by the prediction network. After pairing them, they are input into the joint network composed of a fully connected layer and a nonlinear activation function, and output a sequence of label category probability distributions. A forward search is performed in the biaxial combination space to complete path decoding and output a semantic text sequence.

[0053] The beneficial effects of this invention are:

[0054] This invention significantly improves the applicability and responsiveness of voice interaction systems in home environments by constructing an intelligent control method that integrates speech recognition and context awareness. Firstly, the proposed speech recognition model based on a recurrent neural network transduction structure introduces a gated perception-enhanced prediction network during the speech recognition process, effectively improving the ability to recognize semantic units in continuous natural speech. This adapts to the characteristics of elderly users, such as slow speech speed and ambiguous semantic expressions, thereby improving the overall recognition accuracy.

[0055] This invention proposes a two-stage trigger matching strategy based on edit distance and contextual semantic fusion. It performs robust detection on wake-up keywords in user speech, which can identify both standard trigger phrases and semantically similar non-standard expressions. This enhances the system's semantic perception capability in real home environments and reduces false wake-up and missed wake-up rates.

[0056] In the context classification stage, this invention jointly models the environmental state vector and semantic wake-up request to construct an attention-guided multilayer perceptron model, effectively improving the ability to understand complex contexts. Especially in high-risk scenarios such as insufficient lighting, user falls, and abnormal stillness, this model can achieve highly accurate context classification and automatically trigger lighting control or alarm responses, achieving a dual improvement in safety and intelligence. Overall, this invention possesses significant advantages in accurate voice recognition, comprehensive context awareness, and intelligent response mechanisms, making it suitable for building smart home control systems that better meet user needs. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 This is an overall flowchart of a smart home control method based on speech recognition and context awareness proposed in this invention.

[0059] Figure 2 This is a schematic diagram of a speech recognition model based on a recurrent neural network transduction structure proposed in this invention;

[0060] Figure 3This is a schematic diagram of a two-stage triggering matching strategy proposed in this invention. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0062] refer to Figure 1-3 A smart home control method based on voice recognition and context awareness includes the following steps:

[0063] S1. Collect voice signals and environmental status information in the user's living environment;

[0064] S2. Perform preprocessing on the speech signal;

[0065] S3. Input the speech feature sequence into a speech recognition model based on a recurrent neural network transduction structure for recognition processing. The speech recognition model includes an encoder, a prediction network and a joint network. The prediction network introduces a gated perception-enhanced prediction network and outputs a semantic text sequence by fusing the input vector sequence and the predicted state features.

[0066] S4. Compare the semantic text sequence with the preset semantic trigger word list to construct a two-stage trigger matching strategy. If a matching result exists, it is determined to be a voice wake-up request.

[0067] S5. Input the environmental state vector and the voice wake-up request into the attention-guided multilayer perceptron model, perform context classification, and output the current context type label.

[0068] S6. Generate different instructions based on the current context type label.

[0069] This invention proposes a smart home control method based on speech recognition and context awareness. By collecting speech signals and environmental state information in the user's living environment, and combining speech signal preprocessing and feature extraction, a speech recognition model based on a recurrent neural network transduction structure is used to generate semantic text sequences. A two-stage trigger matching strategy is constructed to achieve accurate recognition of voice wake-up requests. Furthermore, the environmental state vector and wake-up request are input into an attention-guided multilayer perceptron model to complete intelligent classification of context types. Finally, corresponding control commands are generated based on the classification results to achieve automated response and human-computer interaction control of home devices.

[0070] In this embodiment, the voice signal is acquired by a microphone array, and the environmental status information includes light intensity, user action status, bed pressure status, and current time data.

[0071] In this embodiment, the preprocessing includes performing spectral subtraction speech enhancement processing to obtain a denoised audio signal, and extracting Mel filter bank energy features to generate a standardized speech feature sequence.

[0072] In this embodiment, S3 specifically includes:

[0073] S31. The standardized speech feature sequence is input into the encoder part of the speech recognition model based on the recurrent neural network transduction structure. The encoder is constructed using a multi-layer stacked bidirectional recurrent neural network. At each time step, it processes the forward and backward information flows of speech features simultaneously, generates forward hidden states and backward hidden states respectively, and constructs a hidden state sequence that integrates context through splicing operations, forming an acoustic representation containing the semantic dependencies between the preceding and following states. This is used to provide input features with context modeling capabilities for the subsequent prediction network and joint network.

[0074] S32. During the speech recognition inference process, the output label history corresponding to each time step of the encoder is recorded to form a semantic label history sequence, and each label in the sequence is mapped to a fixed-dimensional label embedding vector through a lookup table; the current environmental state information, including light intensity, user action state and bed pressure state, is collected synchronously and encoded into an environmental state vector using normalization and linear transformation operations; at each prediction time step, the current label embedding vector and the environmental state vector are concatenated to generate a fused input vector sequence, which is used as the input of the prediction network.

[0075] S33. The fused input vector sequence is input into a prediction network that incorporates a gated perception mechanism. The prediction network is an improved version of the original LSTM structure, constructed as a gated perception enhanced prediction network, specifically including the following processing steps:

[0076] The fused input vector at each time step is fed into an LSTM structure for temporal modeling to obtain the initial predicted hidden state at the current time step. A set of context state caches is maintained to record the predicted hidden state information sequence of the previous several time steps. Based on the current initial predicted hidden state as the query vector, a gated context attention mechanism is executed with the context state cache. The gated context attention mechanism constructs a gated attention weight matrix to dynamically adjust the correlation strength between historical semantic states and the current prediction task, and outputs a context-aware attention feature representation.

[0077] The initial predicted hidden state at the current time step is concatenated with the attention feature representation and input into the multi-scale feature fusion unit. In the multi-scale feature fusion unit, the concatenated features are transformed and fused through multiple linear mapping channels of different scales to extract the predicted state features at different semantic granularities.

[0078] S34. Pair the fused input vector generated by the encoder with the predicted state features output by the prediction network, and input them into the joint network. The joint network consists of fully connected layers and nonlinear activation functions, which are used to perform nonlinear mapping on each pair of states and output the corresponding label category probability distribution sequence.

[0079] S35 performs a forward search in the dual-axis combination space formed by the fused input vector sequence generated by the encoder and the predicted state features output by the prediction network, enumerating all possible alignment paths; for each candidate path, it performs adjacent duplicate label compression and whitespace removal operations, and finally decodes to obtain the semantic text sequence.

[0080] This invention achieves accurate recognition of semantic text in speech signals by constructing a speech recognition model based on a recurrent neural network transduction structure. Specifically, a standardized speech feature sequence is input into an encoder constructed using a bidirectional recurrent neural network to generate an acoustic representation incorporating contextual information. During the inference phase, a semantic label history sequence is constructed based on the encoder output, and combined with synchronously acquired environmental state information to generate a fused input vector sequence. The fused input vector is then input into a prediction network incorporating a gated perception mechanism. Through a gated contextual attention mechanism and a multi-scale feature fusion unit, predicted state features are extracted. Subsequently, the encoder output and the prediction network output are paired and input into a joint network to calculate the label probability distribution. Forward search and path decoding are then performed in a biaxial combination space, ultimately outputting an accurate semantic text sequence.

[0081] In this embodiment, S4 specifically includes:

[0082] S41. Perform word segmentation on the semantic text sequence according to the natural language processing flow to generate a semantic fragment set composed of multiple semantic fragments;

[0083] S42. Construct a semantic trigger word list for age-appropriate wake-up recognition scenarios. The semantic trigger word list contains a set of standard phrases and equivalent fuzzy phrases representing requests for lighting, seeking help, and abnormal status feedback.

[0084] S43. In the first stage of the two-stage trigger matching strategy, for each semantic segment in the semantic segment set, perform the first round of trigger detection operation, and determine the string similarity between the semantic segment and any word in the semantic trigger vocabulary based on the edit distance algorithm. :

[0085] ;

[0086] like If the value is higher than the set threshold, it is marked as a candidate match; Represents semantic fragments The string between and any word in the semantic trigger vocabulary Edit distance;

[0087] S44. In the second stage of the two-stage triggering matching strategy, for the semantic segment that has been marked as a candidate match, extract the semantic segments corresponding to the first two and last two time steps in the semantic text sequence to form a context segment sequence of length 5; perform semantic fusion between the current semantic segment and the context segment sequence to generate a context-aware semantic representation vector. ; Retrieve the standard term corresponding to the current candidate match from the semantic trigger vocabulary, and extract the context representation template vector of the standard term in the training set. Calculate semantic similarity score :

[0088] ;

[0089] like If the threshold for semantic triggering is higher than the set threshold, the current semantic fragment is determined to be a valid trigger and the semantic triggering match is confirmed to be successful.

[0090] S45. When there is at least one semantic segment in the set of semantic segments that triggers a successful match, the current semantic text sequence is determined to be a wake-up request, a matching identifier signal is output, and the context determination process is entered; if none of the semantic segments meet the matching conditions, it is determined to be non-wake-up voice information and subsequent control logic is not triggered.

[0091] This invention proposes a semantic wake-up determination method combining a two-stage trigger matching strategy. For semantic text sequences generated by speech recognition, the method first performs word segmentation through natural language processing to construct a set of semantic segments. Then, a semantic trigger vocabulary is built based on age-appropriate scenarios, covering common expressions such as requests for lighting, calls for help, and abnormal feedback. In the first-stage trigger detection, a string similarity determination mechanism based on the edit distance algorithm is used to perform coarse matching between semantic segments and vocabulary entries, filtering out candidate matches. In the second-stage trigger detection, semantic context segments are further extracted, and context-aware representation vectors are generated through semantic fusion. Semantic similarity is calculated between these vectors and the context representation template vectors extracted from the training set for the semantic trigger words. If the similarity score exceeds a set threshold, the segment is confirmed as a valid wake-up item. The presence or absence of at least one valid wake-up match determines whether the entire semantic text represents a voice wake-up request. This method possesses strong adaptability to fuzzy expressions and contextual understanding capabilities, significantly improving the accuracy and robustness of voice wake-up recognition.

[0092] In this embodiment, S5 specifically includes:

[0093] S51. When the semantic wake-up matching process outputs a matching identifier signal, the environmental status information at the current time point is collected. The environmental status information includes light intensity data collected by the light sensor, user action status identifier obtained by the action recognition module, bed pressure status code obtained by the mattress pressure sensor, and current time data obtained by the system clock.

[0094] S52. Perform linear normalization on the light intensity data; convert the user action status identifier into a one-hot encoded form; perform discrete mapping encoding on the bed pressure status code; split the current time data into two numerical features according to hours and minutes; and concatenate the above four types of features in a fixed order to form an environmental status vector.

[0095] S53. Construct a multilayer perceptron model based on an attention-guided mechanism as a context classification network. The model consists of an input layer, two hidden layers, and an output layer. An attention weighting unit is inserted between each hidden layer. The attention weighting unit constructs an attention score based on the input environment state vector and the recognized semantic text sequence, and adjusts the contribution of the hidden layer feature mapping through a weighting operation.

[0096] S54. Denote the environment state vector generated in step S52 as... The semantic text sequence confirmed in steps S41–S45 is denoted as A joint input vector is constructed through concatenation operations. The input vector is fed into an attention-guided multilayer perceptron model. This attention-guided multilayer perceptron model consists of an input layer, two hidden layers, and an output layer. The input layer accepts a joint input vector. The output of the first hidden layer is The output of the second hidden layer is ;

[0097] In the attention-guided multilayer perceptron model In the hidden layer, Indicates the first The dimension of the feature vectors in the hidden layer, which is the dimension of each feature vector. Semantic text sequence notes Attention is scored, and during the weighted calculation process, all the first-order attention scores are assigned a weighted score. The index of weight estimation performed by each hidden unit, and the attention score. The calculation method is as follows:

[0098] ;

[0099] in, and These are the linear transformation matrices applied to the hidden state and the semantic vector, respectively. For attention projection vectors, This represents the hyperbolic tangent activation function. It is the first The first hidden layer Each feature vector is weighted, and the importance of each feature representation is adjusted to generate a weighted hidden state representation vector. :

[0100] ;

[0101] The weighted hidden state representation vector mentioned above serves as the input to the next layer, and undergoes a nonlinear activation transformation sequentially. Finally, a context classification probability vector is generated at the output layer. Each component This indicates that the input sample belongs to the context type label. The predicted probability, where The number of categories for predefined context type labels.

[0102] S55. Select the situation type label with the highest probability from the output classification probability vector as the final classification result, output the current situation type label, and proceed to the subsequent home response task process.

[0103] This invention addresses the need for understanding the home environment after voice wake-up by proposing a context-aware method that integrates environmental state and semantic text information. Specifically, upon detecting wake-up semantics, the system collects multi-dimensional environmental state information in real time, including light intensity, user action state, bed pressure status, and current time. This information is then normalized, encoded, and concatenated to generate an environmental state vector. Combined with the recognized semantic text sequence, a joint input vector is constructed and fed into a multilayer perceptron model incorporating an attention-guided mechanism. In the hidden layer, this model assigns differentiated attention weights to different feature dimensions based on semantic information guidance, dynamically adjusting the contribution of each dimension to context determination. This generates a more semantically relevant weighted feature representation, and finally, a context classification probability vector is generated in the output layer. The label corresponding to the highest probability is selected as the current context type. This method achieves multi-modal fusion perception under semantic guidance, effectively improving the accuracy of context classification and the responsiveness of smart home control.

[0104] In this embodiment, S6 specifically includes:

[0105] S61. Receive the context type label output by the attention-guided multilayer perceptron model. When the context type label is "waking up at night with insufficient light," enter the lighting control process; when the context type label is "falling,"...

[0106] When the system is "falling down", "abnormally stationary", or "continuously calling", the emergency response procedure will be initiated.

[0107] S62. In the lighting control process, a lighting control command is generated, which includes a predefined command identifier "LIGHT_ON" and a target light intensity value parameter; the lighting control command is encapsulated into a control message through a local area network protocol and sent to the smart lighting device;

[0108] S63. In the emergency response process, perform the following operations in sequence:

[0109] Generate a voice broadcast command, wherein the voice broadcast command includes a pre-recorded distress call text;

[0110] Send the voice broadcast command to the voice broadcast device to drive the voice broadcast device to play the distress call prompt;

[0111] An alarm notification message is generated, which includes a scenario type label, user location information, and a timestamp; the alarm notification message is sent to a preset care terminal through a remote communication interface to achieve remote alarm.

[0112] This invention, after completing the situation type identification, intelligently schedules home control tasks based on the identification results, significantly improving the system's response efficiency and user-friendliness. Specifically, when the situation type label is determined to be "getting up at night with insufficient light," the system immediately triggers the lighting control process, automatically generating and sending a control command containing the command identifier "LIGHT_ON" and light intensity parameters, ensuring that the ambient lighting responds promptly to user behavior. In emergency situations such as "falling," "abnormal stillness," or "continuous calling," the system enters an emergency linkage process, successively activating the voice broadcast device to play a distress call prompt, and constructing an alarm notification message containing the situation type, user location information, and timestamp, which is sent to the care terminal through a remote communication interface to achieve remote emergency alarm. This invention realizes a closed-loop home control system based on semantic situation recognition, balancing convenience and emergency risk response capabilities, and has good practicality and intelligent value.

[0113] In this embodiment, the speech recognition model based on the recurrent neural network transduction structure includes an encoder, a prediction network, and a joint network:

[0114] The encoder receives a standardized speech feature sequence as input and uses a multi-layer stacked bidirectional recurrent neural network to generate a forward hidden state and a backward hidden state at each time step. The sequence of hidden states with fused context is formed by concatenation.

[0115] The concatenation result of the label embedding vector and the environment state vector received by the prediction network is used as the fusion input vector sequence. The LSTM structure with gated perception mechanism is used for temporal modeling to generate the initial prediction hidden state. A context state cache is constructed and an attention feature representation is generated through a gated context attention mechanism. The initial prediction hidden state and the attention feature representation are concatenated and input to the multi-scale feature fusion unit to extract the prediction state features.

[0116] The joint network receives the fused input vector sequence output by the encoder and the predicted state features output by the prediction network. After pairing them, they are input into the joint network composed of a fully connected layer and a nonlinear activation function, and output a sequence of label category probability distributions. A forward search is performed in the biaxial combination space to complete path decoding and output a semantic text sequence.

[0117] This invention employs a speech recognition model based on a recurrent neural network transduction structure, fusing speech features with environmental state information to significantly improve the accuracy and environmental adaptability of semantic recognition. Specifically, the encoder extracts the hidden state sequence of the fused context through a multi-layered stacked bidirectional recurrent neural network, providing rich semantic information for subsequent prediction; the prediction network introduces a gating perception mechanism, fusing label embeddings and environmental state information, and generates dynamic semantic context representations by combining a gating context attention mechanism, while enhancing the predicted state features through multi-scale feature fusion units; the joint network pairs the features output by the encoder and prediction network, generates label probability distributions through nonlinear mapping, and performs a forward search in a dual-axis combination space to output the final semantic text sequence. This invention achieves joint modeling of speech semantic recognition and environmental perception, enhancing the robustness and accuracy of speech recognition in complex home scenarios.

[0118] Example 1:

[0119] To verify the feasibility of this invention in practice, it was deployed and tested in a smart elderly care scenario. This scenario simulates the voice interaction and emergency response needs of an elderly person living alone in their daily home life, aiming to solve the problems of low wake-up accuracy, insufficient context awareness, and untimely response to emergencies in traditional smart home systems. The test environment was set up in a simulated home bedroom, equipped with a voice acquisition device, environmental sensing devices (light sensor, motion recognition camera, mattress pressure sensor), a smart lighting system, a voice broadcast device, and an alarm notification module.

[0120] In application, the system first deploys a voice acquisition module to acquire user language commands in real time through a microphone array, such as "I want to get up" or "I fell down." The voice signal undergoes spectral subtraction for noise reduction locally, and the energy of the Mel filter bank is extracted as a feature input to the improved RNN-T speech recognition model. This model constructs an encoder based on a bidirectional RNN and integrates environmental conditions (such as current illumination of 12 lux, user's lying position, mattress pressure concentrated in the central axis area, and system time 03:26) as auxiliary input to the prediction module, making the recognition accuracy more stable in non-standard sentence expressions or emotionally fluctuating contexts.

[0121] When the recognition result is a wake-up request, the system further executes a two-stage semantic matching process. For example, if a user says "It's so dark I can't get up," the system first uses edit distance matching to create candidate matches between "can't get up" and "get up," and then combines the contextual semantics of "it's so dark" to determine that it is a "nighttime wake-up" scenario in a low-light environment, thus confirming it as a wake-up request.

[0122] The context-aware module is activated, and the environmental state vector (light intensity = 12, user) at the above time points is processed.

[0123] The input (action = 0, mattress pressure code = 2, time = 03:26) is fused with semantic text. The context classification model focuses on the "low light + lying down" combination based on attention weights, ultimately outputting the context label "getting up at night with insufficient light," triggering the lighting control process. The system sends the "LIGHT_ON" command to the intelligent lighting control system, setting the brightness to 80%. The entire response process takes no more than 2 seconds, and the user successfully gets up. Through a 7-day simulation test, this system performed excellently in terms of speech recognition accuracy, trigger success rate, and context discrimination accuracy. The statistical results are shown in Table 1 below:

[0124] Table 1 Performance Evaluation of Smart Home Control System in Typical Scenarios

[0125] Test Scenario Speech recognition accuracy Wake-up judgment accuracy Context recognition accuracy Average response time (seconds) Control execution success rate Waking up at night (in low light) 96.8% 100% 98.2% 1.9 100% Falling and calling for help 97.4% 100% 100% 2.2 100% Normal state of bed rest during the day 98.1% No trigger No trigger - - Calling out to someone (not a wake-up call) 94.3% 93.3% 92.1% 2.4 100% Three consecutive light requests 95.6% 100% 96.7% 2.0 100% Abnormal stillness (prolonged immobility) 96.2% 100% 100% 2.3 100%

[0126] In simulated emergency scenarios, such as a user suddenly falling and shouting "Help me!", the system can trigger semantic matching through the keyword "help me," and accurately identify the "fall" situation by combining action recognition (falling state), rapid transfer of mattress pressure, system time, and other information. The system immediately executes the linkage process, playing a voice prompt "Family members have been notified, please wait," and simultaneously encapsulates the "fall" tag, the user's location "master bedroom," and the timestamp "04:53" into a JSON message, which is sent to the care platform terminal via Wi-Fi. The experiment simulated 15 emergency call events, achieving a 100% response success rate and an average response time of 2.1 seconds, far superior to traditional rule-based matching systems (average response time 3.8 seconds, false recognition rate 12%).

[0127] This invention demonstrates advantages such as high recognition rate, low false trigger rate and timely response in smart home age-friendly applications, significantly improving the system's adaptability to complex semantics and multi-dimensional environmental states. It is particularly suitable for scenarios involving the nighttime lighting needs of elderly people living alone and automatic response to sudden health events.

[0128] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A smart home control method based on speech recognition and context awareness, characterized in that, Includes the following steps: S1. Collect voice signals and environmental status information in the user's living environment; S2. Perform preprocessing on the speech signal; S3. Input the speech feature sequence into a speech recognition model based on a recurrent neural network transduction structure for recognition processing. The speech recognition model includes an encoder, a prediction network and a joint network. The prediction network introduces a gated perception-enhanced prediction network and outputs a semantic text sequence by fusing the input vector sequence and the predicted state features. S4. Compare the semantic text sequence with the preset semantic trigger word list to construct a two-stage trigger matching strategy. If a matching result exists, it is determined to be a voice wake-up request. S5. Input the environmental state vector and the voice wake-up request into the attention-guided multilayer perceptron model, perform context classification, and output the current context type label. S6. Generate different instructions based on the current context type label.

2. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, The voice signal is acquired by a microphone array, and the environmental status information includes light intensity, user action status, bed pressure status, and current time data.

3. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, The preprocessing includes performing spectral subtraction speech enhancement processing to obtain a denoised audio signal, and extracting Mel filter bank energy features to generate a standardized speech feature sequence.

4. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, S3 specifically includes: S31. The standardized speech feature sequence is input into the encoder part of the speech recognition model based on the recurrent neural network transduction structure. The encoder is constructed using a multi-layer stacked bidirectional recurrent neural network. At each time step, the forward and backward information flows of the standardized speech feature sequence are processed simultaneously to generate forward hidden state and reverse hidden state respectively. The hidden state sequence with fused context is constructed by splicing operation. S32. Record the hidden state sequence corresponding to each time step of the encoder, align the hidden state sequence with the historical labels, and map each historical label to a fixed-dimensional label embedding vector by looking up a table; synchronously collect the current environmental state information, and encode the environmental state information into an environmental state vector using normalization and linear transformation operations; at each prediction time step, concatenate the current label embedding vector with the environmental state vector to generate a fused input vector sequence. S33. The fused input vector sequence is input into a prediction network with a gated perception mechanism. The prediction network is improved based on the original LSTM structure and constructed as a gated perception enhanced prediction network. S34. The fused input vector sequence generated by the encoder is paired with the predicted state features output by the prediction network and then input into the joint network. The joint network consists of fully connected layers and nonlinear activation functions, which are used to perform nonlinear mapping on each pair of states and output the corresponding label category probability distribution sequence. S35 performs a forward search in the dual-axis combination space formed by the fused input vector sequence generated by the encoder and the predicted state features output by the prediction network, enumerating all possible alignment paths; for each candidate path, it performs adjacent duplicate label compression and whitespace removal operations, and finally decodes to obtain the semantic text sequence.

5. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, The gated perception enhancement prediction network specifically includes the following processing flow: The fused input vector at each time step is fed into an LSTM structure for temporal modeling to obtain the initial predicted hidden state at the current time step. Simultaneously, a set of context state caches is maintained to record the sequence of predicted hidden state information at each time step. Based on the current initial predicted hidden state as the query vector, a gated context attention mechanism is executed with the context state cache. This gated context attention mechanism constructs a gated attention weight matrix and dynamically adjusts the association strength between historical semantic states and the current prediction task, outputting a context-aware attention feature representation. The initial predicted hidden state at the current time step is concatenated with the attention feature representation and input into the multi-scale feature fusion unit. In the multi-scale feature fusion unit, the concatenated features are transformed and fused through multiple linear mapping channels of different scales to extract the predicted state features at different semantic granularities.

6. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, S4 specifically includes: S41. Perform word segmentation on the semantic text sequence according to the natural language processing flow to generate a semantic fragment set composed of multiple semantic fragments; S42. Construct a semantic trigger word list for age-appropriate wake-up recognition scenarios. The semantic trigger word list contains a set of standard phrases and equivalent fuzzy phrases representing requests for lighting, seeking help, and abnormal status feedback. S43. In the first stage of the two-stage trigger matching strategy, the first round of trigger detection operation is performed on each semantic segment in the semantic segment set. The string similarity between the semantic segment and any word in the semantic trigger word list is judged based on the edit distance algorithm. If the similarity is higher than the set threshold, it is marked as a candidate matching item. S44. In the second stage of the two-stage trigger matching strategy, for semantic segments that have been marked as candidate matches, extract the context segment sequence and perform the second round of trigger detection operation; perform semantic fusion between the current semantic segment and the context segment sequence to generate a context-aware semantic representation; call the standard word corresponding to the current candidate match from the semantic trigger vocabulary and extract the context representation template corresponding to the standard word in the training set; calculate the semantic similarity score between the context-aware semantic representation and the context representation template. If the semantic similarity score is higher than the set semantic trigger threshold, the current semantic segment is determined to be a valid trigger and the semantic trigger matching is confirmed to be successful. S45. When there is at least one semantic segment in the set of semantic segments that triggers a successful match, the current semantic text sequence is determined to be a voice wake-up request, a matching identifier signal is output, and the context determination process is entered; if none of the semantic segments meet the matching conditions, it is determined to be non-wake-up voice information and subsequent control logic is not triggered.

7. A smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, S5 specifically includes: S51. When the current semantic text sequence is a wake-up request, collect the environmental state information at the current time point; S52. Perform linear normalization on the light intensity data; convert the user action status identifier into a one-hot encoded form; perform discrete mapping encoding on the bed pressure status code; split the current time data into two numerical features according to hours and minutes; and concatenate the above light intensity data, user action status, bed pressure status code and time data in a fixed order to form an environmental status vector. S53. Construct an attention-guided multilayer perceptron model as a context classification network. The multilayer perceptron model consists of an input layer, two hidden layers, and an output layer. An attention weighting unit is inserted between each hidden layer. The attention weighting unit constructs an attention score based on the input environment state vector and the recognized semantic text sequence, and adjusts the contribution of the hidden layer feature mapping through weighting operations. S54. The environmental state vector and the semantic text sequence are concatenated as input and fed into the attention-guided multilayer perceptron model. The predicted probability is generated in the output layer by sequentially passing through the hidden layer calculation, attention weighting and nonlinear activation. S55. Select the scenario type corresponding to the highest predicted probability from the output classification probability vector as the final classification result, and output the current scenario type label.

8. The smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, S6 specifically includes: S61. Receive the context type label output by the attention-guided multilayer perceptron model. When the context type label is "waking up at night with insufficient light," enter the lighting control process; when the context type label is "falling,"... When the system is "falling down", "abnormally stationary", or "continuously calling", the emergency response procedure will be initiated. S62. In the lighting control process, a lighting control command is generated, which includes a predefined command identifier and a target light intensity value parameter; the lighting control command is encapsulated into a control message via a local area network protocol and sent to the intelligent lighting device. S63. In the emergency response process, perform the following operations in sequence: Generate a voice broadcast command, wherein the voice broadcast command includes a pre-recorded distress call text; Send the voice broadcast command to the voice broadcast device to drive the voice broadcast device to play the distress call prompt; Generate an alarm notification message, the alarm notification message including a scenario type label, user location information and a timestamp; Remote alarms are achieved by sending alarm notification messages to preset care terminals through a remote communication interface.

9. A smart home control method based on speech recognition and context awareness according to claim 1, characterized in that, The speech recognition model based on the recurrent neural network transduction structure includes an encoder, a prediction network, and a joint network: The encoder receives a standardized speech feature sequence as input and uses a multi-layer stacked bidirectional recurrent neural network to generate a forward hidden state and a backward hidden state at each time step. The sequence of hidden states with fused context is formed by concatenation. The concatenation result of the label embedding vector and the environment state vector received by the prediction network is used as the fusion input vector sequence. The LSTM structure with gated perception mechanism is used for temporal modeling to generate the initial prediction hidden state. Build a context state cache and generate attention feature representations through a gated context attention mechanism; The initial predicted hidden state is concatenated with the attention feature representation and input into the multi-scale feature fusion unit to extract the predicted state features. The joint network receives the fused input vector sequence output by the encoder and the predicted state features output by the prediction network. After pairing them, they are input into the joint network composed of a fully connected layer and a nonlinear activation function, and output a sequence of label category probability distributions. A forward search is performed in the biaxial combination space to complete path decoding and output a semantic text sequence.

Images