Voice recognition-based intelligent control system for age-appropriate kitchen appliances

Abstract

The application discloses a kind of old kitchen appliance intelligent control system based on speech recognition, comprising: data acquisition processing module, for generating speech feature sequence, speech event marker set and appliance state vector;Model construction module, for building kitchen control Mamba model;State condition coding module, for generating appliance state associated speech representation;Intermittent intention keeping module, for building and updating control intention keeping state, generating old speech intention representation;Control action decoding module, for generating speech text sequence and kitchen control action sequence;Frame action verification module, for frame-level alignment, generating frame-level alignment relationship and control instruction set;Hierarchical control execution module, for executing hierarchical control response, recording execution feedback and updating appliance state vector.The application improves the accuracy of intermittent speech recognition and the safety of kitchen appliance control.

Description

Technical Field

[0001] This invention relates to the field of smart home and home appliance control, and in particular to an age-friendly smart control system for kitchen appliances based on voice recognition. Background Technology

[0002] With the development of smart home technology, voice control has been gradually applied to kitchen appliances such as gas stoves, microwave ovens, rice cookers, and range hoods. Existing voice control systems typically recognize continuous, clear, and complete voice commands, then convert the recognized text into corresponding device control commands, which can meet the basic operational needs of general users.

[0003] Elderly users often experience prolonged pauses, repetitive statements, impromptu corrections, incomplete expressions, and multiple control intentions contained within a single sentence when using voice control. Current technology has limited processing capabilities for these discontinuous speech patterns, easily ending recognition prematurely at pauses, misinterpreting repetitive content as multiple commands, or failing to accurately update the original control intention based on subsequent corrections. This leads to incorrect combinations between the appliance, control action, action intensity, and timing of execution.

[0004] Furthermore, existing voice control systems for kitchen appliances typically focus on text recognition accuracy, lacking synchronous verification between voice content, control actions, and the appliance's current operating status. Generated control actions may not fall within the appliance's current executable range, and there may be a lack of traceable correspondence between text content and control fields. For high-risk operations requiring confirmation or prohibition, existing systems often employ fixed permission rules, making it difficult to reliably judge based on voice evidence, the appliance's executable status, and security levels simultaneously. Therefore, issues of misoperation and insufficient control security persist.

[0005] Therefore, how to provide an age-friendly smart control system for kitchen appliances based on voice recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an age-friendly intelligent control system for kitchen appliances based on speech recognition. This invention constructs a kitchen control Mamba model using speech feature sequences, a set of speech event markers, and appliance state vectors. An appliance state condition unit generates an appliance state-related speech representation, and an intermittent intent preservation unit processes speech pauses, repeated speech segments, and corrected speech segments to generate an age-friendly speech intent representation. An action isomorphic decoding unit simultaneously generates a speech text sequence and a kitchen control action sequence. A frame action verification unit performs frame-level alignment of feature frames, speech text sequences, and kitchen control action sequences, and combines executable actions and confirmation levels to generate control commands to be executed, control commands to be confirmed, and control commands to be prohibited from execution. This invention can improve the accuracy of control intent recognition under intermittent and corrected speech conditions, reduce incorrect combinations of kitchen control actions, and enhance the safety and reliability of age-friendly voice control.

[0007] According to an embodiment of the present invention, an age-friendly smart control system for kitchen appliances based on voice recognition includes: The data acquisition and processing module is used to collect elderly people's voice data and kitchen appliance status data in the kitchen environment, perform data preprocessing, and generate voice feature sequences, voice event tag sets, and appliance status vectors. The model building module is used to build a kitchen control Mamba model based on the Speech-Mamba model, including an appliance state condition unit, an intermittent intent holding unit, an action isomorphic decoding unit, and a frame action verification unit. The state condition coding module is used to introduce the speech feature sequence and the appliance state vector into the state space coding process based on the appliance state condition unit to generate the appliance state-related speech representation. The intermittent intent holding module is used in the intermittent intent holding unit to construct the control intent holding state based on the electrical appliance status associated with the voice representation, update the control intent holding state based on the voice event tag set, and generate an age-appropriate voice intent representation. The control action decoding module is used to call the action isomorphic decoding unit to decode based on the age-appropriate speech intent representation, and generate speech text sequence and kitchen control action sequence; The frame action verification module is used to perform frame-level alignment on the speech feature sequence, speech text sequence and kitchen control action sequence based on the frame action verification unit, generate frame-level alignment relationship, and convert the kitchen control action sequence into a set of control instructions according to the frame-level alignment relationship. The hierarchical control execution module is used to perform hierarchical control response processing based on the set of control commands, record execution feedback, and update the electrical state vector.

[0008] Optionally, the data preprocessing includes endpoint detection, noise reduction, acoustic feature extraction, and voice event tagging of the elderly's voice data to generate a voice feature sequence and a voice event tag set, wherein the voice event tag set includes voice pause tags, repeated voice segment tags, and corrected voice segment tags. The data preprocessing also includes encoding the status data of kitchen appliances to generate appliance status vectors, including operating status, executable actions, and safety level.

[0009] Optionally, the model building module includes: The Mamba encoder, Mamba decoder, selective state-space model, and connection-time classification monotonic alignment mechanism in the Speech-Mamba model are invoked. Based on the selective state space model, the original single-condition parameter generation structure involving only speech features is transformed into a dual-condition parameter generation structure involving both speech features and appliance state, thus constructing the appliance state condition unit. Based on the Mamba encoder, the original single encoded state organization structure is transformed into an acoustic real-time state organization structure and a control intent-maintaining state organization structure. Event-driven state update paths and state fusion paths are set, and discontinuous intent-maintaining units are constructed. Based on Mamba decoder, the original single text decoding structure is transformed into a text decoding structure and action decoding structure in a shared decoding state, and an action isomorphic decoding unit is constructed. After the action isomorphic decoding unit, a new frame action verification unit is added. Based on the connection temporal classification monotonic alignment mechanism, the binary alignment structure between the original speech feature frame and the text tag is expanded into a three-level alignment structure between the speech feature frame, the text tag and the action field. A kitchen control Mamba model is constructed based on the appliance state condition unit, the intermittent intent holding unit, the action isomorphic decoding unit, and the frame action verification unit.

[0010] Optionally, the state condition encoding module includes: According to the order of speech frames in the speech feature sequence, the electrical appliance state vector is matched with each speech frame, the operating state is converted into an operating state embedding, the executable actions are encoded to generate action codes, and the safety level is converted into a safety level code. The speech features and running states of each speech frame are embedded, spliced, linearly mapped, and nonlinearly activated to generate step size adjustment coefficients. The step size parameters corresponding to each speech frame are adjusted, and the discrete state transition relationship corresponding to each speech frame is determined. The speech features of each speech frame are concatenated and linearly mapped with the action code to generate the input mapping parameters corresponding to each speech frame. An input mapping mask is generated according to the executable action. The input mapping parameters are multiplied element by element with the input mapping mask to obtain the adjusted input mapping parameters. The speech features of each speech frame are concatenated with the security level code, linearly mapped and normalized to generate output mapping weights. The output mapping parameters corresponding to each speech frame are multiplied element by element with the output mapping weights to obtain the adjusted output mapping parameters. Based on the order of speech frames, state-space coding is performed by combining discrete state transition relationships, adjusted input mapping parameters, and adjusted output mapping parameters to generate the current state output. Based on the current state output corresponding to each voice frame, an associated voice representation of the appliance's state is generated.

[0011] Optionally, the intermittent intent-maintaining module includes: Based on the discontinuous intent preservation unit, the electrical appliance state-related speech representation is divided into continuous speech segments according to the position of the speech event tag set, and linear mapping is performed to generate the acoustic instantaneous state of the current continuous speech segment. When the first consecutive speech segment arrives, a control intent hold state is established, and the acoustic real-time state of the first consecutive speech segment is written into the control intent hold state. When subsequent consecutive speech segments arrive, the control intent hold state updated by the previous consecutive speech segment is used as the control intent hold state corresponding to the current consecutive speech segment. Read the set of voice event markers, determine the voice pause position, repeated voice segment position and corrected voice segment position in the voice representation associated with the electrical appliance status, and update the control intention hold state according to the voice event type corresponding to each position; The acoustic instantaneous state and the updated control intent hold state are weighted and summed to generate an age-appropriate speech intent representation.

[0012] Optionally, the control action decoding module includes: The age-appropriate speech intent representation input action isomorphic decoding unit is constructed to build a shared decoding state, which is simultaneously provided to the text decoding structure and the action decoding structure; The text decoding structure performs a linear mapping of the shared decoding state at the current decoding moment to the text markers, generates the text prediction score corresponding to each text marker, normalizes it, and selects the text marker with the largest value as the text marker at the current decoding moment to generate the speech text sequence. Based on the executable actions in the electrical appliance state vector, the action decoding structure performs a linear mapping of control actions to the shared decoding state according to the number of control action categories, generates a predicted score corresponding to each control action, normalizes it, and selects the control action with the largest value as the control action at the current decoding moment. The action decoding structure performs a linear mapping on the shared decoding state according to the number of electrical object categories, the number of action intensity categories, and the number of execution timing categories, respectively, and generates electrical object prediction scores, action intensity prediction scores, and execution timing prediction scores in sequence. These scores are then normalized, and the category with the largest value is selected to obtain the electrical object, action intensity, and execution timing at the current decoding moment. Determine the confirmation level at the current decoding moment based on the security level in the electrical appliance state vector; Combine the appliance object, control action, action intensity, execution timing, and confirmation level in the order of the fields to generate the kitchen control action field; After the text decoding structure generates a decoding end marker, the recursive update of the shared decoding state is stopped, the kitchen control action fields are arranged, and a kitchen control action sequence is generated.

[0013] Optionally, the frame action verification module includes: In the frame action verification unit, the text tagging results are calculated frame by frame according to the arrangement order of the feature frames in the speech feature sequence; Based on the original feature frame positions retained in the frame-by-frame text tagging results, read the consecutive feature frames corresponding to each text tag before merging, determine the start feature frame and the end feature frame, and establish the correspondence between each text tag in the speech text sequence and the start and end feature frames; Read the text start position and text end position corresponding to each kitchen control action in the kitchen control action sequence, determine the start feature frame and end feature frame of each kitchen control action, and obtain the feature frame interval corresponding to each kitchen control action. Read the text tags corresponding to the appliance object, control action, action intensity and execution timing in each kitchen control action, and determine the feature frame interval corresponding to each field based on the start feature frame and end feature frame of each text tag. Combine the feature frame interval corresponding to the same kitchen control action and the category name of the field to determine the kitchen control actions with complete field alignment and those with abnormal field alignment. Record the text start position, text end position, start feature frame, end feature frame, and the judgment results of field alignment integrity and field alignment abnormality corresponding to each kitchen control action, and generate frame-level alignment relationship. For kitchen control actions with complete field alignment, the control actions are compared with the executable actions in the appliance state vector to generate control instructions to be executed and control instructions to be confirmed. Based on kitchen control actions with field alignment anomalies, synchronously construct control commands that are prohibited from execution; The control instructions to be executed, the control instructions to be confirmed, and the control instructions to be prohibited from execution are grouped into a set of control instructions.

[0014] Optionally, the hierarchical control execution module includes: Read the control command to be executed, send it to the kitchen appliance corresponding to the appliance object, and complete the status adjustment; Read the control command to be confirmed, convert the appliance object and control action into a voice confirmation prompt and play it, collect the elderly's confirmation voice, and determine whether to execute the control action of the corresponding kitchen appliance; Read the control command that prohibits execution, do not send control actions to the kitchen appliance corresponding to the appliance object, and maintain the current operating state of the kitchen appliance; Record the execution feedback after the instruction is executed; Update the operating state and executable actions in the electrical appliance state vector based on the execution feedback; Read the fault status, protection status and alarm status of kitchen appliances after execution, and update the safety level.

[0015] The beneficial effects of this invention are: First, this invention performs state-space encoding on the speech feature sequence based on the electrical appliance state vector, and uses a discontinuous intent preservation unit to process speech pauses, repeated speech segments, and corrected speech segments. It can maintain and update control intents even when the elderly express themselves discontinuously, repeat content, or make corrections midway, reducing the problems of intent interruption caused by pauses, instruction repetition caused by repeated content, and failure to replace corrected content, thereby improving the accuracy of age-friendly speech intent recognition.

[0016] Secondly, this invention generates voice text sequences and kitchen control action sequences synchronously in a shared decoding state through an action isomorphic decoding unit, and limits the control actions according to the executable actions in the appliance state vector, so that the appliance object, control action, action intensity, execution timing and confirmation level can be formed synchronously with the voice text, reducing the probability of text recognition results and kitchen control actions being separated and incorrect combination of action fields, and improving the completeness and reliability of kitchen control action generation.

[0017] Furthermore, this invention utilizes a frame action verification unit to perform frame-level alignment of feature frames, voice text sequences, and kitchen control action sequences. It then combines the frame-level alignment relationship, executable actions, and confirmation levels to generate control instructions to be executed, control instructions to be confirmed, and control instructions to be prohibited from execution. This allows for the interception of control actions that lack voice evidence, have abnormal field correspondences, or do not conform to the current state of the appliance. This prevents high-risk actions from being directly executed and improves the safety and traceability of voice control for kitchen appliances. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an age-friendly smart control method for kitchen appliances based on voice recognition proposed in this invention. Figure 2 This is a flowchart of how the electrical state condition unit and the intermittent intent holding unit in this invention process the speech feature sequence and generate an age-appropriate speech intent representation. Figure 3 This is a flowchart illustrating how the action isomorphic decoding unit in this invention synchronously generates a speech text sequence and a kitchen control action sequence, and how the frame action verification unit performs frame-level alignment and control command classification. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A voice recognition-based intelligent control system for age-friendly kitchen appliances includes: The data acquisition and processing module is used to collect elderly voice data and kitchen appliance status data in the kitchen environment, perform data preprocessing, perform endpoint detection, noise reduction, acoustic feature extraction and voice event tagging on the elderly voice data, generate voice feature sequence and voice event tag set. The voice event tag set consists of voice pause tag, repeated voice segment tag and corrected voice segment tag. The module encodes the status data of kitchen appliances to generate appliance status vector. The appliance status vector consists of operating status, executable action and safety level. The model building module is used to build a kitchen control Mamba model based on the Speech-Mamba model. The kitchen control Mamba model includes an appliance state condition unit, an intermittent intent holding unit, an action isomorphic decoding unit, and a frame action verification unit. The state condition coding module is used to introduce the speech feature sequence and the appliance state vector into the state space coding process based on the appliance state condition unit. It adjusts the step size parameter corresponding to each speech frame according to the operating state in the appliance state vector, adjusts the input mapping parameter corresponding to each speech frame according to the executable action in the appliance state vector, and adjusts the output mapping parameter corresponding to each speech frame according to the safety level in the appliance state vector. Based on the adjusted step size parameter, input mapping parameter, and output mapping parameter, the speech feature sequence is state space encoded to generate an appliance state-related speech representation. The discontinuous intent holding module is used in the discontinuous intent holding unit to form an acoustic real-time state based on the electrical state associated with the speech representation, and to generate a control intent holding state based on the processed acoustic real-time state. It determines the speech pause position, repeated speech segment position, and corrected speech segment position based on the speech event tag set. At the speech pause position, it stops writing the current speech content to the control intent holding state. At the repeated speech segment position, it updates the corresponding state component in the control intent holding state based on the acoustic real-time state. At the corrected speech segment position, it replaces the corresponding state component in the control intent holding state based on the acoustic real-time state, and retains the state component that has not been replaced. It merges the acoustic real-time state with the updated control intent holding state to generate an age-appropriate speech intent representation. The control action decoding module is used to call the action isomorphic decoding unit to decode based on the age-appropriate voice intent representation. It limits the control action according to the executable actions in the appliance state vector, determines the confirmation level according to the safety level in the appliance state vector, and synchronously generates the voice text sequence and the kitchen control action sequence. The kitchen control action sequence consists of appliance object, control action, action intensity, execution timing and confirmation level, and feeds back the appliance object, control action, action intensity, execution timing and confirmation level obtained at the current decoding time to the next decoding time. The frame action verification module is used to perform frame-level alignment of feature frames, speech text sequences and kitchen control action sequences in speech feature sequences based on the frame action verification unit, generate frame-level alignment relationships, and convert kitchen control action sequences into control instruction sets according to frame-level alignment relationships and confirmation levels. The control instruction set includes control instructions to be executed, control instructions to be confirmed, and control instructions to be prohibited from execution. The control instructions in the control instruction set carry the corresponding appliance object, control action, action intensity and execution timing. The hierarchical control execution module is used to control the kitchen appliances corresponding to the appliance objects in the control commands to be executed, the control actions, the intensity of the actions, and the timing of the execution. It outputs voice confirmation prompts for the appliance objects in the control commands to be confirmed, maintains the operating status of the corresponding kitchen appliances for the appliance objects in the control commands to be prohibited from execution, records execution feedback, and updates the appliance status vector using the execution feedback.

[0021] In this embodiment, the data preprocessing includes endpoint detection, noise reduction, acoustic feature extraction, and voice event tagging of the elderly's voice data, generating a voice feature sequence and a voice event tag set. The voice event tag set includes voice pause tags, repeated voice segment tags, and corrected voice segment tags. Specifically, the elderly person's speech data is segmented into frames, and the short-time energy and zero-crossing rate of each speech frame are calculated. The position where the short-time energy continuously increases relative to the preceding non-speech frame and the zero-crossing rate continuously changes is determined as the speech start point, and the position where the short-time energy falls back to the level of the subsequent non-speech frame and the zero-crossing rate remains stable is determined as the speech end point. The effective speech segment between the speech start point and the speech end point is extracted. The environmental noise spectrum is estimated based on the non-speech segments before the speech start point and after the speech end point. The environmental noise spectrum is subtracted from the spectrum of the effective speech segment, and the processed spectrum is inversely transformed to obtain the denoised speech segment. The denoised speech segment is then subjected to pre-emphasis, framing, windowing, Fourier transform, Mel filtering, logarithmic operation, and discrete cosine transform in sequence to extract the acoustic features of each speech frame. The acoustic features of each speech frame are arranged in chronological order to generate a speech feature sequence. The interval position formed by consecutive silence frames between adjacent effective speech segments is identified, and the first silence frame in the consecutive silence frame is identified as the interval position between the two segments. The position of the first speech frame and the position of the last speech frame are recorded as the pause start frame position and the pause end frame position, respectively, and a speech pause marker carrying the pause start frame position and the pause end frame position is generated. The syllable arrangement and acoustic feature changes of adjacent speech segments are compared. When the syllable arrangement and acoustic feature changes are consistent, the position of the first speech frame and the position of the last speech frame of the next speech segment are recorded as the repeat start frame position and the repeat end frame position, respectively, and a repeat speech segment marker carrying the repeat start frame position and the repeat end frame position is generated. Semantic negation words, instruction cancellation words and content modification words in valid speech segments are matched for keywords. After matching the corresponding keywords, the valid speech segment after the keyword is determined as the modified speech segment. The position of the first speech frame and the position of the last speech frame of the modified speech segment are recorded as the modification start frame position and the modification end frame position, respectively, and a modified speech segment marker carrying the modification start position and the modification end frame position is generated. The data preprocessing also includes encoding the status data of kitchen appliances to generate appliance status vectors, which include operating status, executable actions, and safety levels.

[0022] In this embodiment, the model building module includes: The Speech-Mamba model is an end-to-end speech recognition model for automatic recognition of long speech sequences. It introduces the Mamba selective state-space model into the speech encoding and decoding structure. By selectively retaining, transmitting, and forgetting sequence information based on the input speech content, it extracts short-term acoustic features and long-distance contextual information in the speech. This model is mainly used to convert input speech into text sequences and improve the modeling ability of long speech context dependencies with near-linear sequence computational complexity. The Mamba encoder, Mamba decoder, selective state space model, and connection-temporal classification monotonic alignment mechanism in the Speech-Mamba model are invoked, while the Mamba block, normalization structure, attention structure, and residual connection in the Mamba encoder and Mamba decoder are retained as the foundation for building the kitchen control Mamba model. Based on the selective state space model, the original single-condition parameter generation structure involving only speech features is transformed into a dual-condition parameter generation structure involving both speech features and appliance state. Appliance state condition units are constructed in the dual-condition parameter generation structure and connected to the state space coding backbone of the selective state space model. Based on the Mamba encoder, the original single encoded state organization structure is transformed into an acoustic instantaneous state organization structure and a control intent-maintaining state organization structure. An event-driven state update path is set between the acoustic instantaneous state organization structure and the control intent-maintaining state organization structure, and a state fusion path is set after them. The output of the acoustic instantaneous state organization structure is connected to the initial write position, state accumulation position, and state replacement position of the control intent-maintaining state organization structure. The outputs of the acoustic instantaneous state organization structure and the control intent-maintaining state organization structure are connected to the state fusion path to construct a discontinuous intent-maintaining unit. Based on Mamba decoder, the original single text decoding structure is transformed into a text decoding structure and action decoding structure in a shared decoding state. A synchronous decoding connection is established between the text decoding structure and the action decoding structure, and the two are jointly constructed as an action isomorphic decoding unit. The output of the discontinuous intent holding unit is connected to the input of the action isomorphic decoding unit, and the action feedback path of the action isomorphic decoding unit is connected to the recursive update position of the shared decoding state. After the action isomorphic decoding unit, a new frame action verification unit is added. Based on the connection time classification monotonic alignment mechanism, the binary alignment structure between the original speech feature frame and the text tag is expanded into a three-level alignment structure between the speech feature frame, the text tag and the action field. The speech feature sequence is input into the frame action verification unit, and the text output and action output of the action isomorphic decoding unit are connected to the frame action verification unit. Based on the appliance state condition unit, the discontinuous intent preservation unit, the action isomorphic decoding unit, and the frame action verification unit, the normalized structure, attention structure, residual connection, and joint training framework in the Speech-Mamba model are retained. The combination of the modified unit and the retained structure is completed to construct the kitchen control Mamba model. During the training of the kitchen control Mamba model, voice samples from elderly people's kitchens, corresponding kitchen appliance status data, voice text annotations, voice event location annotations, and kitchen control action annotations are collected to form a training sample set. The voice event location annotations include voice pause positions, repeated voice segment positions, and corrected voice segment positions. The kitchen control action annotations include appliance objects, control actions, action intensity, execution timing, and confirmation levels. Data preprocessing is performed on the training sample set to obtain training voice feature sequences, training voice event tag sets, and training appliance state vectors, and these three are input into the kitchen control Mamba model. During training, the appliance state condition unit generates a training appliance state-related speech representation based on the training speech feature sequence and the training appliance state vector; the intermittent intent preservation unit generates a training aging-friendly speech intent representation based on the training speech event tag set; the action isomorphic decoding unit synchronously generates a training speech text sequence and a training kitchen control action sequence based on the training aging-friendly speech intent representation; and the frame action verification unit generates a training frame-level alignment relationship based on the training speech feature sequence, the training speech text sequence, and the training kitchen control action sequence. The loss calculation consists of text recognition loss, action field loss, event retention loss, and frame action alignment loss. The text recognition loss is calculated based on the cross-entropy between the predicted probability of each text tag in the training speech-text sequence and the corresponding text tag in the speech-text annotation. The action field loss calculates the cross-entropy between the predicted results of the appliance object, control action, action intensity, execution timing, and confirmation level and the corresponding fields in the kitchen control action annotation, and then sums the cross-entropy of the five fields. The event retention loss is calculated based on the speech event location annotation. At the speech pause location, the difference in control intention retention state before and after the pause is compared. At the repeated speech segment location, the consistency between the corresponding field of the repeated segment and the corresponding field in the control intention retention state is compared. At the corrected speech segment location, the cross-entropy between the predicted result of the corrected field and the corresponding labeled field of the corrected speech segment is compared. The frame action alignment loss is calculated based on the monotonic alignment method of connection temporal classification. The probabilities of all alignment paths that conform to the temporal order between the feature frames in the speech feature sequence and the speech-text annotation are accumulated, and then the negative logarithm of the accumulated probability is taken as the frame action alignment loss. The AdamW optimizer is used to update the parameters of the kitchen control Mamba model. The initial learning rate of the Speech-Mamba model inherited parameters is set to 0.00001, and the initial learning rate of the appliance state condition unit, the intermittent intent preservation unit, the action isomorphic decoding unit, and the frame action verification unit is also set to 0.0001. The batch size is set to 16. The total validation loss is calculated on the validation sample set after each training epoch. When the decrease in the total validation loss is less than 0.0001 for 10 consecutive training epochs, or when the training epochs reach 100, training is stopped, and the model parameters corresponding to the minimum total validation loss are saved to obtain the trained kitchen control Mamba model.

[0023] In this embodiment, the state condition encoding module includes: According to the order of speech frames in the speech feature sequence, the electrical appliance state vector is matched with each speech frame, the operating state is converted into an operating state embedding, the executable actions are encoded according to the electrical appliance object, control action, action intensity and execution timing to generate action codes, and the safety level is converted into a safety level code. The speech features of each speech frame are embedded with the running state and concatenated. The concatenation result is linearly mapped and nonlinearly activated to generate step size adjustment coefficients. The state matrix trained in the selective state space model is obtained. The step size parameters corresponding to each speech frame are adjusted according to the step size adjustment coefficients. The discrete state transition relationship corresponding to each speech frame is determined according to the adjusted step size parameters and the state matrix. Specifically, the original step size parameters corresponding to each speech frame are multiplied element by element with the step size adjustment coefficients, and the product is positiveized to obtain the adjusted step size parameters. The adjusted step size parameters are then subjected to element-wise operations after dimensional expansion of the state matrix. Matrix exponentiation is performed on the operation results to obtain the discrete state transition relationship corresponding to each speech frame. The speech features of each speech frame are concatenated with the action code. The concatenation result is linearly mapped to generate the input mapping parameters corresponding to each speech frame. An input mapping mask is generated according to the executable action. The mask value at the position corresponding to the executable action is set to one, and the mask value at the other positions is set to zero. The input mapping parameters are multiplied element by element with the input mapping mask to obtain the adjusted input mapping parameters. The speech features of each speech frame are concatenated with the security level code. The concatenation result is linearly mapped and normalized to generate output mapping weights. The output mapping parameters corresponding to each speech frame are multiplied element by element with the output mapping weights to obtain the adjusted output mapping parameters. The output mapping parameters refer to the input-related mapping parameters generated by the selective state space model based on the speech features of the current speech frame, which are used to project the current state into the current state output. The position of each parameter of the output mapping parameters controls the degree of contribution of the corresponding state component in the current state to the current state output. Based on the order of speech frames, state-space coding is performed by combining discrete state transition relationships, adjusted input mapping parameters, and adjusted output mapping parameters. Specifically, the historical state corresponding to the first speech frame is initialized as a zero vector. For the current speech frame, a matrix multiplication operation is performed between the historical state corresponding to the previous speech frame and the discrete state transition relationship corresponding to the current speech frame to obtain the historical state retention result. A matrix multiplication operation is performed between the speech features of the current speech frame and the adjusted input mapping parameters to obtain the input writing result of the current speech frame. The historical state retention result and the input writing result are added element by element to generate the current state. A matrix multiplication operation is performed between the adjusted output mapping parameters and the current state, and the product results corresponding to the same output channel are summed to generate the current state output. The current state is used as the historical state when updating the state of the next speech frame, until the state-space coding of all speech frames in the speech feature sequence is completed. The current status output corresponding to each voice frame is obtained sequentially, and the current status output is arranged according to the time order of the voice frames to generate an appliance status associated voice representation.

[0024] In this embodiment, the intermittent intent preservation module includes: Based on the discontinuous intent preservation unit, the electrical appliance state-related speech representation is divided into continuous speech segments according to the position of the speech event marker set. The electrical appliance state-related speech representation corresponding to the current continuous speech segment is linearly mapped to generate electrical appliance object state components, control action state components, action intensity state components, and execution timing state components respectively. The electrical appliance object state components, control action state components, action intensity state components, and execution timing state components are concatenated according to the preset field order to form the acoustic real-time state of the current continuous speech segment. When the first consecutive speech segment arrives, a control intent hold state consistent with the acoustic instantaneous state dimension is established and initialized to a zero vector. The acoustic instantaneous state of the first consecutive speech segment is written into the control intent hold state according to the electrical object state component, control action state component, action intensity state component, and execution timing state component, so that the control intent hold state forms electrical object candidates, control action candidates, action intensity candidates, and execution timing candidates, respectively. When subsequent consecutive speech segments arrive, the control intent hold state updated by the previous consecutive speech segment is used as the control intent hold state corresponding to the current consecutive speech segment. Read the pause start frame position and pause end frame position carried by the voice pause mark, the repeat start frame position and repeat end frame position carried by the repeat voice segment mark, and the correction start frame position and correction end frame position carried by the correction voice segment mark from the voice event mark set. Determine the voice pause position, repeat voice segment position and correction voice segment position in the electrical state associated voice representation respectively, and update the control intention hold state according to the voice event type corresponding to each position. Specifically, at the speech pause position, the writing of the electrical appliance state associated speech representation of the corresponding silence frame into the control intent hold state is stopped. The electrical appliance object candidates, control action candidates, action intensity candidates and execution timing candidates formed before the speech pause position are retained, and the retained control intent hold state is passed to the continuous speech segment after the speech pause position. At the location of a repeated speech segment, the appliance object state component corresponding to the repeated speech segment is matched with the appliance object candidate in the control intent holding state, the control action state component is matched with the control action candidate, the action intensity state component is matched with the action intensity candidate, and the execution timing state component is matched with the execution timing candidate. The matched state components are added element by element to the corresponding candidate, and the addition result is normalized. The normalized result is written back to the appliance object candidate, control action candidate, action intensity candidate, and execution timing candidate, respectively. The state components that do not match are not written, so that the repeated speech segment serves as a repeated confirmation of the existing appliance control intent and does not form a new independent appliance control intent. At the location of the corrected speech segment, the electrical object, control action, action intensity and execution timing that have changed are determined based on the acoustic real-time state of the corrected speech segment. The changed field is identified as the corrected field. The state component corresponding to the corrected field is used to replace the candidate with the same name in the control intention holding state, and the remaining candidates in the control intention holding state that have not been identified as corrected fields are retained, thus obtaining the updated control intention holding state. Based on the speech event type corresponding to the current continuous speech segment, determine the fusion weight of the acoustic instant state and the updated control intent hold state, and perform a weighted sum of the acoustic instant state and the updated control intent hold state to generate an age-appropriate speech intent representation corresponding to the current continuous speech segment. The determination of fusion weights includes: converting the speech event type corresponding to the current continuous speech segment into an event type code; concatenating the event type code, the acoustic instantaneous state, and the updated control intent hold state; performing a linear mapping on the concatenation result to generate two weight scores; performing Softmax normalization on the two weight scores to obtain the fusion weight of the acoustic instantaneous state and the fusion weight of the control intent hold state, respectively; and the sum of the two fusion weights is one. When the current continuous speech segment corresponds to a speech pause marker, the fusion weight of the acoustic instantaneous state is set to zero, and the fusion weight of the control intent hold state is set to one. After endpoint detection determines that the elderly person's speech data has reached the speech endpoint, it outputs the age-friendly speech intent representations corresponding to all continuous speech segments and clears the control intent hold state.

[0025] In this embodiment, the control action decoding module includes: The age-appropriate speech intent representation is input to the action isomorphic decoding unit, and the zero vector consistent with the dimension of the shared decoding state is set as the initial shared decoding state. At the first decoding moment, according to the number of the preset decoding start marker in the text marker vocabulary, the corresponding embedding vector is read from the embedding matrix of the text embedding layer to generate the text input representation of the first decoding moment. The zero vector consistent with the dimension of the action feedback representation is set as the action feedback representation of the first decoding moment. The text input representation, action feedback representation and age-appropriate speech intent representation are concatenated, and the concatenation result is linearly mapped to generate the decoding input of the first decoding moment. The decoding input of the first decoding moment and the initial shared decoding state are input to the Mamba decoding structure for state update to generate the shared decoding state of the first decoding moment. The shared decoding state of the first decoding moment is provided to both the text decoding structure and the action decoding structure. The preset decoding start marker refers to the sequence start marker set separately when constructing the text marker vocabulary. It is used to indicate that the current decoding process has not yet generated speech text markers. The decoding start marker does not correspond to the actual text content in the elderly speech data, but is written into the model configuration as a fixed special marker in the text marker vocabulary. The text marker vocabulary is a pre-established table that records all text markers that can be output by the text decoding structure and their unique numbers. The text markers refer to the discrete markers corresponding to the characters, words or sub-words used to compose the speech recognition text in the text marker vocabulary. In subsequent decoding moments, based on the number of the text token generated in the previous decoding moment in the text token vocabulary, the corresponding embedding vector is read from the embedding matrix of the text embedding layer to generate the text input representation for the current decoding moment. The action feedback representation generated in the previous decoding moment is read. The text input representation for the current decoding moment, the action feedback representation for the previous decoding moment, and the age-appropriate speech intent representation are concatenated. The concatenation result is linearly mapped to generate the decoding input for the current decoding moment. The decoding input for the current decoding moment and the shared decoding state for the previous decoding moment are input into the Mamba decoding structure for state update to generate the shared decoding state for the current decoding moment. The shared decoding state for the current decoding moment is simultaneously provided to the text decoding structure and the action decoding structure. The text decoding structure performs a linear mapping of text tags to the shared decoding state at the current decoding moment according to the number of text tags in the text tag vocabulary, generates a text prediction score corresponding to each text tag in the text tag vocabulary, normalizes the text prediction score, selects the text tag with the largest value in the normalization result as the text tag at the current decoding moment, arranges the text tags generated at each decoding moment in chronological order, deletes the decoding start tag and decoding end tag, and converts the remaining text tags into corresponding characters, words or sub-words in sequence to generate a speech text sequence; Based on the executable actions in the appliance state vector, the control actions that the kitchen appliances can and cannot perform are determined. The action decoding structure performs a linear mapping of control actions to the shared decoding state at the current decoding moment according to the number of control action categories, generating a predicted score for each control action. The minimum value among the predicted scores corresponding to the control actions that the kitchen appliances can perform is selected, and 1 is subtracted from the minimum value. The resulting value is then used to replace the predicted score corresponding to the control actions that the kitchen appliances cannot perform. The replaced predicted scores are normalized, and the control action with the largest value in the normalized result is selected as the control action at the current decoding moment. The action decoding structure linearly maps the shared decoding state at the current decoding moment according to the number of electrical object categories, the number of action intensity categories, and the number of execution timing categories. It sequentially generates electrical object prediction scores corresponding to each electrical object category, action intensity prediction scores corresponding to each action intensity category, and execution timing prediction scores corresponding to each execution timing category. It normalizes each type of prediction score and selects the category with the largest value among the normalization results to obtain the electrical object, action intensity, and execution timing at the current decoding moment. Based on the safety level in the appliance state vector, determine the confirmation level at the current decoding moment. When the safety level indicates that the current control action can be executed directly, the confirmation level is set to "direct execution"; when the safety level indicates that the current control action requires user confirmation before execution, the confirmation level is set to "confirmation required"; when the safety level indicates that the current control action is not allowed to be executed, the confirmation level is set to "execution prohibited". Replace the confirmation level in the kitchen control action field at the current decoding moment with the confirmation level determined based on the safety level. Combine the appliance object, control action, action intensity, execution timing, and confirmation level in the order of the fields to generate the kitchen control action field at the current decoding moment; Record the position of the text markers generated at each decoding moment in the speech text sequence, and associate the appliance object, control action, action intensity, execution timing and confirmation level formed at the same decoding moment with the position of the corresponding text marker; when multiple consecutive text markers correspond to the same set of appliance objects, control actions, action intensity, execution timing and confirmation level, determine the position of the first text marker and the position of the last text marker as the text start position and text end position of the kitchen control action field of that set, respectively; The electrical object, control action, action intensity, execution timing, and confirmation level formed at the current decoding moment are converted into field codes respectively. The field codes are concatenated according to the field order of electrical object, control action, action intensity, execution timing, and confirmation level to generate the action feedback representation at the current decoding moment. The action feedback representation at the current decoding moment is then provided to the next decoding moment. After the text decoding structure generates a decoding end marker, the recursive update of the shared decoding state is stopped. The kitchen control action fields formed at each decoding time are arranged in chronological order to generate a kitchen control action sequence. The voice-text sequence generated synchronously on the same decoding timeline as the kitchen control action sequence is then output.

[0026] In this embodiment, the frame action verification module includes: In the frame action verification unit, according to the arrangement order of feature frames in the speech feature sequence, the prediction score of each feature frame corresponding to each text tag and blank tag in the text tag vocabulary is calculated, and the text tag or blank tag with the largest prediction score is selected as the output tag of the corresponding feature frame; the output tags are arranged according to the arrangement order of feature frames, and blank tags are deleted while retaining the original feature frame position of each output tag, and the same text tags that appear repeatedly are merged into 1 text tag, so as to obtain the frame-by-frame text tag result that is consistent with the arrangement order of text tags in the speech text sequence; Based on the original feature frame positions preserved in the frame-by-frame text tagging results, read the consecutive feature frames corresponding to each text tag before merging, determine the feature frame where the text tag first appears as the starting feature frame, and determine the feature frame where the text tag last appears as the ending feature frame, and establish the correspondence between each text tag in the speech text sequence and the starting and ending feature frames; Read the text start position and text end position corresponding to each kitchen control action in the kitchen control action sequence, determine the text markers that are continuously arranged between the text start position and the text end position, when a kitchen control action corresponds to multiple consecutive text markers, determine the start feature frame of the first text marker as the start feature frame of the kitchen control action, and determine the end feature frame of the last text marker as the end feature frame of the kitchen control action, so as to obtain the feature frame interval corresponding to the kitchen control action. When a speech text sequence contains multiple kitchen control actions, the speech text sequence is divided into continuous text parts corresponding to each kitchen control action according to the start and end positions of the text corresponding to each kitchen control action. The feature frame interval corresponding to each kitchen control action is determined according to the start feature frame of the first text marker and the end feature frame of the last text marker in each continuous text part. When the text positions corresponding to adjacent kitchen control actions do not overlap, the corresponding feature frame intervals are retained. When the text positions corresponding to adjacent kitchen control actions overlap, the overlapping kitchen control actions are identified as field alignment anomalies. Read the text tags corresponding to the appliance object, control action, action intensity, and execution timing in each kitchen control action, and determine the feature frame interval corresponding to each field based on the start and end feature frames of each text tag. When the appliance object, control action, action intensity, and execution timing all have corresponding text tags, the feature frame intervals corresponding to each field are all located within the feature frame interval corresponding to the same kitchen control action, and the text content obtained by converting the text tags corresponding to each field according to the text tag vocabulary is consistent with the category name of the corresponding field, the kitchen control action is determined to be field-aligned and complete. When any field in the appliance object, control action, action intensity, and execution timing lacks a corresponding text tag, the feature frame intervals corresponding to different fields fall into the text position ranges corresponding to different kitchen control actions, the fields corresponding to different kitchen control actions occupy overlapping feature frame intervals that cannot be separated according to the text start position and text end position, or the text content corresponding to any field is inconsistent with the category name of the corresponding field, the kitchen control action is determined to be a field alignment anomaly. Record the text start position, text end position, start feature frame, end feature frame, and the judgment results of field alignment integrity and field alignment abnormality corresponding to each kitchen control action, and generate frame-level alignment relationship. For kitchen control actions with complete field alignment, the control action is compared with the executable actions in the appliance state vector. When the control action is an executable action and the confirmation level is direct execution, the appliance object, control action, action intensity, and execution timing are combined into a control instruction to be executed. When the control action is an executable action and the confirmation level is confirmation required, the appliance object, control action, action intensity, and execution timing are combined into a control instruction to be confirmed. When a kitchen control action is determined to be a field alignment anomaly, a control action that does not belong to an executable action in the appliance state vector, and a confirmation level that prohibits execution, the appliance object, control action, action intensity, and execution timing are combined into a control instruction that prohibits execution. The control instructions to be executed, the control instructions to be confirmed, and the control instructions to be prohibited from execution are grouped into a set of control instructions.

[0027] In this embodiment, the hierarchical control execution module includes: Read the appliance object, control action, action intensity, and execution timing from the control instruction to be executed. When the execution timing arrives, send the control action and action intensity to the kitchen appliance corresponding to the appliance object, and the kitchen appliance will complete the state adjustment according to the control action and action intensity. Read the appliance object and control action in the control command to be confirmed, convert the appliance object and control action into a voice confirmation prompt containing the name of the kitchen appliance and the operation to be performed, and play it. Collect the confirmation voice input by the elderly in response to the voice confirmation prompt, and use the confirmation voice as new elderly voice data to re-perform endpoint detection, noise reduction, acoustic feature extraction, voice event marking, intent recognition and control action decoding. Determine whether to execute the control action of the corresponding kitchen appliance based on the regenerated control command. Read the appliance objects and control actions in the prohibited control instructions, do not send control actions to the kitchen appliances corresponding to the appliance objects, maintain the current operating status of the kitchen appliances, and record the prohibited control actions; if the current elderly voice data has not been corrected, do not repeat the prohibited control actions based on the same voice content. After the pending control command is executed, the pending confirmation control command is confirmed, and the prohibition control command is intercepted, record whether the control command has been executed, the operating status of the kitchen appliance after execution, the actual intensity of the action, and the actual timing of execution to form execution feedback. Replace the running status in the appliance status vector with the running status in the execution feedback, read the functional availability of the kitchen appliance after execution, write the control actions that can be executed normally into the executable actions in the appliance status vector, and delete the control actions that cannot be executed. Read the fault status, protection status, and alarm status of the kitchen appliance after execution. When the kitchen appliance does not have a fault, protection, or alarm, update the safety level to allow direct execution. When the kitchen appliance needs to perform a control action after user confirmation, update the safety level to require user confirmation before execution. When the kitchen appliance has a fault, protection, or alarm that prohibits the continued execution of the corresponding control action, update the safety level to disallow execution. The updated operating status, executable actions, and security level are combined into an updated appliance state vector, which is then used for the next recognition and control processing of elderly voice data. The updated appliance state vector is not used to modify the already completed voice decoding results.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to an age-friendly kitchen appliance control scenario consisting of a gas stove, induction cooker, rice cooker, microwave oven, steam oven, and range hood. Elderly users participating in the test controlled the kitchen appliances via natural voice commands during daily cooking. Voice commands covered appliance start / stop, power adjustment, temperature adjustment, timer setting, delay execution, and status inquiry. The test focused on situations where elderly users are prone to pauses, repetitions, temporary corrections, incomplete sentences, and multiple control actions within the same voice command, to verify the accuracy and safety of the system in generating kitchen control actions under discontinuous expression conditions.

[0029] During the test, voice data of the elderly and status data of kitchen appliances were collected to construct voice feature sequences, voice event tag sets, and appliance status vectors.

[0030] In practical applications, the speech feature sequence and the appliance state vector are jointly input into the kitchen control Mamba model. The appliance state condition unit performs state-space encoding on the speech feature sequence based on the operating status, executable actions, and safety level of the kitchen appliances, generating an appliance state-associated speech representation. This process can retain the range of actions that the kitchen appliances can currently perform while recognizing the speech content, avoiding the system generating control actions that do not match the state of the kitchen appliances based solely on the speech content.

[0031] When the elderly person's speech contains a relatively long pause, the intermittent intent retention unit stops writing the speech content during the pause to the control intent retention state, while retaining the electrical appliance object, control action, action intensity, and execution timing already formed before the pause. After the elderly person continues speaking, the subsequent speech content is combined with the already retained control intent. The test speech "Turn the induction cooker...to low heat" was not prematurely terminated during the pause, and "low heat" after the pause was able to be added to the original control intent.

[0032] When the elderly person repeatedly expresses control commands, the appliance object, control action, action intensity, and timing of the repeated speech segment are matched with the corresponding content in the control intent holding state. Matching content is used to reinforce the existing control intent, preventing the generation of new independent control actions. The test speech "Turn the heat down, turn it down a little" ultimately resulted in a single power-down control action, without creating two repeated control commands.

[0033] When an elderly person temporarily corrects their expression, the corresponding field in the corrected speech segment replaces the original field in the control intention retention state, while the uncorrected fields remain retained. In the test speech "Set the steam oven to 180 degrees, no, set it to 160 degrees," the appliance and control action remain unchanged, but the action intensity is changed from 180 degrees to 160 degrees. In the test speech "Turn on the induction cooker, no, turn on the range hood," the appliance is changed from the induction cooker to the range hood, and the control action remains "turn on."

[0034] The action isomorphic decoding unit, based on age-appropriate speech intent representation, synchronously generates speech text sequences and kitchen control action sequences under shared decoding conditions. The generation of control actions is limited by executable actions in the appliance state vector; unexecutable control actions will not be selected as the current decoding result. For example, the voice command "Turn up the flame of the gas stove that is currently off" will not directly generate an executable flame adjustment control command if the gas stove does not currently have flame adjustment capabilities.

[0035] The frame action verification unit further performs frame-level alignment on the feature frames, speech text sequence, and kitchen control action sequence in the speech feature sequence. After the start and end feature frames corresponding to each text marker are determined, the appliance object, control action, action intensity, and execution timing are mapped to the corresponding feature frame intervals. When each field has corresponding text content, and the feature frame interval corresponding to the field is within the speech range corresponding to the same kitchen control action, the kitchen control action is determined to be field-aligned correctly. When a field lacks a corresponding text marker, different fields fall into conflicting speech ranges, or the text content is inconsistent with the action field, the kitchen control action is determined to be field-aligned incorrectly.

[0036] For kitchen control actions with complete field alignment, executable actions, and a confirmation level of "direct execution," the system generates a control instruction to be executed. For kitchen control actions with complete field alignment, executable actions, and a confirmation level of "required confirmation," the system outputs a voice confirmation prompt containing the appliance object and the control action. The elderly person's voice input in response to the confirmation prompt is then re-entered into the voice processing flow as new elderly person voice data. For kitchen control actions with abnormal field alignment, non-executable actions, or a confirmation level of "prohibited execution," the system generates a control instruction to prohibit execution and maintains the current operating status of the corresponding kitchen appliance.

[0037] This embodiment selected 1200 voice data points from elderly individuals, including 300 continuous speech entries, 260 paused speech entries, 220 repetitive speech segments, 220 corrected speech segments, and 200 speech entries involving multiple control actions. Each voice entry was manually labeled with the appliance object, control action, action intensity, execution timing, and confirmation level. The method of this invention was compared with traditional keyword matching control methods and conventional speech recognition and fixed rule control methods during testing. Traditional keyword matching control methods directly generate control commands based on appliance names and action words in the recognized text; conventional speech recognition and fixed rule control methods first generate complete text and then extract appliance objects and control actions according to fixed rules. All three methods used the same test voice and kitchen appliance status data.

[0038] Table 1. Comparison of the overall performance of voice control in age-friendly kitchen appliances

[0039] As shown in Table 1, the method of this invention achieves 89.1% accuracy in intermittent speech intent recognition, which is 17.3 percentage points higher than the traditional keyword matching control method and 9.5 percentage points higher than the conventional speech recognition and fixed rule control method. The traditional keyword matching control method tends to split the preceding and following speech into two independent segments when the elderly person's pause is long, causing a loss of connection between the appliance and the control action. While conventional speech recognition and fixed rule control methods can recognize complete text, they lack the continuous retention of control intent during pauses, and some speech still exhibits missing fields. The method of this invention stops writing the current speech content at the speech pause position while retaining the control intent state. Subsequent continuous speech segments can continue to update the existing control intent, thus maintaining a relatively stable correspondence between the appliance, control action, action intensity, and execution timing before and after the pause.

[0040] Regarding the accuracy of correcting voice field updates, the method of this invention achieves 87.6%, compared to 64.5% for traditional keyword matching control methods and 72.8% for conventional speech recognition and fixed rule control methods. Both methods tend to retain the numbers, intensity, or appliance names before and after correction, leading to ambiguous final field values. The method of this invention replaces the corresponding fields in the control intent holding state based on the position of the corrected voice segment, updating only the changed appliance object, control action, action intensity, or execution timing. Fields not corrected are retained, thus reducing the problem of field misalignment after the entire control intent is re-recognized.

[0041] In terms of accuracy in generating kitchen control actions, the method of this invention achieves 88.4%, which is 7.7 percentage points higher than conventional speech recognition and fixed-rule control methods. This performance improvement primarily stems from the synchronous generation of the speech text sequence and the kitchen control action sequence under shared decoding conditions, and the integration of executable actions from the appliance state vector during the control action selection process. Comparative methods typically extract control actions after speech text generation; when the text contains repetitions, corrections, or omissions, subsequent fixed rules are prone to generating incorrect combinations of appliance objects and control actions. The method of this invention synchronously generates each action field during the decoding stage, reducing the likelihood of text recognition results and control action results becoming disconnected.

[0042] Regarding the accuracy of multi-control action separation, the method of this invention achieves 86.8%, while the traditional keyword matching control method achieves only 66.2%, and the conventional speech recognition and fixed rule control method achieves 74.9%. In speech containing multiple control actions, the text start and end positions corresponding to each kitchen control action can be further mapped to the corresponding feature frame intervals. The frame action verification unit uses this to distinguish the speech range occupied by different kitchen control actions, reducing the incorrect combination of appliance objects from the previous control action with the action intensity of the next control action. This indicator is lower than the accuracy of single-kitchen control action generation, mainly because some multi-control action speech contains omitted appliance names and continuous rapid expressions, which still increases the difficulty of field separation.

[0043] Regarding the correct interception rate of high-risk actions, the method of this invention achieved 91.2%, not exceeding 93%. This result is 9.8 percentage points higher than traditional keyword matching control methods and 5.6 percentage points higher than conventional speech recognition and fixed rule control methods. The method of this invention does not classify solely based on confirmation level, but simultaneously checks field alignment results, whether the control action is executable, and the confirmation level. When a kitchen control action lacks corresponding voice content, the action field is inconsistent with the text content, or the control action does not conform to the current operating status of the kitchen appliance, the system generates a prohibition on executing the control command. Multiple conditions participate in the classification, reducing missed interceptions caused by relying solely on fixed permission rules.

[0044] In terms of average response time, the method of this invention is 1.08 seconds, which is higher than the 0.72 seconds of the traditional keyword matching control method and the 0.94 seconds of the conventional speech recognition and fixed rule control method. The increased processing time is mainly used for intermittent intent preservation, synchronous action decoding, and frame-level alignment verification. The average response time of 1.08 seconds can still meet the normal interaction requirements in kitchen voice control, and in exchange for stable improvements in intermittent speech recognition, correction field updates, separation of multiple control actions, and interception of high-risk actions. After execution, the system records the control command execution status, the running status after execution, the actual action intensity, and the actual execution timing, and updates the appliance state vector accordingly. The updated appliance state vector is used for the next recognition and control processing of the elderly's voice data, without modifying the already completed voice text sequence and kitchen control action sequence.

[0045] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A voice recognition-based intelligent control system for age-friendly kitchen appliances, characterized in that, include: The data acquisition and processing module is used to collect elderly people's voice data and kitchen appliance status data in the kitchen environment, perform data preprocessing, and generate voice feature sequences, voice event tag sets, and appliance status vectors. The model building module is used to build a kitchen control Mamba model based on the Speech-Mamba model, including an appliance state condition unit, an intermittent intent holding unit, an action isomorphic decoding unit, and a frame action verification unit. The state condition coding module is used to introduce the speech feature sequence and the appliance state vector into the state space coding process based on the appliance state condition unit to generate the appliance state-related speech representation. The intermittent intent holding module is used in the intermittent intent holding unit to construct the control intent holding state based on the electrical appliance status associated with the voice representation, update the control intent holding state based on the voice event tag set, and generate an age-appropriate voice intent representation. The control action decoding module is used to call the action isomorphic decoding unit to decode based on the age-appropriate speech intent representation, and generate speech text sequence and kitchen control action sequence; The frame action verification module is used to perform frame-level alignment on the speech feature sequence, speech text sequence and kitchen control action sequence based on the frame action verification unit, generate frame-level alignment relationship, and convert the kitchen control action sequence into a set of control instructions according to the frame-level alignment relationship. The hierarchical control execution module is used to perform hierarchical control response processing based on the set of control commands, record execution feedback, and update the electrical state vector.

2. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The data preprocessing includes endpoint detection, noise reduction, acoustic feature extraction, and speech event tagging of the elderly’s speech data, generating a speech feature sequence and a speech event tag set. The speech event tag set includes speech pause tags, repeated speech segment tags, and corrected speech segment tags. The data preprocessing also includes encoding the status data of kitchen appliances to generate appliance status vectors, including operating status, executable actions, and safety level.

3. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The model building module includes: The Mamba encoder, Mamba decoder, selective state-space model, and connection-time classification monotonic alignment mechanism in the Speech-Mamba model are invoked. Based on the selective state space model, the original single-condition parameter generation structure involving only speech features is transformed into a dual-condition parameter generation structure involving both speech features and appliance state, thus constructing the appliance state condition unit. Based on the Mamba encoder, the original single encoded state organization structure is transformed into an acoustic real-time state organization structure and a control intent-maintaining state organization structure. Event-driven state update paths and state fusion paths are set, and discontinuous intent-maintaining units are constructed. Based on Mamba decoder, the original single text decoding structure is transformed into a text decoding structure and action decoding structure in a shared decoding state, and an action isomorphic decoding unit is constructed. After the action isomorphic decoding unit, a new frame action verification unit is added. Based on the connection temporal classification monotonic alignment mechanism, the binary alignment structure between the original speech feature frame and the text tag is expanded into a three-level alignment structure between the speech feature frame, the text tag and the action field. A kitchen control Mamba model is constructed based on the appliance state condition unit, the intermittent intent holding unit, the action isomorphic decoding unit, and the frame action verification unit.

4. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The state condition encoding module includes: According to the order of speech frames in the speech feature sequence, the electrical appliance state vector is matched with each speech frame, the operating state is converted into an operating state embedding, the executable actions are encoded to generate action codes, and the safety level is converted into a safety level code. The speech features and running states of each speech frame are embedded, spliced, linearly mapped, and nonlinearly activated to generate step size adjustment coefficients. The step size parameters corresponding to each speech frame are adjusted, and the discrete state transition relationship corresponding to each speech frame is determined. The speech features of each speech frame are concatenated and linearly mapped with the action code to generate the input mapping parameters corresponding to each speech frame. An input mapping mask is generated according to the executable action. The input mapping parameters are multiplied element by element with the input mapping mask to obtain the adjusted input mapping parameters. The speech features of each speech frame are concatenated with the security level code, linearly mapped and normalized to generate output mapping weights. The output mapping parameters corresponding to each speech frame are multiplied element by element with the output mapping weights to obtain the adjusted output mapping parameters. Based on the order of speech frames, state-space coding is performed by combining discrete state transition relationships, adjusted input mapping parameters, and adjusted output mapping parameters to generate the current state output. Based on the current state output corresponding to each voice frame, an associated voice representation of the appliance's state is generated.

5. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The discontinuous intent preservation module includes: Based on the discontinuous intent preservation unit, the electrical appliance state-related speech representation is divided into continuous speech segments according to the position of the speech event tag set, and linear mapping is performed to generate the acoustic instantaneous state of the current continuous speech segment. When the first consecutive speech segment arrives, a control intent hold state is established, and the acoustic real-time state of the first consecutive speech segment is written into the control intent hold state. When subsequent consecutive speech segments arrive, the control intent hold state updated by the previous consecutive speech segment is used as the control intent hold state corresponding to the current consecutive speech segment. Read the set of voice event markers, determine the voice pause position, repeated voice segment position and corrected voice segment position in the voice representation associated with the electrical appliance status, and update the control intention hold state according to the voice event type corresponding to each position; The acoustic instantaneous state and the updated control intent hold state are weighted and summed to generate an age-appropriate speech intent representation.

6. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The control action decoding module includes: The age-appropriate speech intent representation input action isomorphic decoding unit is constructed to build a shared decoding state, which is simultaneously provided to the text decoding structure and the action decoding structure; The text decoding structure performs a linear mapping of the shared decoding state at the current decoding moment to the text markers, generates the text prediction score corresponding to each text marker, normalizes it, and selects the text marker with the largest value as the text marker at the current decoding moment to generate the speech text sequence. Based on the executable actions in the electrical appliance state vector, the action decoding structure performs a linear mapping of control actions to the shared decoding state according to the number of control action categories, generates a predicted score corresponding to each control action, normalizes it, and selects the control action with the largest value as the control action at the current decoding moment. The action decoding structure performs a linear mapping on the shared decoding state according to the number of electrical object categories, the number of action intensity categories, and the number of execution timing categories, respectively, and generates electrical object prediction scores, action intensity prediction scores, and execution timing prediction scores in sequence. These scores are then normalized, and the category with the largest value is selected to obtain the electrical object, action intensity, and execution timing at the current decoding moment. Determine the confirmation level at the current decoding moment based on the security level in the electrical appliance state vector; Combine the appliance object, control action, action intensity, execution timing, and confirmation level in the order of the fields to generate the kitchen control action field; After the text decoding structure generates a decoding end marker, the recursive update of the shared decoding state is stopped, the kitchen control action fields are arranged, and a kitchen control action sequence is generated.

7. A voice recognition-based intelligent control system for age-friendly kitchen appliances according to claim 1, characterized in that, The frame action verification module includes: In the frame action verification unit, the text tagging results are calculated frame by frame according to the arrangement order of the feature frames in the speech feature sequence; Based on the original feature frame positions retained in the frame-by-frame text tagging results, read the consecutive feature frames corresponding to each text tag before merging, determine the start feature frame and the end feature frame, and establish the correspondence between each text tag in the speech text sequence and the start and end feature frames; Read the text start position and text end position corresponding to each kitchen control action in the kitchen control action sequence, determine the start feature frame and end feature frame of each kitchen control action, and obtain the feature frame interval corresponding to each kitchen control action. Read the text tags corresponding to the appliance object, control action, action intensity and execution timing in each kitchen control action, and determine the feature frame interval corresponding to each field based on the start feature frame and end feature frame of each text tag. Combine the feature frame interval corresponding to the same kitchen control action and the category name of the field to determine the kitchen control actions with complete field alignment and those with abnormal field alignment. Record the text start position, text end position, start feature frame, end feature frame, and the judgment results of field alignment integrity and field alignment abnormality corresponding to each kitchen control action, and generate frame-level alignment relationship. For kitchen control actions with complete field alignment, the control actions are compared with the executable actions in the appliance state vector to generate control instructions to be executed and control instructions to be confirmed. Based on kitchen control actions with field alignment anomalies, synchronously construct control commands that are prohibited from execution; The control instructions to be executed, the control instructions to be confirmed, and the control instructions to be prohibited from execution are grouped into a set of control instructions.

8. The intelligent control system for age-friendly kitchen appliances based on voice recognition according to claim 1, characterized in that, The hierarchical control execution module includes: Read the control command to be executed, send it to the kitchen appliance corresponding to the appliance object, and complete the status adjustment; Read the control command to be confirmed, convert the appliance object and control action into a voice confirmation prompt and play it, collect the elderly's confirmation voice, and determine whether to execute the control action of the corresponding kitchen appliance; Read the control command that prohibits execution, do not send control actions to the kitchen appliance corresponding to the appliance object, and maintain the current operating state of the kitchen appliance; Record the execution feedback after the instruction is executed; Update the operating state and executable actions in the electrical appliance state vector based on the execution feedback; Read the fault status, protection status and alarm status of kitchen appliances after execution, and update the safety level.

Images