Equipment voice control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an equipment voice control method and device, electronic equipment and a storage medium. The method comprises the following steps: receiving a voice control instruction; performing audio feature extraction on the voice control instruction, and determining a target audio feature; when a preset target wake-up word is detected in the target audio feature, analyzing the target audio feature based on a natural language recognition model, and determining a control signal; and executing the control signal. According to the embodiment of the invention, the recognition and control of the voice control instruction are completed locally on the equipment, and the voice control can be realized under the condition that the equipment is not networked, so that the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things home appliances, and in particular to a device voice control method, a device voice control device, an electronic device and a storage medium. Background Art

[0002] Existing smart home systems usually rely on cloud computing and Internet connections to implement voice control functions. This model makes it impossible for users to use voice control functions in an offline environment, affecting the user experience. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a device voice control method, a device voice control apparatus, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, in a first aspect of the present invention, an embodiment of the present invention discloses a device voice control method, which is applied to a home appliance, wherein a natural language recognition model is locally deployed on the home appliance, and the natural language recognition module is trained based on a home appliance control semantic library, and the method comprises:

[0005] Receive voice control instructions;

[0006] Extracting audio features from the voice control command to determine target audio features;

[0007] When a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal;

[0008] The control signal is executed.

[0009] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features includes:

[0010] Mel frequency cepstral coefficient features are extracted from the voice control instruction as target audio features.

[0011] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features further includes:

[0012] The Mel-frequency cepstral coefficient feature is filtered.

[0013] Optionally, the method further comprises:

[0014] Determining execution status information of the control signal;

[0015] The execution status information is fed back.

[0016] Optionally, the step of feeding back the execution status information includes:

[0017] Converting the execution status information into text information;

[0018] Converting the text information into an audio signal to be played;

[0019] Output the text information and / or the audio signal to be played.

[0020] Optionally, the household device further has a local database, and the method further includes:

[0021] Reading historical voice commands from the local database;

[0022] When the historical voice instruction does not match the voice control instruction, the voice control instruction is associated with the control signal to generate association information.

[0023] Optionally, the method further comprises:

[0024] When the historical voice instruction matches the voice control instruction, reading the historical control parameter associated with the historical voice instruction;

[0025] The control signal is used to update the historical control parameter.

[0026] Optionally, the method further comprises:

[0027] The associated information is stored in the local database.

[0028] Optionally, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

[0029] In a second aspect of the present invention, an embodiment of the present invention discloses a device voice control apparatus, which is applied to a home appliance, wherein a natural language recognition model is locally deployed on the home appliance, and the natural language recognition module is trained based on a home appliance control semantic library, and the device comprises:

[0030] A receiving module, used for receiving voice control instructions;

[0031] An extraction module, used to extract audio features from the voice control instruction and determine target audio features;

[0032] A parsing module, configured to parse the target audio feature and determine a control signal based on the natural language recognition model when a preset target wake-up word is detected in the target audio feature;

[0033] A control module is used to execute the control signal.

[0034] Optionally, the extraction module includes:

[0035] The extraction submodule is used to extract the Mel frequency cepstral coefficient feature from the voice control instruction as the target audio feature.

[0036] Optionally, the extraction module further includes:

[0037] The filter submodule is used to perform filtering on the Mel-frequency cepstral coefficient feature.

[0038] Optionally, the device further comprises:

[0039] An execution determination module, used to determine execution status information of the control signal;

[0040] A feedback module is used to feed back the execution status information.

[0041] Optionally, the feedback module includes:

[0042] A first conversion submodule, used for converting the execution status information into text information;

[0043] A second conversion submodule, used for converting the text information into an audio signal to be played;

[0044] The output submodule is used to output the text information and / or the audio signal to be played.

[0045] Optionally, the household device further has a local database, and the device further comprises:

[0046] A first reading module, used for reading historical voice instructions in the local database;

[0047] The association module is used to associate the voice control instruction with the control signal to generate association information when the historical voice instruction does not match the voice control instruction.

[0048] Optionally, the device further comprises:

[0049] A second reading module, configured to read a historical control parameter associated with the historical voice instruction when the historical voice instruction matches the voice control instruction;

[0050] An updating module is used to update the historical control parameters using the control signal.

[0051] Optionally, the device further comprises:

[0052] A storage module is used to store the associated information in the local database.

[0053] Optionally, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

[0054] In the third aspect of the present invention, an embodiment of the present invention discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the motor fault detection method as described above when executed by the processor.

[0055] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the motor fault detection method as described above are implemented.

[0056] The embodiments of the present invention include the following advantages:

[0057] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; and the control signal is executed. The embodiment of the present invention is completed locally on the device by combining voice recognition with semantic parsing. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the fluency and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flowchart of a method for voice control of a device according to an embodiment of the present invention;

[0059] Figure 2 is a flowchart of another embodiment of a device voice control method of the present invention;

[0060] Figure 3 is a flowchart of steps of another embodiment of a device voice control method of the present invention;

[0061] Figure 4 is a flowchart of an example of a method for voice control of a device according to the present invention;

[0062] Figure 5 is a structural block diagram of an embodiment of a device voice control apparatus of the present invention;

[0063] Figure 6 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0064] Figure 7 It is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Reference Figure 1 , shows a flow chart of steps of an embodiment of a device voice control method of the present invention, the device voice control method is applied to a home appliance, the home appliance has a natural language recognition model deployed locally, and the natural language recognition module is trained based on a home appliance control semantic library. The device voice control method may specifically include the following steps:

[0067] Step 101, receiving a voice control instruction;

[0068] When a user issues a voice control command, the voice control command can be received by a sound wave recognition sensor such as a microphone.

[0069] Step 102, extracting audio features from the voice control instruction to determine target audio features;

[0070] Perform speech recognition on the acquired voice control instructions, extract features from the audio, and determine the target audio features.

[0071] Step 103, when a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal;

[0072] The target audio features are detected. When a preset target wake-up word is detected in the natural language in the target audio features, it can be determined that the current voice control command requires relevant control of the device. The target audio features can be analyzed based on the natural language recognition model to determine the control signal.

[0073] The preset target wake-up word can be determined according to actual conditions, and the embodiment of the present invention does not make any specific limitation to this.

[0074] Step 104: execute the control signal.

[0075] Execute the control signal to realize the control function required by the voice control command.

[0076] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine a target audio feature; when a preset target wake-up word is detected in the target audio feature, the target audio feature is parsed based on a natural language recognition model to determine a control signal; and the control signal is executed. This is accomplished by combining voice recognition with semantic parsing locally on the device. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the smoothness and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience.

[0077] Reference Figure 2 , showing a step flow chart of another embodiment of a device voice control method of the present invention, the device voice control method is applied to a home appliance, the home appliance has a natural language recognition model locally deployed, and the natural language recognition module is trained based on a home appliance control semantic library.

[0078] In the embodiment of the present invention, the home appliance is an IoT device with voice control function, including but not limited to television, air conditioner, fan, smart speaker, refrigerator, washing machine, sweeper, smart night light and smart remote control. For TV, users can switch channels, adjust volume, search programs, etc. through voice commands. It can also be linked with other smart home systems to achieve more complex scene control. For air conditioners, users can adjust temperature, switch modes, turn on and off, etc. through voice commands. For fans, users can control the switch, wind speed, timing, etc. of the fan through voice commands. It also supports the fresh air circulation function to bring users a more comfortable use experience. For smart speakers, cross-page voice control can play music, answer questions, and can also be used as the control center of smart homes to control the operation of other smart home appliances through voice commands. For refrigerators, users can use voice commands to query the food situation in the refrigerator, adjust the temperature, etc. Some refrigerators also support linkage with smart shopping systems to facilitate users to purchase and manage food. For washing machines, users can use voice commands to select washing modes, adjust water levels, etc. It also supports linkage with smart home systems to achieve more intelligent washing control. For sweepers, users can use voice commands to control the start, stop, cleaning area, etc. of the sweeper. It also supports linkage with smart home systems to achieve smarter cleaning plans. For smart night lights, users can use voice commands to control the switch, brightness, etc. of the night lights. It also supports automatic brightness adjustment function to bring users a more comfortable use experience. For smart remote controllers, users can use voice commands to control the operating status of home appliances.

[0079] The device voice control method may specifically include the following steps:

[0080] Step 201, receiving a voice control instruction;

[0081] Can receive real-time voice control instructions input by users.

[0082] Step 202, extracting audio features from the voice control instruction to determine target audio features;

[0083] The audio feature extraction can be performed on the total audio frequency information of the voice control command to determine the required target audio feature.

[0084] Specifically, the step of extracting audio features from the voice control instruction and determining target audio features includes: extracting Mel-frequency cepstral coefficient features from the voice control instruction as target audio features.

[0085] Mel frequency cepstral coefficient features can be extracted from the audio signal in the voice control command as the target audio feature. The Mel frequency cepstral coefficients () are the coefficients that make up the Mel frequency cepstral, which are derived from the cepstral spectrum of the audio segment. The Mel frequency cepstral is a linear transformation of the logarithmic energy spectrum based on the nonlinear Mel scale of the sound frequency.

[0086] The Mel scale describes the nonlinear characteristics of human ear frequency, which is very similar to the processing method of human auditory perception. The original audio signal is converted into a representation on the Mel frequency scale, and the cepstral coefficients of the spectrum are calculated to capture the spectral envelope of the audio signal. The calculation of Mel frequency cepstral coefficients usually includes the following steps: Preprocessing, decomposing the speech signal into multiple short time frames (frames), each frame is usually 20 to 40 milliseconds long, and there is a certain overlap between frames (such as 50%). Pre-emphasize the speech signal and enhance the high-frequency components through a high-pass filter. Fourier transform, perform a fast Fourier transform (FFT) on each frame, convert the signal from the time domain to the frequency domain, and obtain the spectrum. Mel filtering, filter the spectrum using a Mel scale filter bank. The Mel filter bank is a set of nonlinearly distributed triangular bandpass filters that are evenly distributed on the Mel scale. Sum the energy in each filter and then take the logarithm. Discrete cosine transform (DCT), perform discrete cosine transform on the logarithmic energy obtained in the previous step to obtain the Mel frequency cepstral coefficients. Usually, the first 12 (or more) DCT coefficients are retained, and then the energy of a current frame is concatenated to form a 13-dimensional (or more) MFCC feature vector. By extracting the Mel-frequency cepstral coefficient features as the target audio features, the processing method of human auditory perception can be better simulated. The cepstral coefficients are obtained by discrete cosine transform, which describes the information of the time-frequency spectrum with less data, has lower correlation between features, and can make better use of subsequent data processing to improve data processing efficiency.

[0087] For example, the corresponding process of extracting Mel frequency cepstral coefficient features is that after the voice control command is collected, the audio signal in the voice control command can be framed and the continuous audio signal can be divided into multiple small frames. Each frame usually lasts 20-30 milliseconds, which can make the processing faster and facilitate subsequent feature extraction. The Mel frequency cepstral coefficient feature can be extracted from each frame as the target audio feature.

[0088] Furthermore, the step of extracting audio features from the voice control instruction and determining target audio features also includes: filtering the Mel-frequency cepstral coefficient features.

[0089] In addition, the Mel frequency cepstral coefficient feature can also be filtered to eliminate the noise of the audio frequency, in order to reduce the impact of background noise on speech recognition, suppress background noise and enhance the clarity of the target speech signal. The filtering method can be adaptive filtering and spectrum subtraction. Adaptive filtering technology dynamically adjusts the parameters of the filter according to the characteristics of the input audio signal and environmental conditions, thereby achieving effective denoising and enhancement of the audio signal. The corresponding processing process is to perform window function processing on the framed audio signal, and multiply each frame signal by a window function (such as a Hamming window) to reduce spectrum leakage and boundary effects. Then enter the adaptive filtering stage, including: filter initialization, setting the initial parameters of the adaptive filter, including the filter order, initial filter coefficients, etc. The selection of the filter order should be determined according to the actual application requirements. Noise estimation, in speech denoising applications, it is necessary to estimate the background noise signal through a silent segment or other methods. Accurate noise estimation is the key to the subsequent filtering effect. Iteratively update the filter coefficients, according to the real-time input audio signal and noise estimation, iteratively update the filter coefficients through an adaptive algorithm (such as an LMS algorithm, an NLMS algorithm, etc.). These algorithms are based on the minimum mean square error criterion and minimize the error between the output signal and the expected signal by continuously adjusting the filter coefficients. Signal reconstruction: filter the input signal with the updated filter coefficients to obtain a denoised or enhanced audio signal. Through the inverse window function and overlap-add (OLA) method, the frames are spliced ​​into a continuous time domain audio signal. The basic principle of spectral subtraction is to estimate the power spectrum of the noisy signal in the frequency domain, subtract the estimated noise power spectrum from it, and then return to the time domain through inverse transformation to obtain the denoised signal. The core is to accurately estimate the power spectrum of the original signal and the noise, and make appropriate subtraction operations between them to ensure the quality of the signal. The processing process is as follows: Signal reading and framing: read the original audio signal. Divide the continuous time domain signal into several overlapping short time frames so that the signal is assumed to be stationary in each frame. The commonly used frame length is 20 to 30 milliseconds, and the frame shift is usually half of the frame length. Window function processing: multiply each frame signal by a window function (such as a Hamming window) to reduce spectral leakage and boundary effects. The selection and processing of the window function are crucial to the accuracy of subsequent spectrum analysis. Frequency domain conversion: Perform fast Fourier transform (FFT) on each frame of the signal to convert the time domain signal into a frequency domain signal to obtain a spectrum representation. Noise spectrum estimation: Estimation of the noise spectrum is a key step in spectrum subtraction. The noise spectrum can be estimated by a variety of methods, such as minimum value control based on signal processing methods, energy threshold judgment, noise threshold processing, etc., or noise estimation based on statistical models, as well as deep learning methods. In practical applications, it is usually assumed that there are silent segments without speech activity at the beginning or end of the signal, and the noise spectrum is calculated through these silent segments.Spectral subtraction, directly subtract the estimated noise spectrum from the spectrum of the noisy speech to obtain the spectrum of the enhanced speech. In order to avoid negative values ​​or over-reduction, over-subtraction coefficients and floor coefficients are usually introduced. The over-subtraction coefficient is used to control the intensity of denoising, and is usually set between 1.0 and 2.0; the floor coefficient is used to prevent speech distortion caused by over-reduction of the spectrum, and is usually between 0.002 and 0.1. Amplitude and phase recovery, retain the original phase information and only modify the amplitude spectrum. Reconstruct the frequency domain signal based on the modified amplitude spectrum and the original phase information. Time domain conversion, perform inverse fast Fourier transform (IFFT) on the modified spectrum to obtain the time domain signal. Each frame is spliced ​​into a continuous speech signal by the overlap-addition (OLA) method.

[0090] Step 203, when a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal;

[0091] When the target audio feature is obtained, it will be detected whether the preset target wake-up word is included. When the preset target wake-up word is detected in the target audio feature, it means that the voice control instruction is used to instruct the processing of a certain function, and the target audio feature can be parsed by the natural language recognition model to determine the corresponding control signal. After successfully identifying the user's voice control command and obtaining the control signal, the corresponding control signal can be immediately sent to the actuator of the home appliance. The control signal can be sent by selecting a suitable communication protocol according to the interface type of the specific device, such as general input and output, I2C (Inter-Integrated Circuit, bidirectional two-wire synchronous serial bus) or serial communication for transmission.

[0092] Step 204, executing the control signal;

[0093] Execute the control signal to achieve the corresponding control.

[0094] Step 205, determining the execution status information of the control signal;

[0095] After executing the control signal, the execution status information of the device executing the control signal can be determined. The execution status information is used to represent the completion status of the device executing the control signal.

[0096] Step 206: Feedback the execution status information.

[0097] The device can provide users with feedback on execution status information, allowing users to understand the specific implementation status.

[0098] Specifically, the step of feeding back the execution status information includes: converting the execution status information into text information; converting the text information into an audio signal to be played; and outputting the text information and / or the audio signal to be played.

[0099] First, the execution status information can be converted into text information. Then the text information is converted into an audio signal to be played. The conversion process may include the following steps: Input text processing, receiving text information input by the user, ensuring that the text format is correct, and preprocessing it, such as removing redundant characters, word segmentation, punctuation recognition, etc. Syntax analysis, performing grammatical analysis on the text to determine the structure, grammatical rules and semantic information of the sentence. This helps the system understand the meaning of the text and generate more natural speech. Semantic understanding, in-depth analysis of the semantic content of the text, understanding the vocabulary, phrases and contextual relationships in the sentence. This helps the system generate speech that conforms to the context and emotion. Select a speech model, select a suitable speech model according to user needs and scenarios. These models are usually trained based on machine learning or deep learning technology, and can simulate speech of different genders, ages, accents and speaking speeds. Generate speech waveforms, use a speech synthesis engine to convert the processed text information into a speech waveform. This process may involve two main speech synthesis technologies: splicing synthesis and parameter synthesis. Splicing synthesis forms a complete speech output by splicing pre-recorded speech segments together. Parameter synthesis, generating speech waveforms through mathematical models, this technology can generate more natural and fluent speech, but the computational complexity is high. Adjust speech parameters, adjust the volume, speaking speed, pitch and intonation of the speech according to the needs to obtain satisfactory speech effects and generate audio signals to be played.

[0100] Finally, text information and / or an audio signal to be played may be output according to the interaction mode between the device and the user, so that the user can understand the execution status through the text information and / or the audio signal to be played.

[0101] For example, after the device executes a command, voice feedback can be used to inform the user of the result of the operation. When the user issues a command to "turn on the light", text-to-speech technology can be used to generate voice feedback in real time to inform the user that "the light is turned on". This instant feedback not only enhances the user's interactive experience, but also enables users to confirm whether the device has accurately executed their instructions.

[0102] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; the control signal is executed; the execution status information of the control signal is determined; and the execution status information is fed back. The embodiment of the present invention is completed by combining voice recognition and semantic parsing locally on the device. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved, while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios without network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the fluency and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience. In addition, by feeding back the execution status information, the user can intuitively understand the execution cleanup, which not only enhances the user's interactive experience, but also enables the user to confirm whether the system has accurately executed the instruction.

[0103] Reference Figure 3 , shows a flowchart of another embodiment of a device voice control method of the present invention, the device voice control method is applied to a home appliance, and the home appliance is locally deployed with a natural language recognition model and a local database. The natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and the local database.

[0104] In an embodiment of the present invention, the basis of the natural language recognition model can be a long short-term memory network or a convolutional neural network. The long short-term memory network consists of an input layer, a hidden layer and an output layer. Among them, the hidden layer contains multiple LSTM units, each of which has a memory cell (Cell State) and three gating mechanisms (forget gate, input gate and output gate). Forget gate: determines which information in the memory cell should be forgotten. It generates an output between 0 and 1 through a sigmoid activation function, which controls the degree of attenuation of information in the memory cell. Input gate: controls the update of the memory cell by the input information of the current time step. It determines how much current information needs to be written into the memory cell. Memory cell update: According to the output of the forget gate and the control of the input gate, the memory cell will be updated. Output gate: determines which information in the memory cell will be output to the hidden state and affects the output of the current time step. The convolutional neural network consists of an input layer, a convolution layer, an activation layer, a pooling layer, a fully connected layer and an output layer. Convolution layer: uses multiple convolution kernels (or filters) to perform convolution operations on the input data and extract feature maps. Activation layer: Use nonlinear activation functions (such as ReLU) to increase the nonlinear expression ability of the network. Pooling layer: Used to reduce the spatial dimension of the feature map, reduce the amount of calculation, and retain important feature information. Fully connected layer: Flatten the feature map and connect it to the fully connected neural network layer.

[0105] The home device control semantic library stores customized parsing rules for specific voice commands for furniture devices. By presetting a specific device control semantic library (such as "turn on the light" and "adjust the temperature") and the function of user-definable extended commands, combined with locally stored semantic historical data, the parsing is more targeted and accurate. At the same time, local storage and parsing ensure the privacy of user data. Unlike generalized parsing methods that rely on cloud models, it is more suitable for home privacy-sensitive scenarios. That is, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home device control semantic library and the local database.

[0106] The device voice control method may specifically include the following steps:

[0107] Step 301, receiving a voice control instruction;

[0108] Can receive voice control commands input by users

[0109] Step 302, extracting audio features from the voice control instruction to determine target audio features;

[0110] Audio feature extraction is performed on the voice control command, and the Mel frequency cepstral coefficients are extracted as the target audio features.

[0111] Step 303: when a preset target wake-up word is detected in the target audio feature, the target audio feature is parsed based on the natural language recognition model to determine a control signal;

[0112] When the preset target wake-up word is detected in the detection mark audio feature, the voice command recognition stage is entered. The natural language recognition model can be used to analyze the target audio features and determine the control signal.

[0113] Step 304, executing the control signal;

[0114] Execute the control signal to realize the corresponding function.

[0115] Step 305, reading the historical voice commands in the local database;

[0116] The voice commands corresponding to past control operations, ie, historical voice commands, may be read from a local database.

[0117] Step 306, when the historical voice instruction does not match the voice control instruction, associating the voice control instruction with the control signal to generate association information;

[0118] It can be judged whether the historical voice instructions match the current voice control instructions. If the historical voice instructions are consistent with the current voice control instructions, they are considered to match; if the historical voice instructions are inconsistent with the current voice control instructions, they are considered to be mismatched. The judgment can be made in the following ways: 1. Based on the comparison of instruction encoding, their binary encoding or assembly text can be directly compared. This method is simple and direct, but requires that the format and encoding of the instructions must be exactly the same. Binary encoding comparison: The binary encodings of the two instructions are compared bit by bit. If they are exactly the same, the instructions are the same. Assembly text comparison: The assembly texts of the two instructions are compared. If the texts are exactly the same (including instruction mnemonics, operands, and addresses, etc.), the instructions are the same. 2. Based on the comparison of instruction semantics, the semantics of instructions refers to the operations or functions performed by the instructions. For some instructions, even if their encodings or texts are not exactly the same, they can be considered to be the same instructions if their semantics are the same. Disassembly and instruction analysis: Use a disassembly tool to disassemble the binary file to obtain the assembly instruction text. Then, the semantics of the two instructions are analyzed, including the functions, operands, and addresses of the instructions. If the semantics of the two instructions are exactly the same, they are the same instructions. Instruction set manual reference: Refer to the instruction set manual for the target processor to understand the semantics and function of each instruction. By comparing the descriptions of two instructions in the manual, you can determine whether they are the same.

[0119] When the historical voice command does not match the voice control command, the voice control command is associated with the control signal to generate associated information;

[0120] Step 307, storing the associated information in the local database;

[0121] The associated information is stored in the local database for subsequent judgment.

[0122] Step 308, when the historical voice instruction matches the voice control instruction, reading the historical control parameter associated with the historical voice instruction;

[0123] When the historical voice command matches the voice control command, the historical control parameters associated with the historical voice command may be read.

[0124] Step 309: Use the control signal to update the historical control parameters.

[0125] The control signal is used to update the historical control parameters so that the control can be more in line with user habits.

[0126] For example, the local database SQLite (database) can be used to store all voice commands and device status information. Whenever a new voice command is recognized, the system will automatically record the command, the corresponding timestamp, and the current status of the device (such as whether the light is on, the air conditioner setting temperature, etc.) to the database. This design not only facilitates subsequent command retrieval and analysis, but also provides users with operation history records to improve the user experience. The system needs to maintain the current status information of each smart device in real time to ensure that the control commands issued by the user can be accurately executed. For example, when the system receives the command to "turn on the light", it must first query the current status of the light in the database to avoid unnecessary operations or conflicts. After each command is executed, the system will also update the status of the corresponding device in the database to reflect the latest operation results.

[0127] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; the control signal is executed; the historical voice instructions in the local database are read; when the historical voice instructions do not match the voice control instructions, the voice control instructions are associated with the control signal to generate associated information; when the historical voice instructions match the voice control instructions, the historical control parameters associated with the historical voice instructions are read; the historical control parameters are updated using the control signal; and the associated information is stored in the local database. The speech recognition and semantic parsing are combined and completed locally on the device. The lightweight neural network model and customized parsing rules for voice commands are used to significantly improve the parsing speed and greatly reduce the computing power requirements. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities. Users can get immediate feedback after issuing commands, which significantly improves the smoothness and naturalness of operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience; and by setting all data processing and storage locally, it ensures that the user's voice data is not uploaded to the cloud, effectively protecting user privacy and improving the acceptance of voice control functions.

[0128] In order to make the implementation process of the embodiment of the present invention clear to those skilled in the art, an example is used below, which can be referred to as Figure 4 , showing a step flow chart of an example of a device voice control method of the present invention.

[0129] 1. Voice input: When the user initiates a voice control command, it belongs to the corresponding voice.

[0130] 2. Audio processing: perform signal segmentation on the input voice control command and extract the corresponding target voice features.

[0131] 3. Keyword detection: detect whether the target speech features contain keywords.

[0132] 4. Command recognition: After detecting the keyword, the command is identified and the control signal is determined.

[0133] 5. Equipment control: control the equipment based on the control signal.

[0134] 6. User interaction feedback: after executing the control signal, feedback the execution status to the user.

[0135] 7. Data storage (device status): store the processing data in the local database for subsequent use.

[0136] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0137] Reference Figure 5 , shows a structural block diagram of an embodiment of a device voice control device of the present invention, the device voice control device is applied to a home appliance, the home appliance has a natural language recognition model deployed locally, and the natural language recognition module is trained based on the home appliance control semantic library. The device voice control device may specifically include the following modules:

[0138] Receiving module 501, used for receiving voice control instructions;

[0139] An extraction module 502 is used to extract audio features from the voice control instruction and determine target audio features;

[0140] The parsing module 503 is used to parse the target audio feature based on the natural language recognition model and determine the control signal when a preset target wake-up word is detected in the target audio feature;

[0141] The control module 504 is configured to execute the control signal.

[0142] In an optional embodiment of the present invention, the extraction module 502 includes:

[0143] The extraction submodule is used to extract the Mel frequency cepstral coefficient feature from the voice control instruction as the target audio feature.

[0144] In an optional embodiment of the present invention, the extraction module 502 further includes:

[0145] The filter submodule is used to perform filtering processing on the Mel-frequency cepstral coefficient feature.

[0146] In an optional embodiment of the present invention, the device further comprises:

[0147] An execution determination module, used to determine execution status information of the control signal;

[0148] A feedback module is used to feed back the execution status information.

[0149] In an optional embodiment of the present invention, the feedback module includes:

[0150] A first conversion submodule, used for converting the execution status information into text information;

[0151] A second conversion submodule, used for converting the text information into an audio signal to be played;

[0152] The output submodule is used to output the text information and / or the audio signal to be played.

[0153] In an optional embodiment of the present invention, the household appliance further has a local database, and the device further comprises:

[0154] A first reading module, used for reading historical voice instructions in the local database;

[0155] The association module is used to associate the voice control instruction with the control signal to generate association information when the historical voice instruction does not match the voice control instruction.

[0156] In an optional embodiment of the present invention, the device further comprises:

[0157] A second reading module, configured to read a historical control parameter associated with the historical voice instruction when the historical voice instruction matches the voice control instruction;

[0158] An updating module is used to update the historical control parameters using the control signal.

[0159] In an optional embodiment of the present invention, the device further comprises:

[0160] A storage module is used to store the associated information in the local database.

[0161] In an optional embodiment of the present invention, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

[0162] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; and the control signal is executed. The embodiment of the present invention is completed locally on the device by combining voice recognition with semantic parsing. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the fluency and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience.

[0163] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] Reference Figure 6 , an embodiment of the present invention further provides an electronic device, including:

[0165] A processor 601 and a storage medium 602, wherein the storage medium 602 stores a computer program executable by the processor 601. When the electronic device is controlled to run, the processor 601 executes the computer program to implement the device voice control method as described in any one of the embodiments of the present invention.

[0166] The device voice control method is applied to a home device, wherein a natural language recognition model is locally deployed on the home device, and the natural language recognition module is trained based on a home device control semantic library. The method includes:

[0167] Receive voice control instructions;

[0168] Extracting audio features from the voice control command to determine target audio features;

[0169] When a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal;

[0170] The control signal is executed.

[0171] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features includes:

[0172] Mel frequency cepstral coefficient features are extracted from the voice control instruction as target audio features.

[0173] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features further includes:

[0174] The Mel-frequency cepstral coefficient feature is filtered.

[0175] Optionally, the method further comprises:

[0176] Determining execution status information of the control signal;

[0177] The execution status information is fed back.

[0178] Optionally, the step of feeding back the execution status information includes:

[0179] Converting the execution status information into text information;

[0180] Converting the text information into an audio signal to be played;

[0181] Output the text information and / or the audio signal to be played.

[0182] Optionally, the household device further has a local database, and the method further includes:

[0183] Reading historical voice commands from the local database;

[0184] When the historical voice instruction does not match the voice control instruction, the voice control instruction is associated with the control signal to generate association information.

[0185] Optionally, the method further comprises:

[0186] When the historical voice instruction matches the voice control instruction, reading the historical control parameter associated with the historical voice instruction;

[0187] The control signal is used to update the historical control parameter.

[0188] Optionally, the method further comprises:

[0189] The associated information is stored in the local database.

[0190] Optionally, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

[0191] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; and the control signal is executed. The embodiment of the present invention is completed locally on the device by combining voice recognition with semantic parsing. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the fluency and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience.

[0192] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0193] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0194] Reference Figure 7 The embodiment of the present invention further provides a computer-readable storage medium 701, on which a computer program is stored. When the computer program is executed by a processor, the device voice control method as described in any one of the embodiments of the present invention is executed.

[0195] The device voice control method is applied to a home device, wherein a natural language recognition model is locally deployed on the home device, and the natural language recognition module is trained based on a home device control semantic library. The method includes:

[0196] Receive voice control instructions;

[0197] Extracting audio features from the voice control command to determine target audio features;

[0198] When a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal;

[0199] The control signal is executed.

[0200] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features includes:

[0201] Mel frequency cepstral coefficient features are extracted from the voice control instruction as target audio features.

[0202] Optionally, the step of extracting audio features from the voice control instruction and determining target audio features further includes:

[0203] The Mel-frequency cepstral coefficient feature is filtered.

[0204] Optionally, the method further comprises:

[0205] Determining execution status information of the control signal;

[0206] The execution status information is fed back.

[0207] Optionally, the step of feeding back the execution status information includes:

[0208] Converting the execution status information into text information;

[0209] Converting the text information into an audio signal to be played;

[0210] Output the text information and / or the audio signal to be played.

[0211] Optionally, the household device further has a local database, and the method further includes:

[0212] Reading historical voice commands from the local database;

[0213] When the historical voice instruction does not match the voice control instruction, the voice control instruction is associated with the control signal to generate association information.

[0214] Optionally, the method further comprises:

[0215] When the historical voice instruction matches the voice control instruction, reading the historical control parameter associated with the historical voice instruction;

[0216] The control signal is used to update the historical control parameter.

[0217] Optionally, the method further comprises:

[0218] The associated information is stored in the local database.

[0219] Optionally, the natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

[0220] The embodiment of the present invention receives a voice control instruction; extracts audio features from the voice control instruction to determine the target audio features; when a preset target wake-up word is detected in the target audio features, the target audio features are parsed based on a natural language recognition model to determine a control signal; and the control signal is executed. The embodiment of the present invention is completed locally on the device by combining voice recognition with semantic parsing. By using a lightweight neural network model and customized parsing rules for voice commands, the parsing speed is significantly improved while the computing power requirements are greatly reduced. This real-time semantic parsing not only avoids network delays, but also quickly and accurately maps user instructions to specific device operations in an offline environment, adapting to smart home scenarios with no network or low latency requirements. Local voice recognition and control provide instant response capabilities, and users can get feedback immediately after issuing instructions, significantly improving the fluency and naturalness of the operation. Voice control can also be achieved when the device is not connected to the Internet, improving the user experience.

[0221] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0222] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0223] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0224] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0225] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0226] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0227] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0228] The above is a detailed introduction to a device voice control method, a device voice control apparatus, an electronic device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A device voice control method, characterized in that: Applied to a home appliance, the home appliance has a natural language recognition model deployed locally, and the natural language recognition module is trained based on a home appliance control semantic library. The method includes: Receive voice control instructions; Extracting audio features from the voice control command to determine target audio features; When a preset target wake-up word is detected in the target audio feature, the target audio feature is analyzed based on the natural language recognition model to determine a control signal; The control signal is executed.

2. The method according to claim 1, characterized in that: The step of extracting audio features from the voice control instruction and determining target audio features comprises: Mel frequency cepstral coefficient features are extracted from the voice control instruction as target audio features.

3. The method according to claim 2, characterized in that The step of extracting audio features from the voice control instruction and determining target audio features also includes: The Mel-frequency cepstral coefficient feature is filtered.

4. The method according to claim 1, characterized in that The method further comprises: Determining execution status information of the control signal; The execution status information is fed back.

5. The method according to claim 4, characterized in that The step of feeding back the execution status information comprises: Converting the execution status information into text information; Converting the text information into an audio signal to be played; Output the text information and / or the audio signal to be played.

6. The method according to claim 1, characterized in that The household device also has a local database, and the method further includes: Reading historical voice commands from the local database; When the historical voice instruction does not match the voice control instruction, the voice control instruction is associated with the control signal to generate association information.

7. The method according to claim 6, characterized in that The method further comprises: When the historical voice instruction matches the voice control instruction, reading the historical control parameter associated with the historical voice instruction; The control signal is used to update the historical control parameter.

8. The method according to claim 6, characterized in that The method further comprises: The associated information is stored in the local database.

9. The method according to claim 8, characterized in that The natural language recognition model is based on a long short-term memory network or a convolutional neural network, and is trained based on the home appliance control semantic library and a local database.

10. A device voice control apparatus, characterized in that: Applied to a home appliance, the home appliance is locally deployed with a natural language recognition model, the natural language recognition module is trained based on a home appliance control semantic library, and the device includes: A receiving module, used for receiving voice control instructions; An extraction module, used to extract audio features from the voice control instruction and determine target audio features; A parsing module, configured to parse the target audio feature and determine a control signal based on the natural language recognition model when a preset target wake-up word is detected in the target audio feature; A control module is used to execute the control signal.

11. An electronic device, characterized in that: The device comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the device voice control method as described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the device voice control method according to any one of claims 1 to 9 are implemented.

Images