Air conditioner voice wake-up method, device, system, and storage medium

Abstract

The application discloses an air conditioner voice wake-up method, device, system and storage medium. The method comprises the following steps: inputting target voice features into a voice wake-up model comprising an acoustic coding module, a first CTC module and a text coding module, wherein the acoustic coding module comprises a convolution module, an acoustic embedding module and a second CTC module; the convolution module extracts acoustic features from the voice features; the second CTC module calculates a first matching degree of the acoustic features and each fixed vocabulary, and executes an air conditioner instruction with the highest first matching degree that exceeds a preset threshold; when the highest first matching degree is less than or equal to the preset threshold, the acoustic embedding module outputs acoustic embedding features based on the acoustic features, the first CTC module aligns the acoustic embedding features and text embedding features of the text coding module to obtain a second matching degree, and an air conditioner instruction with the highest second matching degree that exceeds the preset threshold is executed. The application only needs to maintain one model architecture, different tasks are realized through different module combinations, and the voice wake-up efficiency is improved.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of air conditioner control technology, and in particular to an air conditioner voice wake-up method, device, system, and storage medium. Background Technology

[0002] In the field of air conditioning, voice wake-up methods aim to quickly and accurately detect specific wake words from a continuous speech stream, thereby activating the air conditioner to execute the target command corresponding to the wake word. Existing air conditioner wake-up methods, in order to improve human-computer interaction, support custom wake words; that is, in addition to recognizing pre-set fixed wake words, they can also recognize user-defined open wake words. However, existing air conditioner wake-up methods implement the recognition and activation of both fixed and open wake words by deploying two independent models in the air conditioner: one for handling the recognition of fixed wake words and the other for handling open wake words. This method requires maintaining two complete models, resulting in significant resource consumption in practical applications, leading to poor efficiency in voice wake-up operations and impacting user experience. Summary of the Invention

[0003] This application provides an air conditioner voice wake-up method, device, system, and storage medium, which can improve the efficiency of voice wake-up operations.

[0004] In a first aspect, embodiments of this application provide an air conditioner voice wake-up method, applied to an air conditioning system, the method comprising:

[0005] When a speech signal is received, feature extraction is performed on the speech signal to obtain the target speech features;

[0006] The target speech features are input into a preset speech wake-up model. The speech wake-up model includes an acoustic coding module, a first CTC module with a preset open vocabulary mapping table, and a text coding module. The acoustic coding module includes a convolution module, an acoustic embedding module, and a second CTC module with a preset fixed vocabulary mapping table. The fixed vocabulary mapping table represents the mapping relationship between preset fixed words and air conditioning commands, and the open vocabulary mapping table represents the mapping relationship between open words input by the user and air conditioning commands.

[0007] The convolution module extracts acoustic features from the speech features;

[0008] The second CTC module calculates the first matching degree between the acoustic features and each fixed word in the fixed word mapping table. When the highest first matching degree exceeds a preset threshold, the air conditioning command corresponding to the highest first matching degree is determined as the target command, and the target command is executed.

[0009] When the highest first matching degree is less than or equal to the preset threshold, the acoustic feature is input to the acoustic embedding module to obtain the acoustic embedding feature. The first CTC module aligns the acoustic embedding feature with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees. When the highest second matching degree exceeds the preset threshold, the air conditioning instruction corresponding to the highest second matching degree is determined as the target instruction, and the target instruction is executed. The text embedding feature is associated with the open vocabulary.

[0010] In some embodiments, feature extraction is performed on the speech signal to obtain target speech features, including:

[0011] The speech signal is sequentially passed through a Log-MMSE speech estimator and an MCRA noise estimator for noise reduction to obtain an intermediate speech signal;

[0012] The FBank feature extraction algorithm is used to extract features from the intermediate speech signal to obtain reference speech features;

[0013] Differential feature calculation is performed on the reference speech features to obtain first-order and second-order differential features with a preset number of dimensions.

[0014] The reference speech features, the first-order difference features, and the second-order difference features are determined as the target speech features.

[0015] In some embodiments, the convolution module includes a deep convolutional layer, a first combined layer, a pointwise convolutional layer, and a second combined layer connected in sequence. Both the first combined layer and the second combined layer include a normalization layer and an activation function layer. The convolution module extracts acoustic features from the speech features, including:

[0016] The speech features are input into the deep convolutional layer, so that the deep convolutional layer performs a convolution operation on each channel of the speech features to obtain multiple first intermediate features, wherein different first intermediate features correspond to different channels.

[0017] All the first intermediate features are input into the first combination layer and normalized and activated sequentially to obtain multiple second intermediate features.

[0018] All the second intermediate features are input into the pointwise convolutional layer so that the pointwise convolutional layer linearly combines each of the second intermediate features to obtain the third intermediate feature;

[0019] The third intermediate feature is input into the second combined layer and subjected to normalization and activation processes in sequence to obtain the acoustic feature.

[0020] In some embodiments, the text encoding module includes an input embedding layer, a BLSTM layer, and a text embedding module connected in sequence, and the step of the text encoding module outputting the text embedding features includes:

[0021] When a user inputs a wake-up word setting request, which includes the open vocabulary and the air conditioning command, a mapping relationship is established between each received open vocabulary and the air conditioning command, and the data is saved to the open vocabulary mapping table.

[0022] The open vocabulary is input into the input embedding layer for word segmentation to obtain multiple character features;

[0023] The BLSTM layer converts all the character features into high-dimensional embedding vectors.

[0024] The high-dimensional embedding vector is converted into the text embedding feature by the text embedding module, and the text embedding feature is associated with the corresponding open vocabulary in the open vocabulary mapping table.

[0025] In some embodiments, multiple second matching degrees are obtained by aligning the acoustic embedding features with multiple text embedding features output by the text encoding module through the first CTC module, including:

[0026] Each of the text embedding features is processed by inserting whitespace and repeating characters to obtain each extended sequence;

[0027] Based on any of the extended sequences, the cosine similarity between the acoustic embedding features of each frame and each character of the extended sequence is calculated to obtain a score matrix;

[0028] Based on any of the score matrices, a dynamic programming algorithm is used to calculate the cumulative maximum score when the acoustic embedding feature of each frame is aligned to each character of the corresponding extended sequence. The reference score value corresponding to the last frame aligned to the acoustic embedding feature is determined, and the reference score value is determined as the second matching degree. The path source of each reference score value is recorded to obtain a backtracking pointer matrix.

[0029] In some embodiments, when the highest second matching degree exceeds the preset threshold, determining the air conditioning command corresponding to the highest second matching degree as the target command includes:

[0030] Based on the backtracking pointer matrix, path backtracking is performed on each of the reference score values ​​to obtain the corresponding alignment paths;

[0031] The alignment paths are processed by removing whitespace and merging duplicate characters to obtain the keywords.

[0032] When the highest second matching degree exceeds the preset threshold, the keyword corresponding to the highest second matching degree is determined as the target keyword;

[0033] The air conditioning instruction corresponding to the open vocabulary in the open vocabulary mapping table that is the same as the target keyword is determined as the target instruction.

[0034] In some embodiments, the voice wake-up model is trained using the following method:

[0035] Perform data preprocessing and data augmentation on the training set;

[0036] The feature extraction algorithm is used to extract features from each speech signal in the training set to obtain each target speech feature;

[0037] For the fixed-vocabulary wake-up task, the convolutional module of the acoustic coding module and the second CTC module in the initial model are trained based on the CTC loss function and all the target speech features;

[0038] For the open vocabulary wake-up task, the first CTC module, the text encoding module, the convolutional module of the acoustic encoding module, and the acoustic embedding module in the initial model are trained by combining the CTC loss function, the multi-view loss function, and all the target speech features.

[0039] The training process continues until the model parts corresponding to the fixed vocabulary wake-up task and the open vocabulary wake-up task have both reached convergence, thus obtaining the trained speech wake-up model.

[0040] Secondly, embodiments of this application provide a control device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the air conditioner voice wake-up method as described in the first aspect.

[0041] Thirdly, embodiments of this application also provide an electronic device, including the control device of the second aspect.

[0042] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for performing the air conditioner voice wake-up method as described in the first aspect.

[0043] This application provides an air conditioner voice wake-up method, device, system, and storage medium. The method includes: upon receiving a voice signal, extracting target voice features from the voice signal; inputting the target voice features into a preset voice wake-up model, the voice wake-up model including an acoustic coding module, a first CTC module with a preset open vocabulary mapping table, and a text coding module. The acoustic coding module includes a convolution module, an acoustic embedding module, and a second CTC module with a preset fixed vocabulary mapping table. The fixed vocabulary mapping table represents the mapping relationship between preset fixed words and air conditioner commands, and the open vocabulary mapping table represents the mapping relationship between user-input open words and air conditioner commands; the convolution module extracts acoustic features from the voice features; the second CTC module... The module calculates the first matching degree between the acoustic features and each fixed word in the fixed vocabulary mapping table. When the highest first matching degree exceeds a preset threshold, the air conditioning command corresponding to the highest first matching degree is determined as the target command, and the target command is executed. When the highest first matching degree is less than or equal to the preset threshold, the acoustic features are input to the acoustic embedding module to obtain acoustic embedding features. The first CTC module aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees. When the highest second matching degree exceeds the preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command, and the target command is executed. The text embedding features are associated with the open vocabulary. According to the solution provided in this application embodiment, only one model architecture is deployed in the air conditioning system. Different tasks are implemented through different module combinations of the model. Only one model needs to be maintained, reducing the resource consumption of the air conditioning system, improving the efficiency of voice wake-up operation, and enhancing the user experience. Attached Figure Description

[0044] Figure 1 This is a flowchart of the steps of an air conditioner voice wake-up method provided in one embodiment of this application;

[0045] Figure 2 This is a structural diagram of a control device provided in another embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0048] In the field of air conditioning, voice wake-up methods aim to quickly and accurately detect specific wake words from a continuous speech stream, thereby activating the air conditioner to execute the target command corresponding to the wake word. Existing air conditioner wake-up methods, in order to improve human-computer interaction, support custom wake words; that is, in addition to recognizing pre-set fixed wake words, they can also recognize user-defined open wake words. However, existing air conditioner wake-up methods implement the recognition and activation of both fixed and open wake words by deploying two independent models in the air conditioner: one for handling the recognition of fixed wake words and the other for handling open wake words. This method requires maintaining two complete models, resulting in significant resource consumption in practical applications, leading to poor efficiency in voice wake-up operations and impacting user experience.

[0049] To address the aforementioned problems, this application provides an air conditioner voice wake-up method, device, system, and storage medium. The method includes: upon receiving a voice signal, extracting target voice features from the voice signal; inputting the target voice features into a preset voice wake-up model, the voice wake-up model including an acoustic coding module, a first CTC module with a preset open vocabulary mapping table, and a text coding module; the acoustic coding module including a convolution module, an acoustic embedding module, and a second CTC module with a preset fixed vocabulary mapping table, the fixed vocabulary mapping table representing the mapping relationship between preset fixed words and air conditioner commands, and the open vocabulary mapping table representing the mapping relationship between user-input open words and air conditioner commands; the convolution module extracting acoustic features from the voice features; the first CTC module... The second CTC module calculates the first matching degree between the acoustic features and each fixed word in the fixed vocabulary mapping table. When the highest first matching degree exceeds a preset threshold, the air conditioning command corresponding to the highest first matching degree is determined as the target command, and the target command is executed. When the highest first matching degree is less than or equal to the preset threshold, the acoustic features are input to the acoustic embedding module to obtain acoustic embedding features. The first CTC module aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees. When the highest second matching degree exceeds the preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command, and the target command is executed. The text embedding features are associated with the open vocabulary. According to the solution provided in this application embodiment, only one model architecture is deployed in the air conditioning system. Different tasks are implemented through different module combinations of the model. Only one model needs to be maintained, reducing the resource consumption of the air conditioning system, improving the efficiency of voice wake-up operation, and enhancing the user experience.

[0050] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0051] refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of an air conditioner voice wake-up method according to an embodiment of this application. This application provides an air conditioner voice wake-up method, which includes, but is not limited to, the following steps:

[0052] Step S10: When a speech signal is received, feature extraction is performed on the speech signal to obtain the target speech features.

[0053] It is understandable that, before recognizing the content of the voice signal and triggering the execution of the air conditioning command through the voice wake-up model, this embodiment first extracts features from the voice signal using a feature extraction algorithm to obtain target voice features. The main purpose is to perform front-end processing on the voice signal to eliminate most of the noise in the environment in which the voice signal is located, thereby improving the accuracy of the subsequent model processing results.

[0054] Specifically, in some embodiments, the feature extraction of the speech signal in step S10 to obtain target speech features includes, but is not limited to, the following steps:

[0055] Step S11: The speech signal is sequentially passed through the Log-MMSE speech estimator and the MCRA noise estimator for noise reduction processing to obtain the intermediate speech signal;

[0056] Step S12: Use the FBank feature extraction algorithm to extract features from the intermediate speech signal to obtain reference speech features;

[0057] Step S13: Perform differential feature calculation on the reference speech features to obtain first-order differential features and second-order differential features with a preset number of dimensions in front of the reference speech features;

[0058] Step S14: The reference speech features, first-order difference features, and second-order difference features are determined as the target speech features.

[0059] Understandably, in this embodiment, the speech signal is sequentially processed by a Log-MMSE speech estimator and an MCRA noise estimator to obtain an intermediate speech signal, which can remove most of the stationary noise in the environment while ensuring minimal speech distortion. Next, the FBank feature extraction algorithm is used to extract features from the intermediate speech signal to obtain reference speech features. Differential feature calculation is performed on the reference speech features to obtain first-order and second-order differential features with a preset number of dimensions. Since differential features can well reflect the dynamic characteristics of speech, the target speech features obtained by combining the reference speech features, first-order differential features, and second-order differential features in this embodiment can improve the distinguishability of wake-up words and non-wake-up words at the feature level, thereby improving the accuracy of the subsequent speech wake-up model processing results.

[0060] Step S20: Input the target speech features into a preset speech wake-up model. The speech wake-up model includes an acoustic coding module, a first CTC module with a preset open vocabulary mapping table, and a text coding module. The acoustic coding module includes a convolution module, an acoustic embedding module, and a second CTC module with a preset fixed vocabulary mapping table. The fixed vocabulary mapping table represents the mapping relationship between preset fixed words and air conditioning commands, and the open vocabulary mapping table represents the mapping relationship between open words input by the user and air conditioning commands.

[0061] Specifically, the training method of the voice wake-up model in this embodiment includes, but is not limited to, the following steps:

[0062] Step S21: Perform data preprocessing and data augmentation on the training set;

[0063] Step S22: Extract features from each speech signal in the training set using a feature extraction algorithm to obtain the features of each target speech signal.

[0064] Step S23: For the fixed vocabulary wake-up task, train the convolutional module of the acoustic coding module and the second CTC module in the initial model based on the CTC loss function and all target speech features.

[0065] Step S24: For the open vocabulary wake-up task, train the convolutional modules of the first CTC module, text coding module, and acoustic coding module, as well as the acoustic embedding module in the initial model by combining the CTC loss function, the multi-view loss function, and all target speech features.

[0066] Step S25 continues until the model parts corresponding to the fixed vocabulary wake-up task and the open vocabulary wake-up task have both reached the convergence condition, thus obtaining the trained speech wake-up model.

[0067] It should be noted that the preprocessing method in this embodiment includes filtering commonly used characters using general speech recognition data and data cleaning; it also includes recording specific fixed words (i.e., wake words), or using speech synthesis tools to synthesize customized wake words with various timbres; and annotating the speech signal, etc.

[0068] It should be noted that, in order to enhance the noise immunity of the voice wake-up model, this embodiment adds noise to the training set by using accumulated noise and impulse responses from various indoor, office, and outdoor scenarios to enhance the speech signals in the training set. Specifically, this may include adding noise, adding reverberation, changing speed and pitch, and frequency domain equalization to the speech signals in the training set to expand the training dataset.

[0069] Specifically, in this embodiment, the fixed-vocabulary wake-up task refers to comparing the speech features corresponding to the received speech signal with pre-defined fixed words to determine a first matching degree. When the first matching degree meets the requirements (i.e., exceeds a preset threshold), the match is successful. The open-vocabulary wake-up task refers to comparing the speech features with user-defined open words to determine a second matching degree. When the second matching degree meets the requirements, the match is successful, thereby triggering the execution of the air conditioning command corresponding to the open or fixed word. Furthermore, the fixed-vocabulary wake-up task and the open-vocabulary wake-up task share the convolutional module in the acoustic coding module of the model. That is, the initial input operations of these two different tasks are the same, but the modules used for later processing are different. This embodiment can be described as a modular reuse of the same model architecture, adapting the fixed-vocabulary and open-vocabulary tasks respectively through different training strategies and loss functions. For the fixed-vocabulary wake-up task, the convolutional module of the acoustic coding module and the second CTC module in the initial model are trained based on the CTC loss function and all target speech features. For the open-vocabulary wake-up task, the convolutional modules of the first CTC module, the text coding module, and the acoustic coding module, as well as the acoustic embedding module in the initial model, are trained using the CTC loss function, the multi-view loss function, and all target speech features. This process continues until the model parts corresponding to both the fixed-vocabulary and open-vocabulary wake-up tasks reach convergence, resulting in a trained speech wake-up model. Furthermore, the joint training strategy for the open-vocabulary wake-up task (i.e., training using both the CTC loss function and the multi-view loss function) simultaneously optimizes the general representation capability of the acoustic coding module, which also has a positive impact on the fixed-vocabulary wake-up task.

[0070] It is understood that the air conditioning system using the solution in this embodiment only needs to maintain one model, and can simultaneously implement both fixed-vocabulary wake-up tasks and open-vocabulary wake-up tasks, thereby reducing the resource consumption of the air conditioning system, improving the efficiency of voice wake-up operations, and enhancing the user experience.

[0071] Step S30: The convolution module extracts acoustic features from the speech features.

[0072] Specifically, in some embodiments, the convolution module includes a depthwise convolutional layer, a first combined layer, a pointwise convolutional layer, and a second combined layer connected in sequence. Both the first combined layer and the second combined layer include a normalization layer and an activation function layer. Step S30 includes, but is not limited to, the following steps:

[0073] Step S31: Input the speech features into the deep convolutional layer so that the deep convolutional layer performs convolution operation on each channel of the speech features to obtain multiple first intermediate features, wherein different first intermediate features correspond to different channels;

[0074] Step S32: Input all the first intermediate features into the first combined layer and perform normalization and activation processing in sequence to obtain multiple second intermediate features;

[0075] Step S33: Input all the second intermediate features into the pointwise convolutional layer so that the pointwise convolutional layer linearly combines each of the second intermediate features to obtain the third intermediate feature;

[0076] Step S34: Input the third intermediate feature into the second combined layer and perform normalization and activation processing in sequence to obtain acoustic features.

[0077] It is understood that the acoustic coding module in this embodiment is used to capture the latent acoustic representation of speech features and complete the modeling of speech feature context information. This embodiment extracts acoustic features through a convolutional module comprising a deep convolutional layer, a first combination layer, a pointwise convolutional layer, and a second combination layer connected in sequence. Specifically, speech features are input into the deep convolutional layer, which performs convolution operations on each channel of the speech features to obtain multiple first intermediate features. Different first intermediate features correspond to different channels, i.e., the deep convolutional layer has multiple convolutional layers, each with a convolutional kernel. Each convolutional kernel processes only one channel of the speech features, which greatly reduces the number of parameters of the convolutional kernel. All the first intermediate features are input into the first combination layer for normalization and activation processing to obtain multiple second intermediate features. All the second intermediate features are input into the pointwise convolutional layer, which performs linear combination of each second intermediate feature to obtain a third intermediate feature. All the second intermediate features are input into the pointwise convolutional layer, which performs linear combination of each second intermediate feature to obtain a third intermediate feature. Compared to the traditional standard convolution method, the acoustic feature extraction method in this embodiment can effectively reduce the number of parameters and computational load during processing while ensuring good feature extraction capabilities.

[0078] Step S40: Calculate the first matching degree between acoustic features and each fixed word in the fixed word mapping table through the second CTC module. When the highest first matching degree exceeds a preset threshold, determine the air conditioning command corresponding to the highest first matching degree as the target command and execute the target command.

[0079] Specifically, the specific value of the preset threshold in this embodiment can be determined by those skilled in the art based on the actual situation, and may be 0.8.

[0080] It is understandable that the air conditioner voice wake-up scheme in this embodiment prioritizes the fixed-vocabulary wake-up task. Since the fixed vocabulary corresponding to the fixed-vocabulary wake-up task is a small number of keywords predefined by the system, the model only needs to detect these specific combinations and does not need to understand any speech content, thus significantly reducing the algorithm complexity. In contrast, open-vocabulary wake-up requires additional calculation of text embedding and similarity matching, resulting in higher latency. Therefore, using the scheme in this embodiment can effectively reduce computational costs and average response time.

[0081] Understandably, for the fixed-vocabulary wake-up task, after extracting the acoustic features corresponding to the speech features through the convolutional module of the acoustic coding module, the second CTC module maps the acoustic latent representation to the probability of the target wake-up word (i.e., fixed vocabulary) on the phonemes to calculate the first matching degree. Since the system has multiple preset fixed vocabulary, there are multiple first matching degrees. Different first matching degrees correspond to the similarity between the acoustic features and different fixed vocabulary. The larger the value of the first matching degree, the greater the possibility of meeting the user's actual control needs. In this embodiment, the first matching degree with the largest value is compared with a preset threshold. When the first matching degree with the largest value exceeds the preset threshold, the air conditioning command corresponding to the highest first matching degree is determined as the target command and the target command is executed.

[0082] Step S50: When the highest first matching degree is less than or equal to a preset threshold, the acoustic features are input into the acoustic embedding module to obtain acoustic embedding features. The first CTC module aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees. When the highest second matching degree exceeds the preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command, and the target command is executed. Here, the text embedding features are related to open vocabulary.

[0083] It is understood that, referring to the description of the above embodiments, when the highest first matching degree is less than or equal to a preset threshold, it means that the current speech features cannot match any of the fixed words. At this time, this embodiment performs an open vocabulary wake-up task, inputs acoustic features into the acoustic embedding module to obtain acoustic embedding features, aligns the acoustic embedding features with multiple text embedding features output by the text encoding module through the first CTC module to obtain multiple second matching degrees, and when the highest second matching degree exceeds the preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command and the target command is executed.

[0084] In addition, in this embodiment, if the highest second matching degree is less than or equal to a preset threshold, no air conditioning command is executed, and a matching failure message is fed back.

[0085] Specifically, in some embodiments, the first CTC module in step S50 aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees, including but not limited to the following steps:

[0086] Step S51: Insert whitespace and repeat characters into each text embedding feature to obtain each extended sequence;

[0087] Step S52: Based on any extended sequence, calculate the cosine similarity between the acoustic embedding features of each frame and each character of the extended sequence to obtain the score matrix;

[0088] Step S53: Based on any score matrix, use dynamic programming algorithm to calculate the cumulative maximum score when the acoustic embedding feature of each frame is aligned to each character of the corresponding extended sequence, determine the reference score value corresponding to the last frame aligned to the acoustic embedding feature, determine the reference score value as the second matching degree, and record the path source of each reference score value to obtain the backtracking pointer matrix.

[0089] It is understood that the first CTC module in this embodiment is responsible for resolving the alignment problem between the acoustic embedding features corresponding to the speech signal and the text embedding features corresponding to the open vocabulary. By introducing special whitespace symbols into the open vocabulary, the probability of the final output label sequence is obtained by calculating the sum of the probabilities of all possible alignment paths, thus avoiding a complex manual alignment process. Specifically, the steps for calculating the second matching degree between the acoustic embedding features and the text embedding features corresponding to the open vocabulary in this embodiment include: inserting whitespace symbols and repeating characters into each text embedding feature to obtain each extended sequence; based on any extended sequence, calculating the cosine similarity between the acoustic embedding feature of each frame and each character of the extended sequence to obtain a score matrix; based on any score matrix, using a dynamic programming algorithm to calculate the cumulative maximum score when the acoustic embedding feature of each frame is aligned to each character of the corresponding extended sequence, determining the reference score value corresponding to the last frame aligned to the acoustic embedding feature, determining the reference score value as the second matching degree, and recording the path source of each reference score value to obtain a backtracking pointer matrix, providing an effective data basis for subsequently determining whether to trigger the execution of the air conditioning command.

[0090] Specifically, in some embodiments, in step S50, when the highest second matching degree exceeds a preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command, including but not limited to the following steps:

[0091] Step S54: Based on the backtracking pointer matrix, perform path backtracking on each reference score value to obtain the corresponding alignment paths;

[0092] Step S55: Remove whitespace characters and merge duplicate characters from each of the alignment paths to obtain each keyword;

[0093] Step S56: When the highest second matching degree exceeds the preset threshold, the keyword corresponding to the highest second matching degree is determined as the target keyword;

[0094] Step S57: Determine the air conditioning instructions corresponding to the open words in the open word mapping table that are the same as the target keywords as the target instructions.

[0095] As can be understood, referring to the description of the above embodiments, after calculating the second matching degree corresponding to each open word and acoustic feature, path backtracking is performed on each reference score value based on the backtracking pointer matrix to obtain the corresponding alignment paths; whitespace characters are removed and duplicate characters are merged on each alignment path to obtain each keyword; when the highest second matching degree exceeds a preset threshold, the keyword corresponding to the highest second matching degree is determined as the target keyword; the air conditioning command corresponding to the open word in the open word mapping table that is the same as the target keyword is determined as the target command. Thus, this embodiment realizes the fixed word wake-up task and the open word wake-up task through a voice wake-up model.

[0096] Specifically, in some embodiments, the text encoding module includes an input embedding layer, a BLSTM layer, and a text embedding module connected in sequence. The step of the text encoding module outputting text embedding features includes, but is not limited to, the following steps:

[0097] Step S61: When a user inputs a wake-up word setting request, which includes open words and air conditioning commands, a mapping relationship is established between each received open word and air conditioning command, and the information is saved to the open word mapping table.

[0098] Step S62: Input the open vocabulary into the input embedding layer for word segmentation to obtain multiple character features;

[0099] Step S63: Convert all character features into high-dimensional embedding vectors using a BLSTM layer;

[0100] Step S64: Convert the high-dimensional embedding vector into text embedding features through the text embedding module, and associate the text embedding features with the corresponding open words in the open vocabulary mapping table.

[0101] It is understood that the text encoding module in this embodiment can convert the open vocabulary (essentially keyword text) input by the user into text embedding features, so as to facilitate the comparison and alignment of acoustic embedding features and provide an effective data basis for calculating the second matching degree. Specifically, in this embodiment, the steps of the text encoding module outputting text embedding features include: when a user inputs a wake-up word setting request, which includes open words and air conditioning commands, a mapping relationship is established between each received open word and the air conditioning command, and the mapping relationship is saved to the open word mapping table; the open words are input to the input embedding layer for word segmentation processing to obtain multiple character features; then, all character features are converted into high-dimensional embedding vectors through the BLSTM layer. Thus, each character feature is mapped to a trainable look-up table, generating a 256-dimensional vector. These vectors capture the semantic information of the character features and provide a basis for subsequent processing. After processing by the BLSTM layer, each character feature is converted into a higher-level embedding vector (i.e., a high-dimensional embedding vector). The high-dimensional embedding vector captures the semantic information of the context of the character feature. Finally, the high-dimensional embedding vector is converted into text embedding features through the text embedding module, and the text embedding features are associated with the corresponding open words in the open word mapping table.

[0102] like Figure 2 As shown, Figure 2 This is a structural diagram of a control device provided in one embodiment of this application. The present invention also provides a control device 200, comprising:

[0103] The processor 210 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0104] The memory 220 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 220 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 220 and called and executed by the processor 210 to execute the air conditioner voice wake-up method of the embodiments of this application.

[0105] Input / output interface 230 is used to implement information input and output;

[0106] The communication interface 240 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0107] Bus 250 transmits information between various components of the device (e.g., processor 210, memory 220, input / output interface 230, and communication interface 240);

[0108] The processor 210, memory 220, input / output interface 230 and communication interface 240 are connected to each other within the device via bus 250.

[0109] In addition, this application also provides an electronic device, including the control device 200 described in the above embodiments.

[0110] In addition, this application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described air conditioner voice wake-up method.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0113] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for voice wake-up of an air conditioner, characterized in that, Applied to an air conditioning system, the method includes: When a speech signal is received, feature extraction is performed on the speech signal to obtain the target speech features; The target speech features are input into a preset speech wake-up model. The speech wake-up model includes an acoustic coding module, a first CTC module with a preset open vocabulary mapping table, and a text coding module. The acoustic coding module includes a convolution module, an acoustic embedding module, and a second CTC module with a preset fixed vocabulary mapping table. The fixed vocabulary mapping table represents the mapping relationship between preset fixed words and air conditioning commands, and the open vocabulary mapping table represents the mapping relationship between open words input by the user and air conditioning commands. The convolution module extracts acoustic features from the target speech features; The second CTC module calculates the first matching degree between the acoustic features and each fixed word in the fixed word mapping table. When the highest first matching degree exceeds a preset threshold, the air conditioning command corresponding to the highest first matching degree is determined as the target command, and the target command is executed. When the highest first matching degree is less than or equal to the preset threshold, the acoustic feature is input to the acoustic embedding module to obtain acoustic embedding features. The first CTC module aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees. When the highest second matching degree exceeds the preset threshold, the air conditioning instruction corresponding to the highest second matching degree is determined as the target instruction, and the target instruction is executed. The text embedding features are associated with the open vocabulary. The first CTC module aligns the acoustic embedding features with multiple text embedding features output by the text encoding module to obtain multiple second matching degrees, including: Each of the text embedding features is processed by inserting whitespace and repeating characters to obtain each extended sequence; Based on any of the extended sequences, the cosine similarity between the acoustic embedding features of each frame and each character of the extended sequence is calculated to obtain a score matrix; Based on any of the score matrices, a dynamic programming algorithm is used to calculate the cumulative maximum score when the acoustic embedding feature of each frame is aligned to each character of the corresponding extended sequence. The reference score value corresponding to the last frame aligned to the acoustic embedding feature is determined, and the reference score value is determined as the second matching degree. The path source of each reference score value is recorded to obtain a backtracking pointer matrix.

2. The air conditioner voice wake-up method according to claim 1, characterized in that, Feature extraction is performed on the speech signal to obtain target speech features, including: The speech signal is sequentially passed through a Log-MMSE speech estimator and an MCRA noise estimator for noise reduction to obtain an intermediate speech signal; The FBank feature extraction algorithm is used to extract features from the intermediate speech signal to obtain reference speech features; Differential feature calculation is performed on the reference speech features to obtain first-order and second-order differential features with a preset number of dimensions. The reference speech features, the first-order difference features, and the second-order difference features are determined as the target speech features.

3. The air conditioner voice wake-up method according to claim 1, characterized in that, The convolutional module includes a deep convolutional layer, a first combined layer, a pointwise convolutional layer, and a second combined layer connected in sequence. Both the first and second combined layers include a normalization layer and an activation function layer. The convolutional module extracts acoustic features from the target speech features, including: The target speech features are input into the deep convolutional layer, so that the deep convolutional layer performs a convolution operation on each channel of the target speech features to obtain multiple first intermediate features, wherein different first intermediate features correspond to different channels. All the first intermediate features are input into the first combination layer and normalized and activated sequentially to obtain multiple second intermediate features. All the second intermediate features are input into the pointwise convolutional layer so that the pointwise convolutional layer linearly combines each of the second intermediate features to obtain the third intermediate feature; The third intermediate feature is input into the second combined layer and subjected to normalization and activation processes in sequence to obtain the acoustic feature.

4. The air conditioner voice wake-up method according to claim 1, characterized in that, The text encoding module includes an input embedding layer, a BLSTM layer, and a text embedding module connected in sequence. The step of the text encoding module outputting the text embedding features includes: When a user inputs a wake-up word setting request, which includes the open vocabulary and the air conditioning command, a mapping relationship is established between each received open vocabulary and the air conditioning command, and the data is saved to the open vocabulary mapping table. The open vocabulary is input into the input embedding layer for word segmentation to obtain multiple character features; The BLSTM layer converts all the character features into high-dimensional embedding vectors. The high-dimensional embedding vector is converted into the text embedding feature by the text embedding module, and the text embedding feature is associated with the corresponding open vocabulary in the open vocabulary mapping table.

5. The air conditioner voice wake-up method according to claim 1, characterized in that, When the highest second matching degree exceeds the preset threshold, the air conditioning command corresponding to the highest second matching degree is determined as the target command, including: Based on the backtracking pointer matrix, path backtracking is performed on each of the reference score values ​​to obtain the corresponding alignment paths; The alignment paths are processed by removing whitespace and merging duplicate characters to obtain the keywords. When the highest second matching degree exceeds the preset threshold, the keyword corresponding to the highest second matching degree is determined as the target keyword; The air conditioning instruction corresponding to the open vocabulary in the open vocabulary mapping table that is the same as the target keyword is determined as the target instruction.

6. The air conditioner voice wake-up method according to claim 1, characterized in that, The voice wake-up model was trained using the following method: Perform data preprocessing and data augmentation on the training set; The feature extraction algorithm is used to extract features from each speech signal in the training set to obtain each target speech feature; For the fixed-vocabulary wake-up task, the convolutional module of the acoustic coding module and the second CTC module in the initial model are trained based on the CTC loss function and all the target speech features; For the open vocabulary wake-up task, the first CTC module, the text encoding module, the convolutional module of the acoustic encoding module, and the acoustic embedding module in the initial model are trained by combining the CTC loss function, the multi-view loss function, and all the target speech features. The training process continues until the model parts corresponding to the fixed vocabulary wake-up task and the open vocabulary wake-up task have both reached convergence, thus obtaining the trained speech wake-up model.

7. A control device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the air conditioner voice wake-up method as described in any one of claims 1 to 6.

8. An air conditioning system, characterized in that, Includes the control device as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the air conditioner voice wake-up method as described in any one of claims 1 to 6.

Images