Set top box terminal voice control method and device based on convolutional neural network
By locally building a voice control interface and training a convolutional neural network model on the set-top box terminal, the problems of high latency and privacy leakage of voice control in the prior art are solved, and fast response and efficient voice control are achieved, while enhancing data security.
Patent Information
- Application Number
- CN202411630712.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-04-04
AI Technical Summary
The voice control of existing set-top box terminals relies on cloud processing, resulting in high response delays and risk of user privacy leakage.
The voice control method of set-top box terminal based on convolutional neural network is adopted. By building a voice control interface locally in the set-top box, the command audio data set is generated, and the convolutional neural network model is trained to realize local voice recognition and command execution.
It significantly reduces command response time, provides a smoother user experience, improves voice command recognition efficiency, enhances user privacy protection, and reduces the risk of data leakage.
Smart Images
Figure CN119767082A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of set-top boxes, and in particular to a method and device for voice control of a set-top box terminal based on a convolutional neural network. Background Art
[0002] With the popularity of smart homes, set-top box terminals, as important home entertainment devices, are gradually introducing voice control functions. However, existing voice recognition technologies usually rely on cloud processing, resulting in high latency and the risk of user privacy leakage. Therefore, an efficient and reliable local voice command control method is urgently needed to solve the following problems:
[0003] (1) Dependence on cloud processing: Voice recognition relies on network transmission, resulting in high latency in terminal control response, affecting user experience
[0004] (2) User privacy cannot be guaranteed: Third-party applications usually have voice collection permissions, and users’ voice data may be used improperly, posing the risk of privacy leakage. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a set-top box terminal voice control method and device based on convolutional neural network, which can achieve rapid response of voice control instructions on the set-top box terminal while ensuring user data security.
[0006] In a first aspect, the present invention provides a set-top box terminal voice control method based on a convolutional neural network, comprising:
[0007] Interface instruction construction process: constructing a local operation interface of a set-top box, the local operation interface including a local operation method and a local operation parameter; setting a command action corresponding to each local operation method to obtain a command action text set, and setting a command target corresponding to each local operation parameter to obtain a command target text set;
[0008] Recognition model training process: generating a command audio data set according to the command action text set and the command target text set, then using the command audio data set to train the convolutional neural network model, and loading the trained convolutional neural network model into the local set-top box;
[0009] Instruction extraction and execution process: When the instruction to collect audio signals is received, the voice stream is obtained and preprocessed, and the trained convolutional neural network model is input for reasoning to obtain the instruction action and instruction target, and then the set-top box local operation interface is called to perform the corresponding local operation.
[0010] Furthermore, during the recognition model training process, an instruction audio dataset is generated based on the instruction action text set and the instruction target text set, specifically including: generating an instruction action audio dataset based on the instruction action text set, generating an instruction target audio dataset based on the instruction target text set, and then expanding the instruction action audio dataset and the instruction target audio dataset respectively through data enhancement technology to obtain the instruction audio dataset.
[0011] Furthermore, during the training process of the recognition model, the convolutional neural network model is a neural network model of parallel multi-scale convolution combined with SE attention mechanism, including a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition.
[0012] Furthermore, the dilated convolution branch sequentially includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer; the standard convolution branch sequentially includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer.
[0013] In a second aspect, the present invention provides a set-top box terminal voice control device based on a convolutional neural network, comprising:
[0014] The interface instruction construction module is used to construct a local operation interface of the set-top box, the local operation interface includes a local operation method and a local operation parameter; a command action is set for each local operation method to obtain a command action text set, and a command target is set for each local operation parameter to obtain a command target text set;
[0015] A recognition model training module is used to generate an instruction audio data set according to the instruction action text set and the instruction target text set, and then use the instruction audio data set to train the convolutional neural network model, and load the trained convolutional neural network model to the local set-top box;
[0016] The instruction extraction and execution module is used to obtain and preprocess the voice stream when receiving the instruction to collect audio signals, input the trained convolutional neural network model for reasoning, obtain the instruction action and instruction target, and then call the set-top box local operation interface to perform the corresponding local operation.
[0017] Furthermore, in the interface instruction construction module, an instruction audio dataset is generated according to the instruction action text set and the instruction target text set, specifically including: generating an instruction action audio dataset according to the instruction action text set, generating an instruction target audio dataset according to the instruction target text set, and then expanding the instruction action audio dataset and the instruction target audio dataset respectively through data enhancement technology to obtain the instruction audio dataset.
[0018] Furthermore, in the recognition model training module, the convolutional neural network model is a neural network model of parallel multi-scale convolution combined with SE attention mechanism, including a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition.
[0019] Furthermore, the dilated convolution branch sequentially includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer; the standard convolution branch sequentially includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer.
[0020] The technical solution provided in the embodiments of the present invention has at least the following technical effects:
[0021] Reduced latency: By performing voice recognition locally, the reliance on the cloud is reduced, which significantly reduces the command response time and provides a smoother user experience. The effect of reducing latency depends on the specific network environment and is usually between 50 and 500 milliseconds.
[0022] Improve the efficiency of voice command recognition: Using keyword recognition technology, compared with general voice-to-text technology, it improves the efficiency of computational reasoning and lowers the hardware threshold. The efficiency of computational reasoning is increased by about 50%.
[0023] Enhanced privacy protection: Voice data is processed within the device and does not need to be uploaded to the cloud, reducing the risk of user privacy leakage and enhancing data security.
[0024] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.
[0026] Figure 1 It is a schematic diagram of the overall concept of the present invention;
[0027] Figure 2 is a flow chart of the method in Embodiment 1 of the present invention;
[0028] Figure 3 Schematic diagram of the structure of MS-SE-Net in Embodiment 1 of the present invention;
[0029] Figure 4 It is a schematic diagram of the structure of the device in the second embodiment of the present invention. DETAILED DESCRIPTION
[0030] The embodiment of the present invention provides a set-top box terminal voice control method and device based on a convolutional neural network, which can achieve rapid response to voice control instructions on the set-top box terminal while ensuring user data security.
[0031] The technical solution in the embodiment of the present invention is as follows: Figure 1 As shown, the overall idea is as follows:
[0032] 1. Audio acquisition and preprocessing: The built-in microphone module of the remote control uses a key wake-up method to collect audio signals to reduce power consumption, and processes the audio, including noise suppression and echo cancellation.
[0033] 2. Voice recognition: The set-top box has a built-in voice recognition module, which uses a custom hardware chip to perform convolutional neural network inference on a small embedded device. Through end-to-end voice keyword recognition technology, voice features are matched to commands.
[0034] 3. Command execution: Combined with the local control capability of the set-top box, several preset command keywords are constructed. The set-top box command execution module calls the interface to perform corresponding local operations based on the command results matched by audio recognition.
[0035] Embodiment 1
[0036] This embodiment provides a set-top box terminal voice control method based on convolutional neural network, such as Figure 2 As shown, the following steps may be included:
[0037] S1. Interface instruction construction process: construct a local operation interface of a set-top box, the local operation interface includes a local operation method and a local operation parameter; set a command action for each local operation method to obtain a command action text set, and set a command target for each local operation parameter to obtain a command target text set.
[0038] This patent is applied to the task of voice control command recognition for set-top boxes. Command keywords consist of two parts, command action and command target. Command actions are usually a series of verb keywords with similar meanings, while command targets are noun keywords related to actions. Here are some examples.
[0039]
[0040] S2, recognition model training process: generate an instruction audio data set according to the instruction action text set and the instruction target text set, then use the instruction audio data set to train the convolutional neural network model, and load the trained convolutional neural network model to the local set-top box.
[0041] For specific audio keyword recognition tasks, since the training effect using existing general data sets is not good, this embodiment uses speech synthesis and data enhancement to construct a data set. Use common speech synthesis products on the market (such as Ali Voice, Baidu Voice, etc.) to generate audio data of different styles according to the instruction action text set and the instruction target text set. Finally, the scale of the data set is further expanded through data enhancement technology to improve the generalization ability of the model. The following are optional audio data enhancement technologies suitable for this task and their effects:
[0042]
[0043] Specifically, speech generation technology is used to generate instruction action audio data sets of various styles based on the instruction action text set, and speech generation technology is used to generate instruction target audio data sets of various styles based on the instruction target text set. Then, the instruction action audio data set and the instruction target audio data set are expanded respectively through data enhancement technology, and finally a complete instruction audio data set is obtained, which increases the diversity of data and improves the generalization ability of the model. These two data sets can be trained separately in subsequent steps. Dividing the training and reasoning into two stages can reduce the demand for resources. The post-processing module will combine the results of these two parts to match the complete instruction.
[0044] Preferably, during the training process of the recognition model, this embodiment proposes a neural network model (Multi-Scale Squeeze-and-Excitation Network, MS-SE-Net) that combines parallel multi-scale convolution with SE (Squeeze-and-Excitation Attention) attention mechanism, which uses asymmetric convolution and asymmetric dilated convolution to process feature maps in parallel to capture global features and local features, respectively. The SE attention mechanism is applied to the feature map after the convolution operation to enhance the expressiveness of the feature map. This network model is used in the speech recognition module to optimize computational efficiency and model complexity, ensure resource utilization efficiency of embedded devices, and improve the accuracy of model recognition.
[0045] The general idea proposed by MS-SE-Net is that the input feature map is divided into multiple parallel branches after reducing the number of channels, and each branch uses different convolution kernels to extract features. One branch uses dilated convolution to expand the receptive field by inserting holes in the convolution operation, which is mainly used to capture global features. The other branch uses standard convolution, focusing on capturing local features and maintaining sensitivity to details. The convolution operation in each branch uses asymmetric convolution kernels (such as 1×5 and 5×1) to further enrich the directionality and diversity of feature extraction. The outputs of the parallel branches are fused by point-by-point addition. This step combines global and local features so that the fused feature map contains both extensive contextual information and retains details. The fused feature map is restored to the number of channels through a 1×1 convolution layer. This layer ensures that the number of output channels is consistent with network requirements and can further integrate and enhance features. Through parallel multi-scale convolution operations, the network can process global and local features simultaneously, making the model more robust when processing input data with complex structures and better able to capture multi-scale feature information.
[0046] The overall process of MS-SE-Net is as follows:
[0047] First, the audio features of the input audio signal are extracted using Mel-Frequency Cepstral Coefficients (MFCCs). MFCCs converts the audio signal into a set of coefficients that can effectively capture the frequency distribution and timbre characteristics of the audio. The coefficient matrix is a two-dimensional matrix of dimension C×T, which is used as the feature matrix of the audio and is directly input into MS-SE-Net after simple noise reduction processing.
[0048] The convolution layer of MS-SE-Net uses parallel multi-scale convolution, specifically asymmetric convolution and asymmetric dilated convolution, to extract local and high-level features. Then the SE attention mechanism is applied to the output feature map Y of the multi-scale convolution to obtain the channel-weighted feature map. Then it is passed to the pooling layer, and the feature is reduced and compressed by global average pooling; then the output of the pooling layer is connected to the fully connected layer; finally, the output of the fully connected layer is sent to the output layer, and the Softmax function is used as the output layer activation function to identify the position with the maximum posterior probability as the corresponding speech keyword.
[0049] The specific description is as follows:
[0050] The use of two-dimensional dilation, that is, the convolution kernel dilates in both time and frequency scales, aims to obtain richer time-frequency related information. The time scale corresponds to the time frame (i.e., frame dimension) in the MFCCs feature matrix, which is usually the time series part of the audio signal. The frequency scale corresponds to the frequency features (i.e., spectrum dimension) in the MFCCs feature matrix, which represents the frequency components in the audio signal.
[0051] Assume that the size of the two-dimensional convolution kernel F is (2n+1)×(2n+1), and the expression of the output feature is:
[0052]
[0053] Among them, X and Y represent input features and output features respectively, d represents the expansion rate, p,s∈Z2, and Z represents an integer set.
[0054] The input feature map first passes through a 3×3 2D convolution layer (Conv2d). The convolution operation halves the number of channels, making the size of the feature map become T×F×(C / 2), where T and F represent the time frame and frequency dimensions, respectively. Next, the feature map with reduced channels is processed in parallel through two asymmetric convolution branches. The right branch uses 1×5 and 5×1 2D dilated convolutions with consistent dilation rates to expand the receptive field and enhance the extraction of global features; the left branch uses 1×3 and 3×1 standard 2D convolutions to capture local features. The outputs of the two branches are merged by point-by-point addition. Finally, the merged feature map is restored to the number of channels through a 1×1 convolution layer.
[0055] The SE attention mechanism is applied to the output feature map Y of the multi-scale convolution. The SE module can adaptively learn the relationship between channels, highlight important channels, and suppress irrelevant channels, thereby improving the expressiveness of the feature map. At the same time, since the SE module has low computational complexity and does not require additional parameters or computational overhead, it can be easily integrated into the existing network architecture. First, the input multi-scale convolution feature map Y is globally average pooled to compress spatial information and obtain a global feature description of each channel. For each channel c of the input feature map Y, the result of global average pooling is:
[0056]
[0057] Among them, H and W represent the height and width of the feature map respectively.
[0058] Finally, we get a channel description vector z with a dimension of [C], where C is the number of channels. Next, we map the pooled channel description vector z to the attention weight α through two fully connected layers. The first fully connected layer: reduces the dimension of z from [C] to [C / r], where r is the reduction ratio:
[0059] s=ReLU(W1·z+b1)
[0060] Where W1 is a weight matrix of shape [C / r,C] and b1 is a bias term.
[0061] The second fully connected layer: restore the dimension of s to [C]
[0062] α=σ(W2·s+b2)
[0063] Among them, W2 is a weight matrix of shape [C, C / r], b2 is a bias term, and σ represents the Sigmoid activation function.
[0064] Finally, the generated attention weight α is applied to each channel of the input feature map Y to obtain the weighted output feature map Y out :
[0065] Y out =Y·α
[0066] Here, · represents channel-by-channel multiplication. This operation enhances the features of important channels while suppressing irrelevant or unimportant channels.
[0067] The weighted feature map Y′ is passed to the subsequent fully connected layer to map the previously extracted features to the number of target categories. For the keyword spotting task, the output of the classification layer is a vector in which each element represents the unnormalized score of a category. Finally, the output layer uses the Softmax function to convert the output of the fully connected layer into a probability distribution, and the probability value of each category represents the possibility that the audio clip belongs to the corresponding category.
[0068] The MS-SE-Net neural network module uses asymmetric convolution and asymmetric dilated convolution to process the extracted audio features in parallel. This allows the fused feature map to contain extensive contextual information while retaining details. The SE attention mechanism is applied to the feature map after the convolution operation to enhance the expressiveness of the feature map. The channel-weighted output is connected to the fully connected layer after global average pooling, and the output is converted into a probability distribution through the Softmax layer. The probability value of each category indicates the possibility that the audio clip belongs to that category. The Softmax probability distribution is then converted into the final matching target using a simple threshold judgment and maximum probability selection method through the post-processing module. Matching selection is performed separately for the instruction action and instruction target.
[0069] In this embodiment, if Figure 3As shown, MS-SE-Net includes a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition. The dilated convolution branch includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer in sequence; the standard convolution branch includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer in sequence.
[0070] S3, instruction extraction and execution process: When the instruction to collect audio signals is received, the voice stream is obtained and preprocessed, and the trained convolutional neural network model is input for reasoning to obtain the instruction action and instruction target, and then the set-top box local operation interface is called to perform the corresponding local operation.
[0071] Specifically, the built-in microphone of the remote control is used to collect audio signals, and the key wake-up method is used to reduce power consumption. The audio signal is preprocessed, including noise reduction and audio enhancement, to improve the signal quality. The Mel frequency cepstral coefficients of the audio are extracted as the feature representation of the speech signal. After simple noise reduction processing, the feature matrix is used as the input of the neural network.
[0072] Compared with convolutional neural networks, the parallel multi-scale convolution neural network model combined with SE attention mechanism proposed in the embodiment of the present invention effectively expands the receptive field of the model through multi-scale parallel convolutional neural networks combined with SE attention mechanism, while maintaining low computational complexity. Therefore, this method can also run efficiently on resource-constrained embedded devices (such as set-top box terminals) without adding additional hardware burden. At the same time, compared with convolutional neural networks, the number of model parameters proposed in this embodiment has not increased substantially, but the recognition rate has increased by about 19.8%, significantly improving the recognition accuracy of low-resource embedded devices.
[0073] Based on the same inventive concept, the present application also provides a device corresponding to the method in Example 1, see Example 2 for details.
[0074] Embodiment 2
[0075] In this embodiment, a set-top box terminal voice control device based on a convolutional neural network is provided. Figure 4 As shown, including:
[0076] The interface instruction construction module is used to construct a local operation interface of the set-top box, the local operation interface includes a local operation method and a local operation parameter; a command action is set for each local operation method to obtain a command action text set, and a command target is set for each local operation parameter to obtain a command target text set;
[0077] A recognition model training module is used to generate an instruction audio data set according to the instruction action text set and the instruction target text set, and then use the instruction audio data set to train the convolutional neural network model, and load the trained convolutional neural network model to the local set-top box;
[0078] The instruction extraction and execution module is used to obtain and preprocess the voice stream when receiving the instruction to collect audio signals, input the trained convolutional neural network model for reasoning, obtain the instruction action and instruction target, and then call the set-top box local operation interface to perform the corresponding local operation.
[0079] Furthermore, in the interface instruction construction module, an instruction audio dataset is generated according to the instruction action text set and the instruction target text set, specifically including: generating an instruction action audio dataset according to the instruction action text set, generating an instruction target audio dataset according to the instruction target text set, and then expanding the instruction action audio dataset and the instruction target audio dataset respectively through data enhancement technology to obtain the instruction audio dataset.
[0080] Furthermore, in the recognition model training module, the convolutional neural network model is a neural network model of parallel multi-scale convolution combined with SE attention mechanism, including a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition.
[0081] Furthermore, the dilated convolution branch sequentially includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer; the standard convolution branch sequentially includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer.
[0082] Since the device introduced in the second embodiment of the present invention is a device used to implement the method of the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device based on the method introduced in the first embodiment of the present invention, so it is not described here in detail. All devices used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.
[0083] The present invention reduces the reliance on the cloud by performing voice recognition locally, thereby significantly reducing the command response time and providing a smoother user experience. The effect of reducing latency depends on the specific network environment, usually between 50 and 500 milliseconds; the use of keyword recognition technology improves the efficiency of computational reasoning and reduces the hardware threshold compared to general voice-to-text technology, and the efficiency of computational reasoning is increased by about 50%; voice data is processed inside the device and does not need to be uploaded to the cloud, reducing the risk of user privacy leakage and enhancing data security.
[0084] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0085] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.
[0088] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.
Claims
1. A set-top box terminal voice control method based on convolutional neural network, characterized in that: include: Interface instruction construction process: constructing a local operation interface of a set-top box, the local operation interface includes a local operation method and local operation parameters; A command action is set corresponding to each local operation method to obtain a command action text set, and a command target is set corresponding to each local operation parameter to obtain a command target text set; Recognition model training process: Generate a command audio dataset based on the command action text set and the command target text set, then use the command audio dataset to train the convolutional neural network model, and load the trained convolutional neural network model to the local set-top box; Instruction extraction and execution process: When the instruction to collect audio signals is received, the voice stream is obtained and preprocessed, and the trained convolutional neural network model is input for reasoning to obtain the instruction action and instruction target, and then the set-top box local operation interface is called to perform the corresponding local operation.
2. The method according to claim 1, characterized in that: During the recognition model training process, an instruction audio data set is generated according to the instruction action text set and the instruction target text set, specifically including: generating an instruction action audio data set according to the instruction action text set, generating an instruction target audio data set according to the instruction target text set, and then expanding the instruction action audio data set and the instruction target audio data set respectively through data enhancement technology to obtain the instruction audio data set.
3. The method according to claim 1, characterized in that: During the recognition model training process, the convolutional neural network model is a neural network model of parallel multi-scale convolution combined with SE attention mechanism, including a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition.
4. The method according to claim 3, characterized in that: The dilated convolution branch includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer in sequence; the standard convolution branch includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer in sequence.
5. A set-top box terminal voice control device based on convolutional neural network, characterized in that: include: An interface instruction building module is used to build a local operation interface of a set-top box, wherein the local operation interface includes a local operation method and a local operation parameter; A command action is set corresponding to each local operation method to obtain a command action text set, and a command target is set corresponding to each local operation parameter to obtain a command target text set; A recognition model training module is used to generate a command audio data set from the command action text set and the command target text set, and then train a convolutional neural network model using the command audio data set, and load the trained convolutional neural network model to a local set-top box; The instruction extraction and execution module is used to obtain and preprocess the voice stream when receiving the instruction to collect audio signals, input the trained convolutional neural network model for reasoning, obtain the instruction action and instruction target, and then call the set-top box local operation interface to perform the corresponding local operation.
6. The device according to claim 5, characterized in that: In the interface instruction construction module, an instruction audio dataset is generated according to an instruction action text set and an instruction target text set, specifically including: generating an instruction action audio dataset according to the instruction action text set, generating an instruction target audio dataset according to the instruction target text set, and then expanding the instruction action audio dataset and the instruction target audio dataset respectively by data enhancement technology to obtain the instruction audio dataset.
7. The device according to claim 5, characterized in that: In the recognition model training module, the convolutional neural network model is a neural network model of parallel multi-scale convolution combined with SE attention mechanism, including a two-dimensional convolution layer that reduces the number of channels by half, parallel dilated convolution branches and standard convolution branches, and a two-dimensional convolution layer that restores the number of channels; the parallel dilated convolution branches and standard convolution branches use asymmetric convolution kernels, and their outputs are fused by point-by-point addition.
8. The device according to claim 7, characterized in that: The dilated convolution branch includes a 1×5 two-dimensional dilated convolution layer, a batch normalization layer, a 5×1 two-dimensional dilated convolution layer, and a batch normalization layer in sequence; the standard convolution branch includes a 1×3 two-dimensional convolution layer, a batch normalization layer, a 3×1 two-dimensional convolution layer, and a batch normalization layer in sequence.