Smart classroom speech recognition method, system and device based on neural network, and medium

Abstract

The invention relates to a smart classroom speech recognition method, system and device based on a neural network, and a medium. The method comprises the following steps: acquiring noisy voice data of a classroom environment and performing feature extraction to obtain a voice feature set; identifying a dominant classroom noise type through a noise classifier to obtain a noise type identifier; processing the voice feature set through convolution kernel configuration corresponding to the noise type identifier to obtain local precise features, and processing the local precise features through a multi-head self-attention mechanism to obtain key focusing features; based on a fusion weight corresponding to the noise type identifier, carrying out weighted summation on the local precise feature and the key focusing feature to generate a fusion feature; performing channel-level attention weight calculation on the fusion features and performing feature data screening to obtain lightweight optimization features; and inputting the lightweight optimization features into a classification decoder to obtain a speech recognition text result. The method can be adapted to a complex noise environment in a classroom, and the accuracy of speech recognition is improved.

Description

Technical Field

[0001] This invention belongs to the field of auxiliary teaching technology, and in particular relates to a method, system, device and medium for smart classroom speech recognition based on neural networks. Background Technology

[0002] With the development of technology in the field of smart education, real-time classroom voice interaction technology has emerged, which improves teaching efficiency and supports the promotion of personalized learning by automatically transcribing teacher-student questions and answers and recording and analyzing classroom discussion content in real time.

[0003] Traditional speech recognition mostly adopts solutions adapted to a single steady-state environment such as a quiet office or a standard recording studio. It standardizes the input speech by using fixed noise suppression algorithms, uniformly sized convolutional kernels, and static feature fusion weights. That is, it completes speech feature extraction and recognition based on a preset single noise model, without performing specific optimizations for complex scenarios.

[0004] However, the above methods suffer from extremely poor noise adaptability of traditional speech recognition models. The noise models they design have a very low matching degree with the multi-source dynamic noise field in the classroom, which makes speech features easily distorted or submerged in the classroom environment, resulting in a significant increase in word error rate. In addition, traditional high-performance speech recognition models have a large number of parameters and high computational complexity, making them unsuitable for resource-constrained edge devices such as teaching tablets. Meanwhile, lightweight traditional models designed for mobile devices sacrifice model expressive power and noise resistance through aggressive pruning and quantization, making it difficult to meet the deployment requirements of real-time voice interaction in the classroom. Summary of the Invention

[0005] Therefore, it is necessary to provide a neural network-based smart classroom speech recognition method, system, device, and medium that can adapt to the dynamic characteristics of multi-source noise in the classroom and address the above-mentioned technical problems.

[0006] Firstly, this application provides a neural network-based smart classroom speech recognition method, including: Acquire noisy speech data from the classroom environment and extract acoustic features from the noisy speech data to obtain a speech feature set; The dominant classroom noise type is identified in the speech feature set by a pre-trained noise classifier, and a noise type identifier is obtained. By using the convolution kernel configuration corresponding to the noise type identifier, convolution operations and nonlinear activation processing are performed on the speech feature set to obtain local precise features. Then, key speech information is focused on the local precise features through a multi-head self-attention mechanism to obtain key focused features. Based on noise type identification, the fusion weight is calculated through dynamic weighting, and the local precise features and key focused features are weighted and summed according to the fusion weight to generate fusion features; Channel-level attention weights are calculated for the fused features, and feature data is filtered based on the calculated channel-level attention weights to obtain lightweight optimized features; Lightweight optimized features are input into a pre-trained classifier decoder for temporal classification and decoding to obtain speech recognition text results.

[0007] In one embodiment, the noise classifier includes a first convolutional layer, a pooling layer, a second convolutional layer, and a fully connected output layer connected in sequence. The dominant classroom noise type in the speech feature set is identified by a pre-trained noise classifier, resulting in noise type identifiers, including: Based on the input layer dimension of the noise classifier, the acoustic features in the speech feature set are dimensionally normalized to obtain standardized input features. Local features are extracted from the standardized input features through the first and second convolutional layers, and then downsampled through pooling layers to obtain the feature map. The feature map is unfolded into a feature vector, and the softmax activation function is applied to the feature vector through a fully connected output layer to obtain the classroom noise probability distribution vector. Select the noise type corresponding to the maximum probability value in the classroom noise probability distribution vector to obtain the noise type identifier.

[0008] In one embodiment, by using the convolution kernel configuration corresponding to the noise type identifier, convolution operations and nonlinear activation processing are performed on the speech feature set to obtain local precise features, including: If the noise type is identified as conversation, the convolution kernel configuration is set to the first configuration; if the noise type is identified as table and chair movement noise or multimedia noise, the convolution kernel configuration is set to the second configuration. The first configuration uses a pre-configured fine-grained feature convolution kernel with a unit stride; the second configuration uses a pre-configured large receptive field feature convolution kernel with a first step size. Based on the convolution kernel configuration, a two-dimensional convolution operation is performed on the speech feature set to obtain the initial convolution feature map; The initial convolutional feature maps are batch normalized, and the ReLU nonlinear activation function is applied to the batch normalized initial convolutional feature maps to obtain local precise features.

[0009] In one embodiment, based on the noise type identifier, a fusion weight is calculated using dynamic weighting, and the local precise features and key focused features are weighted and summed according to the fusion weight to generate fused features, including: The noise type identifier is one-hot encoded to obtain an encoded vector, and the encoded vector is input into a pre-trained dynamic weight calculation model to obtain fusion weight coefficients; the fusion weight coefficients include local precise feature weights and key focused feature weights; Based on the weight constraint range corresponding to the noise type identifier, the fusion weight coefficients are adjusted to obtain the adjusted local precise feature weights and key focus feature weights. Then, the local precise features and key focus features are weighted and fused based on the adjusted local precise feature weights and key focus feature weights to obtain the fused features.

[0010] In one embodiment, channel-level attention weights are calculated on the fused features, and feature data is filtered based on the calculated channel-level attention weights to obtain lightweight optimized features, including: Global average pooling is used to compress the two-dimensional feature maps of each channel in the fused features into scalars, resulting in channel-level statistical vectors for each channel. The importance score of each channel is calculated based on the channel-level statistical vector, and the attention weight vector of each channel is obtained. According to the preset pruning ratio, all channel weights in the attention weight vector are sorted in descending order, and the channels with the highest sorting ratio are marked as core channels to obtain binary channel masks; the binary channel mask includes core channels and redundant channels. The fused features are multiplied channel by channel with the binary channel mask to obtain lightweight optimized features.

[0011] Secondly, this application also provides a smart classroom speech recognition system based on neural networks, including: The classroom audio module is used to acquire noisy speech data from the classroom environment and extract acoustic features from the noisy speech data to obtain a speech feature set. The noise type module is used to identify the dominant classroom noise type in the speech feature set through a pre-trained noise classifier, and obtain the noise type identifier. The feature module is used to perform convolution operations and nonlinear activation processing on the speech feature set by identifying the corresponding convolution kernel configuration through noise type to obtain local precise features, and to perform key speech information focusing processing on the local precise features through a multi-head self-attention mechanism to obtain key focused features; The feature fusion module is used to calculate the fusion weight based on the noise type identifier through dynamic weights, and to generate fused features by weighted summation of local precise features and key focused features according to the fusion weights; The feature filtering module is used to calculate channel-level attention weights for the fused features and filter the fused features based on the calculated channel-level attention weights to obtain lightweight optimized features. The text decoding module is used to input lightweight optimized features into a pre-trained classifier decoder for temporal classification decoding to obtain the speech recognition text results.

[0012] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-mentioned neural network-based smart classroom speech recognition methods.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-described neural network-based smart classroom speech recognition methods.

[0014] The aforementioned neural network-based smart classroom speech recognition method, system, device, and medium extract acoustic features from noisy classroom speech, identify the dominant noise type based on a pre-trained noise classifier, and generate a noise type label. Based on the noise type label, a suitable convolutional kernel configuration is dynamically selected for feature extraction, while simultaneously focusing key speech information through an attention mechanism, resulting in both local precise features and key focused features. Fusion weights are dynamically calculated based on the same noise label, and the two types of features, local precise features and key focused features, are weighted and fused. Channel-level attention weights are calculated and filtered for the fused features to achieve feature lightweighting. The lightweight optimized features are then decoded into text, significantly improving speech recognition accuracy in complex and varied classroom noise environments. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the neural network-based smart classroom speech recognition method of the present invention. Figure 2 This is a flowchart illustrating the steps of step S102. Figure 3 This is a flowchart illustrating the steps of step S103. Figure 4 This is a structural diagram of the neural network-based smart classroom speech recognition system of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] In one embodiment, such as Figure 1 As shown, a neural network-based speech recognition method for smart classrooms is provided. This embodiment illustrates the application of this method to a terminal, but it is understood that the method can also be applied to servers and systems including terminals and servers, and implemented through interaction between the terminal and the server. In high-concurrency application scenarios of smart classrooms, such as large university classes with hundreds of students or cross-regional synchronous online classrooms, edge nodes deployed in different locations within the classroom, such as teaching tablets and embedded computing devices, continuously monitor their real-time resource status, including CPU utilization, memory usage, and network throughput. This data is combined with the noise type identifier corresponding to the currently processed speech stream to form a comprehensive real-time node load vector. A central coordinator or a server based on a distributed protocol network is responsible for collecting and parsing the load vectors of all nodes. When the system detects that a source node exceeds a preset overload threshold due to excessive processing load, an intelligent collaborative migration decision mechanism is triggered. This mechanism selects the optimal target node based on the remaining resource capacity of each potential target node, combined with its historical performance or configuration advantages in handling specific types of classroom noise. Based on this decision, the source node encapsulates the key state of the speech task it is processing, including the generated lightweight optimized features and the context information necessary to maintain decoding continuity, into a lightweight context transfer packet. Through a dual-channel transmission mechanism, the context transfer packet and the continuous real-time speech feature stream are transmitted in parallel and stably to the target node, with integrity checks implemented during transmission to ensure data accuracy. Upon successful reception, the target node calculates the minute time-series deviation between speech streams from different nodes based on a unified global clock reference and the high-precision timestamps carried in each data packet, and applies a frame-level latency compensation algorithm for precise alignment. The multiple speech feature streams that have achieved temporal synchronization are respectively fed into independent classification decoder instances for parallel decoding. The recognition results are summarized and integrated, thereby supporting smooth, accurate, and low-latency full-class speech interaction and transcription services in a large-scale, high-concurrency classroom environment. In this embodiment, the method includes the following steps: S101. Obtain noisy speech data from the classroom environment and extract acoustic features from the noisy speech data to obtain a speech feature set.

[0019] In a demonstrative manner, noisy speech data from a classroom environment is collected using devices such as teaching tablets deployed in the teaching setting, covering various classroom interaction scenarios such as teacher lectures, student Q&A, and group discussions. The collected noisy speech data undergoes preprocessing, including pre-emphasis processing to compensate for the attenuation of high-frequency components in the speech signal, which can be implemented using a first-order FIR filter. Further, frame segmentation is performed, dividing the continuous speech signal into a fixed-length frame sequence, with overlapping regions between frames to avoid loss of frame edge information. After framing, a window function is applied to each frame to reduce spectral leakage; common window function types such as Hamming, Hanning, or Blackman windows can be used. After preprocessing, acoustic feature extraction is performed. The extracted acoustic features can be any one or more combinations of Mel-frequency cepstral coefficients (MFCC), Mel spectrograms, log-Mel spectrograms, linear predictive cepstral coefficients (LPCC), or perceptual linear predictive (PLP) features. During feature extraction, each segment of the speech signal needs to be converted into a regular feature matrix form, and all extracted feature matrices together constitute the speech feature set.

[0020] S102. Identify the dominant classroom noise type in the speech feature set using a pre-trained noise classifier to obtain a noise type identifier.

[0021] Optionally, the pre-trained noise classifier is a lightweight neural network model whose network structure meets the deployment requirements of edge devices. While ensuring classification accuracy, the number of parameters and computational complexity are controlled. The model type can be a lightweight variant of a convolutional neural network (CNN), a multilayer perceptron (MLP), or a Transformer encoder. For example, when using the noise classifier to identify the dominant noise type, each feature matrix in the speech feature set is input into the classifier. The classifier calculates the noise type probability distribution corresponding to the input feature through forward propagation. This probability distribution contains the probability values ​​corresponding to various preset noise types in the classroom scenario. The preset noise types must cover the main interfering noise categories in the classroom environment. The dominant classroom noise type is defined as the noise type with the highest probability value in the probability distribution. Based on this dominant noise type, a corresponding noise type identifier is generated. The noise type identifier is a discrete symbol that can uniquely distinguish various types of noise. It can be in the form of integer encoding, one-hot encoding, or string identifier, and its encoding rules are consistent with the training and labeling rules of the noise classifier.

[0022] S103. By using the convolution kernel configuration corresponding to the noise type identifier, convolution operation and nonlinear activation processing are performed on the speech feature set to obtain local precise features. Then, the key speech information is focused on the local precise features through a multi-head self-attention mechanism to obtain key focused features.

[0023] Specifically, convolutional kernel configurations are predefined for different noise type identifiers. Each convolutional kernel configuration includes kernel size, stride, number, and padding method. The design of different configurations is based on the spectral characteristics and interference patterns of various noise types, enabling the corresponding convolutional kernels to effectively extract key speech features after interference by that type of noise. Furthermore, based on the noise type identifier corresponding to the speech feature set, the matching convolutional kernel configuration is called to perform a two-dimensional convolution operation on each feature matrix in the speech feature set. During the convolution operation, the local correlation information of the features is extracted by summing the element-wise products between the convolutional kernel and the local regions of the feature matrix, thus obtaining the initial convolutional features. Normalization is performed on the initial convolutional features to stabilize the training process and improve the model's generalization ability. Batch normalization or layer normalization can be selected for normalization. Then, a non-linear activation function is applied to the normalized features for feature transformation. The non-linear activation function can be ReLU, Swish, GELU, or Mish. After the above processing, local accurate features are obtained, which can highlight the local details of the speech signal and suppress the interference of corresponding types of noise.

[0024] For example, when focusing on key speech information from precise local features, the features are dimensionally reshaped and converted into a sequence format suitable for the input of a multi-head self-attention mechanism. The reshaped feature sequence is represented by the sequence length and feature dimension, where the sequence length corresponds to the temporal dimension of the speech, and the feature dimension corresponds to the feature dimension at each time step. The multi-head self-attention mechanism sets up multiple attention heads in parallel to perform attention calculations on the feature sequence from different subspaces. Each attention head independently learns the dependencies within the feature sequence. Optionally, the reshaped feature sequence is mapped to a query matrix, a key matrix, and a value matrix, respectively. The original attention weight matrix is ​​obtained by performing a dot product operation on the transpose of the query matrix and the key matrix and then scaling it. The original attention weight matrix is ​​then normalized using the softmax function to obtain a normalized attention weight matrix, which reflects the dependence of each time step in the feature sequence on other time steps. The normalized attention weight matrix is ​​then multiplied with the value matrix to obtain the output features of a single attention head. The output features of all attention heads are concatenated and fused using a linear transformation to obtain the output features of the multi-head self-attention mechanism. To preserve the original information of the local precise features and alleviate the gradient vanishing problem in deep networks, the output features of the multi-head self-attention mechanism are residually connected with the reshaped local precise features, and normalization can be selectively performed to finally obtain the key focusing features. These features, through the allocation of attention weights, enhance key speech information and suppress noise interference.

[0025] S104. Based on the noise type identifier, calculate the fusion weight through dynamic weight, and generate the fusion feature by weighted summation of the local precise features and key focused features according to the fusion weight.

[0026] In a schematic representation, the dynamic weight calculation module takes noise type identifiers as input and outputs fusion weights for fusing local precision features and key focus features. This module can be implemented using a pre-defined lookup table, a lightweight neural network, or a parameterized mathematical function. Specifically, using a pre-defined lookup table, the optimal fusion weights for different noise type identifiers are pre-determined through experiments, and the mapping relationship between noise type identifiers and fusion weights is stored in the lookup table, allowing direct indexing of the corresponding fusion weights via the noise type identifier. Using a lightweight neural network, the encoded form of the noise type identifier is used as input, and the fusion weights are output through calculations by a small number of fully connected layers. The network is pre-trained based on training data to learn the optimal mapping relationship. Using a parameterized mathematical function, the fusion weights are calculated using a pre-defined mathematical formula combined with the encoded value of the noise type identifier. The parameters of the mathematical formula need to be optimized and determined during the training process. The fusion weights consist of two components: the weight coefficients corresponding to the local precision features. Weight coefficients corresponding to key focus features Both components must satisfy the non-negativity constraint, and whether or not this constraint is satisfied can be set according to actual needs. The normalization constraint. For example, the weighted summation process of locally precise features and key focused features follows... ,in, Indicates the generated fusion features, Represents precise local features. Indicates key focusing features. Weight coefficients representing local precise features. The weight coefficients represent the key features.

[0027] S105. Calculate channel-level attention weights for the fused features, and filter feature data based on the calculated channel-level attention weights to obtain lightweight optimized features.

[0028] Specifically, global statistical information extraction is performed on the fused features. For each channel of the fused features, the two-dimensional feature map of that channel is compressed into a single scalar value through global average pooling, global max pooling, or a combination of global statistics. This scalar value represents the global information strength of the corresponding channel. The scalar values ​​of all channels are used to construct a channel statistical vector, which is then input into the attention calculation unit. The attention calculation unit can use fully connected networks, gating mechanisms, or sigmoid activation function mapping to perform nonlinear transformations on the channel statistical vector, outputting the attention weight value corresponding to each channel. The attention weight value ranges from 0 to 1, with a larger weight value indicating that the information of the corresponding channel is more important to the speech recognition task.

[0029] Feature data filtering is performed based on the calculated channel-level attention weights. The filtering process follows pre-defined filtering rules, which can be based on a preset pruning ratio, a weight threshold, or an adaptive threshold. Specifically, the preset pruning ratio rule sorts all channels from largest to smallest attention weight value according to a set retention threshold, retaining the top-ranked channels that meet the threshold as core information channels, while discarding redundant channels with lower weights. The weight threshold rule retains channels with attention weight values ​​greater than a set threshold, discarding channels with weight values ​​less than or equal to the threshold. The adaptive threshold rule dynamically determines the threshold based on the statistical characteristics of channel attention weights, such as mean and median, before filtering channels. During feature data filtering, only feature data corresponding to core information channels is retained; feature data from redundant channels is completely discarded. The filtered feature data constitutes lightweight optimized features.

[0030] S106. Input the lightweight optimized features into the pre-trained classification decoder for temporal classification decoding to obtain the speech recognition text result.

[0031] Furthermore, the pre-trained classification decoder is a sequence generation model adapted to the temporal features of speech. Lightweight fully connected networks, recurrent neural networks (RNNs), long short-term memory networks (LSTMs), gated recurrent units (GRUs), or lightweight variants of the Transformer decoder can be used. The pre-training process of the classification decoder is performed in conjunction with the noise classifier, completing end-to-end training based on the same training dataset. The training objective is to minimize the sequence loss function for speech recognition, which can be a connection-time classification (CTC) loss function, a transducer loss function, or an attention mechanism loss function. The lightweight optimized features are input into the classification decoder in chronological order. The decoder processes the features at each time step through forward propagation, outputting the probability distribution of all candidate characters corresponding to that time step. The candidate character set contains all the characters required for the speech recognition task.

[0032] Optionally, the temporal classification decoding process addresses the misalignment between the lengths of the speech feature sequence and the text character sequence. The decoding algorithm can be a greedy decoding algorithm, a beam search algorithm, an N-best search algorithm, or a decoding algorithm with language model constraints. During decoding, based on the character probability distribution at each time step output by the decoder, the optimal character sequence combination is selected according to the rules of the decoding algorithm to generate the initial text sequence. Post-processing is then performed on the initial text sequence to remove whitespace, duplicate characters, and invalid characters. The sequence is then regularized according to the grammatical rules and semantic logic of the language model, ultimately yielding a speech recognition text result that conforms to natural language expression habits.

[0033] In the aforementioned neural network-based smart classroom speech recognition method, a speech feature set is obtained by acquiring noisy speech data from the classroom environment and extracting acoustic features. A pre-trained noise classifier is used to identify the dominant classroom noise type to obtain a noise type identifier. Based on the noise type identifier, the corresponding convolution kernel configuration is matched to perform convolution operations and nonlinear activation processing on the speech feature set to obtain local accurate features. At the same time, a multi-head self-attention mechanism is used to focus key speech information on the local accurate features to obtain key focused features. Dynamic fusion weights are calculated based on the noise type identifier, and the two types of features are weighted and summed to generate fusion features. Channel-level attention weights are used to calculate and filter the fusion features to obtain lightweight optimized features. The lightweight optimized features are input into a pre-trained classifier decoder for temporal classification and decoding. This achieves adaptive adaptation to complex classroom noise environments, effectively improves the accuracy of speech recognition, balances high performance and lightweight design, and enhances the model's generalization ability to different classroom noise scenarios.

[0034] In one embodiment, such as Figure 2 As shown, the noise classifier consists of a first convolutional layer, a pooling layer, a second convolutional layer, and a fully connected output layer connected in sequence. The dominant classroom noise type in the speech feature set is identified by a pre-trained noise classifier, resulting in noise type identifiers, including: S201. Based on the input layer dimension of the noise classifier, the acoustic features in the speech feature set are dimensionally normalized to obtain standardized input features.

[0035] In a schematic representation, the input layer dimension of the noise classifier is a preset fixed dimension, determined based on the typical dimensional range of acoustic features and the computational capabilities of edge devices. Acoustic features of different dimensions are uniformly converted into a feature form that perfectly matches the input layer dimension. The processing method can be selected based on the difference between the acoustic features and the input layer dimension. For example, if the acoustic feature dimension is smaller than the input layer dimension, values ​​are added to a specified dimension of the feature using padding. These padding values ​​can be zero, the statistical mean of the feature, or randomly initialized values. If the acoustic feature dimension is larger than the input layer dimension, redundant feature data is removed using pruning or downsampling. Pruning should retain regions with high information density, while downsampling should be achieved through interpolation or convolutional dimensionality reduction. If the acoustic feature dimension is exactly the same as the input layer dimension, the original feature form is directly retained. After dimension normalization, all acoustic features are converted into a feature tensor of a uniform dimension. This feature tensor is the standardized input feature, whose dimension perfectly matches the input layer dimension of the noise classifier.

[0036] S202. Local feature extraction is performed on the standardized input features through the first and second convolutional layers, and downsampling is performed through the pooling layer to obtain the feature map.

[0037] Furthermore, the standardized input features are fed into the first convolutional layer. This layer performs element-wise multiplication and summation operations on the standardized input features with a pre-defined kernel, extracting basic local patterns from the features. The kernel parameters of the first convolutional layer are designed based on the local correlations of the classroom acoustic features. During convolution, padding can be selected as needed to maintain the dimensionality of the output features matching that of the input features. The primary local features output from the first convolutional layer are then fed into a pooling layer. This layer downsamples the feature values ​​within the local region by aggregating them. Aggregation methods can include max pooling, average pooling, or random pooling to reduce the feature dimensionality and subsequent computation while preserving key information and improving the model's generalization and anti-overfitting capabilities.

[0038] The features downsampled by the pooling layer are fed into the second convolutional layer. The kernel parameters of the second convolutional layer are designed based on the primary local features extracted by the first convolutional layer. Through deeper convolution operations, more complex high-order local correlation information in the features is further mined, achieving deep extraction of local features. The convolution operation logic of the second convolutional layer is consistent with that of the first convolutional layer, and padding can also be selected according to the feature dimension requirements. Through the basic extraction of the first convolutional layer, the downsampling of the pooling layer, and the deep extraction of the second convolutional layer, the standardized input features are transformed into a feature tensor containing multi-level local feature information. This feature tensor is the feature map, and its dimension is determined by the convolution kernel parameters, stride, and padding method of each layer, which can comprehensively represent the noise-related local pattern information in the input features.

[0039] S203. Expand the feature map into feature vectors, and apply the Softmax activation function to the feature vectors through a fully connected output layer to obtain the classroom noise probability distribution vector.

[0040] Optionally, the unfolding process arranges all elements of the feature map into a continuous one-dimensional data sequence according to a preset dimensional order, forming a feature vector. The length of the feature vector is equal to the product of the dimensions of each dimension of the feature map, and it contains all the local feature information in the feature map.

[0041] The expanded feature vector is fed into the fully connected output layer. The fully connected output layer maps the high-dimensional feature vector to a low-dimensional space corresponding to the preset number of noise types through matrix multiplication of the weight matrix and the feature vector, obtaining the original output vector. Each element in the original output vector corresponds to the original score for one preset noise type. The Softmax activation function is applied to the original output vector, converting the original scores into probability values ​​ranging from 0 to 1, with the sum of all probability values ​​being 1. The calculation of the Softmax activation function follows... ,in, This represents the probability value corresponding to the i-th noise type. This represents the original score corresponding to the i-th element in the original output vector. This indicates the total number of preset classroom noise types. Represents the natural constant. This represents the sum of all elements in the original output vector after exponentiation. All elements calculated using this formula... Together, they form the probability distribution vector of classroom noise, which fully reflects the probability of various types of classroom noise corresponding to the input features.

[0042] S204. Select the noise type corresponding to the maximum probability value in the classroom noise probability distribution vector to obtain the noise type identifier.

[0043] Specifically, each element in the classroom noise probability distribution vector corresponds one-to-one with a preset classroom noise type. The probability value directly represents the confidence level of the input feature belonging to the corresponding noise type; the higher the probability value, the higher the confidence level. The dominant noise type is determined by the element with the largest probability value in the probability distribution vector. By traversing all elements in the probability distribution vector, the element with the largest probability value is selected, and its index position in the vector is determined. Based on the preset mapping relationship between the index and the noise type, the corresponding noise type is found through this index position. This noise type is the dominant classroom noise type corresponding to the current input feature. A noise type identifier is generated based on the determined dominant classroom noise type. This identifier uses the same encoding format as during classifier training to ensure that subsequent modules can accurately identify and call the corresponding processing strategies. The encoding format must be unique and identifiable to clearly distinguish between different dominant noise types.

[0044] In one embodiment, such as Figure 3 As shown, by using the convolution kernel configuration corresponding to the noise type identifier, convolution operations and nonlinear activation processing are performed on the speech feature set to obtain local precise features, including: S301. If the noise type is identified as conversational noise, the convolution kernel configuration is determined to be the first configuration. If the noise type is identified as table and chair movement noise or multimedia noise, the convolution kernel configuration is determined to be the second configuration. The first configuration uses a pre-configured fine-grained feature convolution kernel and a unit stride. The second configuration uses a pre-configured large receptive field feature convolution kernel and a first step size.

[0045] Indicatively, the mapping relationship between noise type identification and convolution kernel configuration is established through preset rules based on the interference mechanisms of various noises on speech features. Since the spectra of conversation and target speech highly overlap, interference mainly manifests as local spectral confusion. Therefore, the fine-grained feature convolution kernel design in the first configuration aims to accurately capture subtle local spectral patterns of the speech signal. Through dense sampling with small-sized convolution kernels, it achieves fine differentiation of overlapping spectral regions. The unit stride setting maximizes the preservation of temporal details of speech features, avoiding the loss of detail information due to excessively large strides.

[0046] The sounds of moving tables and chairs are characterized by suddenness and high instantaneous energy, while multimedia noise exhibits broadband and continuous spectrum characteristics. The interference of both types of noise manifests as distortion or overlay of the global spectrum. Therefore, the second configuration's large receptive field feature convolution kernel, by increasing the kernel size, expands the feature perception range of a single convolution operation, effectively capturing the global spectrum correlation information of the speech signal and suppressing interference from broadband or sudden noise. The first step's long setting, while ensuring the effectiveness of feature capture, appropriately reduces the dimensionality of the feature map, balancing computational load and feature representation capability. The number of convolution kernels in both configurations is optimized based on the computational resource constraints of the edge device, strictly controlling the number of parameters and computational complexity while meeting feature extraction requirements.

[0047] S302. Based on the convolution kernel configuration, perform two-dimensional convolution operation on the speech feature set to obtain the initial convolution feature map.

[0048] Specifically, each feature matrix in the speech feature set is used as input to the convolution operation in the form of a two-dimensional tensor. The dimensions of this two-dimensional tensor correspond to the frequency and time dimensions of the features. Based on the determined convolution kernel configuration, the corresponding convolution kernel parameters, including kernel size, number of kernels, stride, and padding, are invoked to perform the two-dimensional convolution operation. During the convolution operation, the convolution kernel slides along the frequency and time dimensions of the input feature matrix at a preset stride, performing product summation on the feature elements within each sliding window to obtain the feature response value corresponding to that window.

[0049] The choice of padding method depends on the convolution kernel size and the dimension of the input feature matrix. If it is necessary to keep the dimension of the output feature map consistent with the dimension of the input feature matrix, a zero-padding or edge-padding strategy is used; if it is necessary to appropriately reduce the feature dimension to reduce computation, a no-padding strategy can be used. Through window-by-window convolution operations, the input feature matrix is ​​converted into a two-dimensional feature tensor containing local correlation information. This feature tensor is the initial convolution feature map, and each element corresponds to the comprehensive response of a specific local region in the input features, which can initially suppress the interference of corresponding types of noise and highlight the local features of the speech signal.

[0050] S303. Perform batch normalization on the initial convolutional feature map, and apply the ReLU nonlinear activation function to the batch normalized initial convolutional feature map to obtain local accurate features.

[0051] Furthermore, the mean and variance of each feature channel in the initial convolutional feature map of the current batch are calculated. Based on these mean and variance, all feature values ​​within each channel are standardized to adjust them to a similar distribution range. A linear transformation is then applied to the standardized feature values ​​using learnable scaling and offset coefficients to restore the expressive power of the features. The batch normalization calculation follows... , ,in, This represents the original feature value of a feature channel in the initial convolutional feature map. This represents the mean of all feature values ​​for this feature channel in the current batch. This represents the variance of all eigenvalues ​​in this feature channel for the current batch. This represents an extremely small positive number set to avoid a denominator of zero. Represents the standardized eigenvalues. This represents the learnable scaling factor. This represents the learnable offset coefficient. This represents the feature values ​​after batch normalization.

[0052] Applying the ReLU nonlinear activation function to the batch-normalized feature maps, this function thresholds the input feature values, setting feature values ​​less than zero to zero and leaving feature values ​​greater than zero unchanged. ,in, This represents the feature values ​​after batch normalization. This represents the activated feature value. The application of the ReLU function introduces nonlinear transformation capabilities, which can simulate the nonlinear characteristics of speech signals, while effectively suppressing the gradient vanishing problem, enhancing feature responses useful for speech recognition, and weakening invalid feature responses corresponding to noise interference. After batch normalization and ReLU activation processing, the feature map becomes a locally accurate feature with both local detail capture and noise suppression capabilities, providing high-quality feature input for subsequent key information focusing processing.

[0053] In one embodiment, based on the noise type identifier, a fusion weight is calculated using dynamic weighting, and the local precise features and key focused features are weighted and summed according to the fusion weight to generate fused features, including: S11. Perform one-hot encoding on the noise type identifier to obtain the encoding vector, and input the encoding vector into the pre-trained dynamic weight calculation model to obtain the fusion weight coefficients; the fusion weight coefficients include local precise feature weights and key focused feature weights.

[0054] In this schematic representation, noise type identifiers are discrete symbols and cannot be directly used as input to the neural network model. Based on the preset total number of classroom noise types, a binary vector with the same dimension is constructed. Only the dimension corresponding to the current noise type identifier has a value of 1, while all other dimensions have a value of 0. This method achieves a unique numerical representation of different noise types and avoids the discrete identifier's value misleading the model's calculations. The dimension of the encoded vector strictly matches the input layer dimension of the dynamic weight calculation model, ensuring that data can be correctly input into the model for computation.

[0055] The dynamic weight calculation model is a lightweight parameterized model. Its structure is designed for lightweight operation and low latency, employing a simplified structure of 2-3 layers of fully connected networks or convolutional neural networks. Non-linear activation functions such as ReLU or Swish can be introduced into the network to improve the model's fitting ability. The pre-training process is based on a labeled dataset, which includes encoded vectors corresponding to different noise types, corresponding local precise features and key focus features, and the optimal fusion weights determined through experiments. The training objective is to minimize the difference between the model's predicted fusion weights and the optimal fusion weights in the labeled data. The loss function can be either the mean squared error loss function or the smoothed L1 loss function. The encoded vectors are input into the pre-trained model, and the model calculates the fusion weights through forward propagation. The model outputs two consecutive values ​​as fusion weight coefficients: one value is the local precise feature weight, representing the contribution ratio of local precise features in the fusion process; the other value is the key focus feature weight, representing the contribution ratio of key focus features in the fusion process.

[0056] S12. Adjust the fusion weight coefficients according to the weight constraint range corresponding to the noise type identifier to obtain the adjusted local accurate feature weights and key focus feature weights. Then, perform weighted fusion of the local accurate features and key focus features based on the adjusted local accurate feature weights and key focus feature weights to obtain the fused features.

[0057] Optionally, the weight constraint range is a pre-defined reasonable interval for each noise type identifier. It is set based on the effective contribution boundaries of the two types of features under different noise environments, determined through extensive experimental statistical analysis. The aim is to prevent the fusion weight coefficients output by the model from exhibiting extreme values ​​due to abnormal data interference, which could lead to excessive amplification or suppression of the contribution of a certain type of feature, thus affecting the fusion effect. The constraint range is represented in interval form, with each noise type identifier corresponding to two intervals: one for the locally accurate feature weights and the other for the key focused feature weights. The upper and lower limits of the intervals are determined through recognition accuracy optimization on the validation set.

[0058] For example, it is determined whether the local precision feature weights and key focus feature weights are within their corresponding constraint intervals. If the value of a weight is within the corresponding constraint interval, its original value remains unchanged; if the value of a weight exceeds the upper limit of the corresponding constraint interval, the weight is adjusted to the upper limit value of the constraint interval; if the value of a weight is below the lower limit of the corresponding constraint interval, the weight is adjusted to the lower limit value of the constraint interval. After the above adjustments, the adjusted local precision feature weights and key focus feature weights, which are reasonable and effective, are obtained.

[0059] The weighted fusion process is achieved through element-wise weighted summation, fusing local precise features and key focused features according to adjusted weights. The fusion formula is as follows: ,in, Indicates the generated fusion features, This represents the adjusted local precision feature weights. Represents precise local features. This represents the adjusted key focus feature weights. This represents the key focusing feature. During the operation, the corresponding elements of the local precise feature and the key focusing feature are multiplied by their respective adjusted weights, and then all the product results are added element by element. The final fused feature combines the detail capture capability of the local precise feature with the global focusing capability of the key focusing feature, and the fusion ratio is adapted to the current dominant noise type, which can provide high-quality input for subsequent feature lightweighting and recognition decoding.

[0060] In one embodiment, channel-level attention weights are calculated on the fused features, and feature data is filtered based on the calculated channel-level attention weights to obtain lightweight optimized features, including: S21. By using global average pooling, the two-dimensional feature maps of each channel in the fused features are compressed into scalars to obtain the channel-level statistical vectors corresponding to each channel.

[0061] Indicatively, the fused features exist in the form of multi-channel two-dimensional feature maps. Each channel's two-dimensional feature map corresponds to the expression of speech features in a specific dimension, and the proportion of speech information to noise interference information carried by different channels varies. The core function of global average pooling is to aggregate information globally from the two-dimensional feature maps of each channel, compressing the high-dimensional feature maps into a single scalar, which represents the overall information strength of the corresponding channel. Specifically, for each channel in the fused features, all elements of the two-dimensional feature map of that channel are traversed, and the arithmetic mean of all elements is calculated. This average value is the scalar value corresponding to that channel. The calculation of global average pooling follows... ,in, Indicates the first The scalar value corresponding to each channel. This represents the height of the two-dimensional feature map, corresponding to the frequency dimension of the speech features. This represents the width of the two-dimensional feature map, corresponding to the time dimension of the speech features. Indicating the fusion feature of the first The first channel, the first line, number The characteristic element values ​​of the column, This represents the total number of elements in a single channel's two-dimensional feature map. The double summation operation is used to accumulate all elements in that channel.

[0062] S22. Calculate the importance score of each channel based on the channel-level statistical vector to obtain the attention weight vector of each channel.

[0063] Optionally, the noise type identifier channel-level statistical vector contains the global information strength of each channel. The core logic for calculating the importance score based on this vector is to learn the contribution of different channels to the speech recognition task through an attention mechanism. The attention calculation unit adopts a lightweight network structure design to adapt to the computing resource constraints of edge devices. A simple network consisting of 1-2 fully connected layers can be selected. Batch Normalization layers can be introduced into the network to stabilize the training process. The output layer uses a Sigmoid activation function to map the score results to the 0-1 interval, ensuring that the score has a probabilistic representation of importance.

[0064] For example, channel-level statistical vectors are input into a predefined attention calculation unit. A fully connected layer performs linear transformations and non-linear activations on the statistical vectors to achieve a non-linear mapping of channel importance. The output is a one-dimensional vector with the same dimension as the statistical vectors. Each element in this vector corresponds to the importance score of the corresponding channel in the fused features, i.e., the attention weight. The calculation of attention weights follows... ,in, Indicates the first Attention weights for each channel, This represents the Sigmoid activation function. and These represent the weight matrices of the two fully connected layers in the attention computation unit. and These represent the bias vectors of the two fully connected layers, respectively. This indicates a batch normalization operation. Indicates the first The attention weights of all channels constitute an attention weight vector, which fully reflects the importance of each channel to the speech recognition task.

[0065] S23. According to the preset pruning ratio, sort all channel weights in the attention weight vector in descending order, and select the channel with the highest sorting ratio as the core channel to obtain the binary channel mask; the binary channel mask includes the core channel and the redundant channel.

[0066] Specifically, the preset pruning ratio strikes a balance between the computational resource constraints of edge devices and the accuracy of speech recognition. Through extensive experimental verification, the pruning ratio was determined to ensure that the retained core channels can carry sufficient key speech information while minimizing redundant data. The pruning ratio is expressed as a percentage or decimal, and its value is dynamically adjusted based on the device performance in the actual application scenario.

[0067] Sort all elements in the attention weight vector in descending order of their values ​​to obtain the sorted weight sequence and the corresponding original channel indices. Calculate the number of core channels to be retained based on a preset pruning ratio; the number of core channels equals the product of the total number of channels in the fused features and the pruning ratio. Select the channels with the highest number of core channels in the sorted sequence and mark them as core channels; mark the remaining channels as redundant channels. Generate a binary channel mask based on the marking results. This mask is a one-dimensional binary vector with the same number of fused feature channels, where the positions corresponding to core channels are set to 1, and the positions corresponding to redundant channels are set to 0. This mask allows for precise filtering of core channel feature data.

[0068] S24. Multiply the fused features with the binary channel mask channel by channel to obtain lightweight optimized features.

[0069] Furthermore, for each channel of the fused features, an element-wise multiplication operation is performed on all elements in the two-dimensional feature map of that channel and the corresponding binary value in the binary channel mask. If the channel is a core channel, the mask value is 1, and the feature data of that channel remains unchanged after the operation; if the channel is a redundant channel, the mask value is 0, and all feature elements of that channel become 0 after the operation, and their corresponding data will no longer participate in subsequent calculations. ,in, Indicating the first lightweight optimization feature The first channel, the first line, number The characteristic element values ​​of the column, Indicating the fusion feature of the first The first channel, the first line, number The characteristic element values ​​of the column, Represents the first in the binary channel mask The binary value (0 or 1) of each position. After channel-by-channel multiplication, the invalid data of redundant channels is masked, and only the key feature data of the core channels are retained, forming a lightweight optimized feature. This feature has significantly reduced dimensionality and data volume, while fully retaining the core information required for speech recognition, and can efficiently adapt to the inference computing needs of edge devices.

[0070] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0071] Based on the same inventive concept, this application also provides a neural network-based smart classroom speech recognition system for implementing the aforementioned neural network-based smart classroom speech recognition method. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more neural network-based smart classroom speech recognition system embodiments provided below can be found in the limitations of the neural network-based smart classroom speech recognition method described above, and will not be repeated here.

[0072] In one exemplary embodiment, such as Figure 4 As shown, a smart classroom speech recognition system based on neural networks is provided, including: Classroom audio module 401 is used to acquire noisy speech data of the classroom environment and extract acoustic features from the noisy speech data to obtain a speech feature set. The noise type module 402 is used to identify the dominant classroom noise type in the speech feature set through a pre-trained noise classifier and obtain a noise type identifier. The feature module 403 is used to perform convolution operations and nonlinear activation processing on the speech feature set by identifying the convolution kernel configuration corresponding to the noise type to obtain local precise features, and to perform key speech information focusing processing on the local precise features through a multi-head self-attention mechanism to obtain key focused features. The feature fusion module 404 is used to calculate the fusion weight based on the noise type identifier through dynamic weight, and to generate fused features by weighted summation of local precise features and key focused features according to the fusion weight; The feature filtering module 405 is used to calculate channel-level attention weights for the fused features and filter the feature data of the fused features based on the calculated channel-level attention weights to obtain lightweight optimized features. The text decoding module 406 is used to input lightweight optimized features into a pre-trained classifier decoder for temporal classification decoding to obtain speech recognition text results.

[0073] In one embodiment, the noise type module 402 is further configured to: Based on the input layer dimension of the noise classifier, the acoustic features in the speech feature set are dimensionally normalized to obtain standardized input features. Local features are extracted from the standardized input features through the first and second convolutional layers, and then downsampled through pooling layers to obtain the feature map. The feature map is unfolded into a feature vector, and the softmax activation function is applied to the feature vector through a fully connected output layer to obtain the classroom noise probability distribution vector. Select the noise type corresponding to the maximum probability value in the classroom noise probability distribution vector to obtain the noise type identifier.

[0074] In one embodiment, the feature module 403 is further configured to: If the noise type is identified as conversation, the convolution kernel configuration is set to the first configuration; if the noise type is identified as table and chair movement noise or multimedia noise, the convolution kernel configuration is set to the second configuration. The first configuration uses a pre-configured fine-grained feature convolution kernel with a unit stride; the second configuration uses a pre-configured large receptive field feature convolution kernel with a first step size. Based on the convolution kernel configuration, a two-dimensional convolution operation is performed on the speech feature set to obtain the initial convolution feature map; The initial convolutional feature maps are batch normalized, and the ReLU nonlinear activation function is applied to the batch normalized initial convolutional feature maps to obtain local precise features.

[0075] In one embodiment, the feature fusion module 404 is further configured to: The noise type identifier is one-hot encoded to obtain an encoded vector, and the encoded vector is input into a pre-trained dynamic weight calculation model to obtain fusion weight coefficients; the fusion weight coefficients include local precise feature weights and key focused feature weights; Based on the weight constraint range corresponding to the noise type identifier, the fusion weight coefficients are adjusted to obtain the adjusted local precise feature weights and key focus feature weights. Then, the local precise features and key focus features are weighted and fused based on the adjusted local precise feature weights and key focus feature weights to obtain the fused features.

[0076] In one embodiment, the feature filtering module 405 is further configured to: Global average pooling is used to compress the two-dimensional feature maps of each channel in the fused features into scalars, resulting in channel-level statistical vectors for each channel. The importance score of each channel is calculated based on the channel-level statistical vector, and the attention weight vector of each channel is obtained. According to the preset pruning ratio, all channel weights in the attention weight vector are sorted in descending order, and the channels with the highest sorting ratio are marked as core channels to obtain binary channel masks; the binary channel mask includes core channels and redundant channels. The fused features are multiplied channel by channel with the binary channel mask to obtain lightweight optimized features.

[0077] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0078] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0079] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0080] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A speech recognition method for smart classrooms based on neural networks, characterized in that, The method includes: Acquire noisy speech data from a classroom environment, and extract acoustic features from the noisy speech data to obtain a speech feature set; The dominant classroom noise type in the speech feature set is identified by a pre-trained noise classifier, and a noise type identifier is obtained. By configuring the convolution kernels corresponding to the noise type identifier, convolution operations and nonlinear activation processing are performed on the speech feature set to obtain local precise features. Then, key speech information is focused on the local precise features through a multi-head self-attention mechanism to obtain key focused features. Based on the noise type identifier, a fusion weight is calculated using dynamic weighting, and the local precise features and the key focused features are weighted and summed according to the fusion weight to generate fusion features; Channel-level attention weights are calculated for the fused features, and feature data is filtered based on the calculated channel-level attention weights to obtain lightweight optimized features; The lightweight optimized features are input into a pre-trained classification decoder for temporal classification decoding to obtain the speech recognition text result.

2. The method according to claim 1, characterized in that: The noise classifier comprises a first convolutional layer, a pooling layer, a second convolutional layer, and a fully connected output layer connected in sequence. The step of identifying the dominant classroom noise type in the speech feature set using a pre-trained noise classifier to obtain a noise type identifier includes: Based on the input layer dimension of the noise classifier, the acoustic features in the speech feature set are subjected to dimension normalization to obtain standardized input features. The standardized input features are extracted locally through the first convolutional layer and the second convolutional layer, and then downsampled through the pooling layer to obtain a feature map. The feature map is expanded into a feature vector, and the Softmax activation function is applied to the feature vector through the fully connected output layer to obtain the classroom noise probability distribution vector. Select the noise type corresponding to the maximum probability value in the classroom noise probability distribution vector to obtain the noise type identifier.

3. The method according to claim 1, characterized in that, The step of performing convolution operations and nonlinear activation processing on the speech feature set using the convolution kernel configuration corresponding to the noise type identifier to obtain local precise features includes: If the noise type is identified as conversational noise, the convolution kernel configuration is determined to be the first configuration; if the noise type is identified as table and chair movement noise or multimedia noise, the convolution kernel configuration is determined to be the second configuration. The first configuration uses a pre-configured fine-grained feature convolution kernel and a unit stride; the second configuration uses a pre-configured large receptive field feature convolution kernel and a first step size. Based on the convolution kernel configuration, a two-dimensional convolution operation is performed on the speech feature set to obtain an initial convolution feature map; The initial convolutional feature map is batch normalized, and the ReLU nonlinear activation function is applied to the batch normalized initial convolutional feature map to obtain the local precise features.

4. The method according to claim 1, characterized in that, The step of calculating fusion weights based on the noise type identifier using dynamic weights, and then weighting and summing the local precise features and the key focused features according to the fusion weights to generate fusion features includes: The noise type identifier is one-hot encoded to obtain an encoded vector, and the encoded vector is input into a pre-trained dynamic weight calculation model to obtain fusion weight coefficients; the fusion weight coefficients include local precise feature weights and key focused feature weights; Based on the weight constraint range corresponding to the noise type identifier, the fusion weight coefficients are adjusted to obtain the adjusted local precise feature weights and key focus feature weights. Then, the local precise features and key focus features are weighted and fused based on the adjusted local precise feature weights and key focus feature weights to obtain the fused features.

5. The method according to claim 1, characterized in that, The process of calculating channel-level attention weights for the fused features and then filtering feature data based on the calculated channel-level attention weights to obtain lightweight optimized features includes: Global average pooling is used to compress the two-dimensional feature maps of each channel in the fused features into scalars, resulting in channel-level statistical vectors for each channel. The importance score of each channel is calculated based on the channel-level statistical vector, and the attention weight vector of each channel is obtained. According to the preset pruning ratio, all channel weights in the attention weight vector are sorted in descending order, and the channels with the highest sorting ratio are marked as core channels to obtain a binary channel mask; the binary channel mask includes core channels and redundant channels. The fusion feature is multiplied channel by channel by channel of the binary channel mask to obtain the lightweight optimized feature.

6. A smart classroom speech recognition system based on neural networks, characterized in that, The system includes: The classroom audio module is used to acquire noisy speech data of the classroom environment and extract acoustic features from the noisy speech data to obtain a speech feature set. The noise type module is used to identify the dominant classroom noise type in the speech feature set through a pre-trained noise classifier, and obtain a noise type identifier; The feature module is used to perform convolution operations and nonlinear activation processing on the speech feature set according to the convolution kernel configuration corresponding to the noise type identifier to obtain local precise features, and to perform key speech information focusing processing on the local precise features through a multi-head self-attention mechanism to obtain key focused features. The feature fusion module is used to calculate the fusion weight based on the noise type identifier using dynamic weights, and to perform a weighted summation of the local precise features and the key focused features according to the fusion weights to generate fused features; The feature filtering module is used to calculate channel-level attention weights for the fused features and filter the feature data of the fused features based on the calculated channel-level attention weights to obtain lightweight optimized features. The text decoding module is used to input the lightweight optimized features into a pre-trained classification decoder for temporal classification decoding to obtain the speech recognition text result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images