Lightweight abnormal acoustic event detection method based on adaptive width self-attention mechanism

Abstract

This invention discloses a lightweight anomalous sound event detection method based on an adaptive width self-attention mechanism. The method first performs signal processing on labeled audio to obtain its time-frequency feature representation. Second, the labeled feature representation (usually a vector or matrix) is fed into an adaptive width self-attention mechanism model. Then, the adaptive width self-attention mechanism model has a predefined loss function and randomly initialized attention weights. The loss value is calculated based on the label using the adaptive self-attention algorithm. Next, the adaptive attention weights are updated using backpropagation. This process is iteratively updated on the three input weights until the loss function reaches its minimum or ideal state. Finally, the weight parameters are saved using a lightweight method. These weight parameters are then used as the model to predict an unlabeled audio segment, enabling fast and accurate detection of anomalous sound events.

Description

Technical Field

[0001] This invention relates to a method for detecting abnormal overlapping sound events using a self-attention mechanism, specifically a lightweight abnormal sound event detection method based on an adaptive width self-attention mechanism. Background Technology

[0002] Abnormal acoustic event detection technology belongs to the research field of acoustic event recognition and has important application value in smart city life such as smart homes, urban road anomaly detection, and fault detection.

[0003] Sound event detection tasks mainly consist of signal processing and machine learning models. Commonly used signal processing methods include adding noise, Fast Fourier Transform (FFT), and Mel Frequency Cepstral Coefficient (MFCC) feature extraction.

[0004] Existing methods for live event detection utilize neural networks to build learning models, including models using Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), which are trained individually or jointly to achieve sound event detection. However, these models often suffer from poor real-time performance, difficulty in optimizing model training, and slow prediction speed. This is ultimately due to the inherent nature of the network structure. While CNNs solve the problem of handling large amounts of data, they cannot learn the event features of sound events, and the large number of parameters easily leads to overfitting. RNNs, although capable of carrying temporal information during information extraction, introduce long-term dependencies and the vanishing gradient phenomenon. Subsequent developments have addressed some of these issues, but because RNNs are inherently sequential structures, parallel computation remains difficult, resulting in relatively slow computation speeds. Compared to CNNs and RNNs, self-attention mechanisms have lower complexity and fewer parameters, thus requiring less computational power and conserving computer resources. They also address the inability of RNNs to be parallelized, offering a significant advantage in computational speed. Furthermore, they can better focus on information in longer sequences, although sometimes they may focus on too much information. In 2020, Dr. Helen's team at Queen Mary University of London compared the recognition performance of the same machine learning model before and after using a controlled-width self-attention mechanism. Their results showed that the model using the controlled-width self-attention mechanism improved its F-score by 8.45% and reduced the error rate by 0.15, demonstrating that controlling the width of attention is effective in improving sound event detection. Moreover, with the development of deep neural networks, network structures are becoming increasingly sophisticated, and the storage space required for models is constantly increasing, making most detection models remain theoretical and difficult to deploy on mobile devices and other portable devices. Summary of the Invention

[0005] To address the problems of large prediction models, slow prediction speed, excessive reliance on computational resources, and difficulty in real-time prediction in current Sound Event Detection (SED) methods, this invention proposes a lightweight abnormal sound event detection method based on an adaptive width self-attention mechanism. This method can classify and detect abnormal sound events contained in an audio segment. Under the same signal processing conditions, it exhibits superior recognition performance compared to CRNN-based methods, solving the problems of slow computation speed and inability to perform parallel operations in RNN-based methods. Furthermore, by utilizing a lightweight approach, the model size is compressed with minimal loss in recognition performance, allowing the model to be deployed on mobile devices or other portable devices.

[0006] The technical solution to achieve the objective of this invention is:

[0007] A lightweight anomalous acoustic event detection method based on an adaptive width self-attention mechanism includes the following steps:

[0008] (1) Construct a synthetic audio dataset and label and classify each audio file containing multiple anomalous sound events;

[0009] (2) Preprocess and extract features from the dataset, and feed it into the built adaptive width self-attention mechanism model for iterative training of the network until the model reaches its optimal state;

[0010] (3) The model is compressed using a lightweight method to obtain a lightweight detection model with an adaptive width self-attention mechanism;

[0011] (4) The audio to be detected is preprocessed and its features are extracted, and then fed into the compressed detection model for detection to obtain the prediction result.

[0012] The labeling and classification in step (1) are as follows: First, obtain a certain number of labeled single-sound event audios, number each type of sound event, and obtain the total number of sound event categories. Then, some sound events are randomly synthesized to obtain synthesized audio, and the audio is then labeled. ,in This indicates that the first [unit / item] was used during synthesis. Classified sound events; finally, export a tag file, which records the audio file name and the category of each type of sound event that occurred under each audio file name.

[0013] The preprocessing and feature extraction described in steps (2) and (4) involve resampling the speech at a sampling rate of 16kHz, then standardizing the audio waveform and mapping the audio waveform data uniformly to... Above, max-normalization is used: ,in: This is data obtained by reading audio files (.wav) using the Python (wav) package; 40-dimensional log-Mel frequency cepstral coefficients are extracted from all audio data using Short Time Fourier Transform (STFT), with the following parameters: Sampling rate Frame overlap Extract the 40-dimensional log-Mel spectrum coefficients and normalize them using z-score:

[0014] Assumption The log-Mel cepstral coefficients obtained by passing the STFT for seconds are ,in, yes Frames per second ,

[0015]

[0016] The mapped log-Mel cepstral coefficients are obtained as follows: Its mean is 0 and its variance is 1.

[0017] The audio tags are transformed from tags in seconds to tags in frames. For each tag file, the following steps are performed to obtain an audio tag encoding matrix in frames. The tag encoding consists of 0 and 1 elements, the number of columns in the matrix is ​​equal to the total number of frames, and the number of rows in the matrix is... Total number of sound event categories. Among them: one contains... The process of converting the audio tag encoding matrix of abnormal sound events (sound-like events) from seconds to frames is as follows:

[0018] First, generate one OK The zero matrix of columns, the audio duration is seconds, number of rows in the matrix For the number of sound event categories; when the label is number 1 Sound-like events occur Time will make the first one The first sound-like event corresponding to the The duration of a row vector is converted to the length of a frame unit, and the corresponding zero vector is replaced with a 1 vector.

[0019] Finally, the matrix formed by combining the vectors of each individual sound event is the audio tag encoding matrix of the synthesized anomalous sound event.

[0020] The method for building the lightweight detection model with adaptive width self-attention mechanism described in step (3) includes:

[0021] 1) Pre-trained model:

[0022] The self-attention mechanism model network was built using the Python framework as follows: the model consists of 3 convolutions, 3 pooling operations, 1 gated recurrent unit (GRU), 1 layer of adaptive width self-attention mechanism, and 1 layer of temporal distribution; wherein: the first layer is the input layer, with 40-dimensional log-Melch cepstral coefficients as input; the second layer is a 2D convolution with 64 input channels (5*5 kernels) followed by (5*1) max 2D pooling; the third layer is a 2D convolution with 64 input channels (5*5 kernels) followed by (4*1) max 2D pooling; the fourth layer is a 2D convolution with 64 input channels (… The convolutional kernels are 5x5 layers followed by (2x1) max 2d pooling; the fifth layer consists of reshape and permute operations to reduce the dimensionality and transpose the output of the fourth layer; the sixth layer uses a GRU with 64 neurons and tanh activation; the seventh layer is an adaptive width self-attention mechanism using add-attention and sigmoid activation; the eighth layer uses sigmoid activation and the Dense parameter represents the time distribution of the number of sound event categories; each convolutional layer uses a kernel of size [missing information]. The convolutional kernel has a stride of 1, and each convolutional layer is normalized and activated by the ReLU function. In addition, dorupout is added to improve the generalization ability of the model.

[0023] Then, the output of the sixth layer is used as input to the attention layer. The input is multiplied by the three attention weight matrices of the attention layer to obtain three attention matrices: query Q, key K, and value V. After a series of operations, the attention weights are obtained, which represent the correlation between the current position of each output and other positions in the sequence. By continuously training and iterating the attention weights, the loss function is minimized, i.e., the model is optimal. Simultaneously, an adaptive width self-attention mechanism is employed, which allows the width to be controlled in each training iteration to achieve optimization. Specifically:

[0024] 2) Self-attention mechanism model

[0025] Feature sequences read and processed from audio files Then multiply by the corresponding attention matrix. Obtain the attention input matrix .

[0026]

[0027] in , This is the dimension output by the attention mechanism. Then, the following calculation is performed:

[0028]

[0029]

[0030] in It is pre-set. This indicates the time position. The final output is:

[0031]

[0032] 3) Adaptive Width Self-Attention Mechanism Model

[0033] The attention width is also included as a training parameter and fed into the model for training, adaptively selecting the attention width. This is implemented by introducing a mask function. :

[0034]

[0035] This function is a distance... Non-increasing functions mapped to [0,1] are derived from Parameterization, where It is the maximum width of the attention set. This represents the slope of the decrease in attention span; that is, the attention score at this point is:

[0036]

[0037] The adaptive width self-attention mechanism improves the effectiveness and reliability of urban road abnormal sound event detection methods by sacrificing some sequence information to a certain extent, saving computation time, filtering interference information, and improving computational efficiency.

[0038] Lightweighting: The trained adaptive width self-attention mechanism detection model uses low-precision (16-bit) floating-point numbers instead of high-precision (32-bit) floating-point numbers for storage and prediction. The general form of lightweighting is expressed as:

[0039]

[0040] in and These are the numbers before and after quantization, respectively. It is a quantification factor. It is the value of 0 in the original range after quantization. Because there are many 0s in the weights and input (such as padding or ReLU), the real number 0 needs to be accurately represented during quantization.

[0041] Among them, quantization factor The choice of quantization factor determines the error between the quantized model and the original model, therefore the quantization factor... The selection of the quantization factor is very important. To ensure that the quantized result is represented within a specified bit range (e.g., 16 bits), the quantization factor here... The selection is based on the following formula:

[0042]

[0043] in and These are the maximum and minimum values ​​of the object before quantization, respectively.

[0044] 4) Lightweight model with adaptive width self-attention mechanism

[0045] The training dataset, i.e., the log-Mel-cepstral coefficients of the synthesized audio, is fed into a lightweight adaptive width self-attention mechanism model. The initial values ​​of the weights in each layer of the model are randomly given by PyTorch, resulting in the output. Where C is the total number of event categories and T is the total number of frames, calculate the true positive prediction label loss. ,Will and The output is obtained by element-wise multiplication. Finally, the following binary cross-entropy loss function is calculated:

[0046]

[0047] Gradient backpropagation is performed using the Adam gradient descent method with a learning rate of 0.001. The weight parameters are updated, and training is iterated until the loss is minimized. The model parameters are then saved to obtain an adaptive width self-attention mechanism detection model.

[0048] The method for using the trained detection model to detect the model to be predicted in step (4) is as follows: the audio to be detected with unknown label is preprocessed and feature extracted in the same way as the adaptive width self-attention mechanism model, and then fed into the trained lightweight model of the adaptive width self-attention mechanism to obtain the neural network probability output and save it.

[0049] The optimal decision threshold is searched using the f1-score as the benchmark. According to the judgment threshold Binarization yields the prediction results from the labels. Based on the label prediction output matrix, the start and end time frame nodes of the sound event are determined. The cosine similarity between adjacent frames corresponding to the frame nodes in the neural network probability output matrix is ​​calculated. If the similarity is greater than 0.5, the frame is extended, which means the timestamp in the label matrix is ​​extended.

[0050] Finally, the prediction matrix after label extension is obtained, the recognition result is obtained, and the prediction is completed.

[0051] The advantages of this invention are: it proposes a method for quickly and accurately detecting the start and end times and event types of audio events containing multiple abnormal sound events. Based on the patent "A Sound Event Labeling and Recognition Method Using Dual Tokens" (patent application number: 202110465526.7), this method obtains a compressed detection model through lightweight training of an adaptive width self-attention mechanism model. The audio to be detected is input into the detection model, and the trained detection model is used to obtain the type of abnormal sound event in real time. This method compensates for the inability of CNNs to carry temporal information and avoids, to some extent, the gradient vanishing and long-term dependency problems that RNNs may cause. It leverages the parallel computing advantages of the self-attention mechanism and adaptively selects the width of the self-attention mechanism to focus on. Furthermore, with the help of lightweight design, it saves memory space while adaptively focusing on a certain width of input information, improving recognition performance and prediction speed. This improves the accuracy and effectiveness of abnormal sound event detection methods for urban roads and is expected to be deployed on mobile devices or other portable devices. Attached Figure Description

[0052] Figure 1 This is a flowchart of the detection method according to an embodiment of the present invention;

[0053] Figure 2 This is a model diagram of the self-attention mechanism in an embodiment of the present invention;

[0054] Figure 3 This is a model diagram of the adaptive width self-attention mechanism according to an embodiment of the present invention;

[0055] Figure 4 This is a diagram of the mask function in an embodiment of the present invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0057] A lightweight anomalous sound event detection method based on an adaptive width self-attention mechanism is presented in the main detection flowchart shown in Figure 1. The main steps include classifying and recognizing an audio segment containing multiple sounds. The detection process is as follows:

[0058] The entire flowchart is mainly divided into 7 modules: First, constructing the synthesized audio dataset; second, preprocessing and extracting features from the dataset; then, feeding it into the pre-built adaptive width self-attention mechanism model for iterative network training until the model reaches its optimal state; when saving the model parameters, a lightweight method is used to compress the model; finally, saving the model. The audio to be predicted, after undergoing the same preprocessing and feature extraction as the dataset, is fed into the saved detection model to obtain the prediction result. The specific method is as follows:

[0059] (1) Label and classify each audio file containing multiple abnormal sound events:

[0060] In machine learning-based anomaly detection, sound event detection models are typically trained on large amounts of labeled audio data. The resulting model is used to predict the type of event and its corresponding time when an unknown audio segment occurs. Since machine learning only has input and output, the size and realism of the dataset often have a significant impact on the quality of the detection results.

[0061] Overlapping sound events inherently contain multiple simultaneous events, making them very difficult to label. Therefore, we generally synthesize them from multiple single sound events to obtain audio containing multiple anomalous sound events. First, we obtain a certain number of labeled single sound event audio samples, number each type of sound event, and obtain the total number of sound event types. Then, some sound events are randomly synthesized to obtain synthesized audio, and the audio is then labeled. ,in This indicates that the first [unit / item] was used during synthesis. Sound events. Finally, export a tag file, which records the audio filenames and the category of each sound event that occurred under each audio filename.

[0062] While artificially synthesized anomalous sound event datasets have clear labels for each sound event category, their fit to real-world anomalous sound events is obviously not high, inevitably leading to prediction bias. Labeling real-world anomalous sound events typically relies on human listening, which is highly subjective, time-consuming, and labor-intensive. Furthermore, the simultaneous occurrence of multiple categories of anomalous sound events exponentially increases the labeling cost. To increase the dataset size and prevent overfitting, the following data augmentation methods were used to triple the original audio data size: random audio scaling, time masking, frequency masking, adding random noise, and audio sample mixing. Ultimately, this data augmentation enriched the dataset and prevented overfitting.

[0063] (2) Audio data preprocessing and feature extraction:

[0064] Because audio may originate from various devices, all speech processing involves resampling at a sampling rate of 16kHz. Then, the audio waveform is standardized, uniformly mapping the audio waveform data to... Above, max-normalization is used: , This is data obtained by reading audio files (.wav) using the Python (wav) package. Then, a Short Time Fourier Transform (STFT) is used to extract 40-dimensional log-Mel frequency cepstral coefficients from all audio data. The specific parameters are as follows: Sampling rate Frame overlap Extract the 40-dimensional log-Mel spectrum coefficients and normalize them using the following z-score: assuming The log-Mel cepstral coefficients obtained by passing the STFT for seconds are ,in, yes Frames per second ,

[0065]

[0066] The mapped log-Mel cepstral coefficients are obtained as follows: Its mean is 0 and its variance is 1.

[0067] For audio tags: Tags in seconds are converted to tags in frames. For each tag file, the following steps are performed to obtain an audio tag encoding matrix in frames. The tag encoding consists of 0 and 1 elements. The number of columns in the matrix is ​​equal to the total number of frames, and the number of rows in the matrix is... The total number of sound event categories, one that includes The process of converting the audio tag encoding matrix of abnormal sound events (sound-like events) from seconds to frames is as follows:

[0068] First, generate one OK The zero matrix of columns, the audio duration is seconds, number of rows in the matrix Number of sound event categories.

[0069] If the tag is Sound-like events occur Time will make the first one The first sound-like event corresponding to the The duration of a row vector is converted to the length of a frame unit, and the corresponding zero vector is replaced with a 1 vector.

[0070] Finally, the matrix formed by combining the vectors of each individual sound event is the audio tag encoding matrix of the synthesized anomalous sound event.

[0071] (3) Building a pre-trained model:

[0072] The self-attention mechanism model network was built using the Python framework as follows: the model went through 3 convolutions, 3 poolings, 1 gated recurrent unit (GRU), 1 layer of adaptive width self-attention mechanism, and 1 layer of temporal distribution. The first layer is the input layer, receiving 40-dimensional log-Melch cepstral coefficients. The second layer is a 2D convolutional layer with 64 input channels (5*5 kernels) followed by a (5*1) max 2D pooling. The third layer is a 2D convolutional layer with 64 input channels (5*5 kernels) followed by a (4*1) max 2D pooling. The fourth layer is a 2D convolutional layer with 64 input channels (5*5 kernels) followed by a (2*1) max 2D pooling. The fifth layer is a reshape and permute operation that reduces the dimensionality and transposes the output of the fourth layer. The sixth layer uses a GRU with 64 neurons and tanh activation. The seventh layer is an adaptive width self-attention mechanism using add-attention and sigmoid activation. The eighth layer uses sigmoid activation and a time-distributed density (Dense) representing the number of sound event categories. Each convolutional layer uses a density of [missing information - likely a value]. The convolutional kernel has a stride of 1, and each convolutional layer is normalized and activated by the ReLU function. In addition, dorupout is added to improve the generalization ability of the model.

[0073] The attention layer is a key feature of this invention. The output of the sixth layer is input into the attention layer, which is then multiplied by its three attention weight matrices to obtain the query Q, key K, and value V attention matrices. These matrices are then processed through a series of operations to obtain the attention weights, representing the correlation between each output's current position and other positions in the sequence. By iteratively training the attention weights, the loss function is minimized, resulting in an optimal model. Since sequence information farther from the current position is relatively less important, a width-controlled self-attention mechanism is employed. To avoid losing important information, an adaptive width self-attention mechanism is used, allowing for optimization by controlling the width in each training iteration. This approach reduces computation time and improves model efficiency to some extent without sacrificing important information.

[0074] Self-attention mechanism models: such as Figure 2 As shown, the input is a feature sequence read and processed from an audio file. Then multiply by the corresponding attention matrix. Obtain the attention input matrix .

[0075]

[0076] in , This is the dimension output by the attention mechanism. Then, the following calculation is performed:

[0077]

[0078]

[0079] in It is pre-set. This indicates the time position. The final output is:

[0080]

[0081] 2) Adaptive Width Self-Attention Mechanism Model: While traditional self-attention mechanisms overcome some shortcomings in SED tasks based on CNNs and RNNs, when the audio duration is too long, the computation time is relatively long because traditional self-attention mechanisms need to pay attention to all sequence information. Furthermore, the distance from the current time... Information from more distant times is relatively less important, and may even contain interfering information. This leads to situations such as... Figure 3 The self-attention mechanism shown controls the attention width. Under this attention model, it is possible to adjust the attention width according to the self-attention mechanism. Perform on the current position This allows for a focus on the surrounding sequence information, eliminating the need to consider all sequence information and improving the efficiency of model computation. It enables the system to significantly reduce computation time and greatly enhance system performance by sacrificing a small amount of sequence information.

[0082] Adaptive Width Self-Attention Mechanism Model: To better conserve computer resources and filter out distracting information, this invention proposes an adaptive width self-attention mechanism. The adaptive width self-attention mechanism incorporates the attention width as a training parameter, integrating it into the model for training and learning, thus adaptively selecting the attention width. Implementation involves introducing a mechanism such as... Figure 4 mask function :

[0083]

[0084] This function is a distance... Non-increasing functions mapped to [0,1] are derived from Parameterization, where It is the maximum width of the attention set. This represents the slope of the decrease in attention span. That is, the attention score at this point is:

[0085]

[0086] The adaptive width self-attention mechanism improves the effectiveness and reliability of urban road abnormal sound event detection methods by sacrificing some sequence information to a certain extent, saving computation time, filtering interference information, and improving computational efficiency.

[0087] Lightweighting: The lightweighting process involves replacing high-precision (32-bit) floating-point numbers with low-precision (16-bit) floating-point numbers during storage and prediction of the trained adaptive width self-attention mechanism detection model. This method can save approximately half of the storage space during storage and reduce the computational latency during prediction, thereby conserving storage resources and accelerating computation. The general form of lightweighting is expressed as:

[0088]

[0089] in and These are the numbers before and after quantization, respectively. It is a quantification factor. It is the value of 0 in the original range after quantization. Because there are many 0s in the weights and input (such as padding or ReLU), the real number 0 needs to be accurately represented during quantization.

[0090] Among them, quantization factor The choice of quantization factor determines the error between the quantized model and the original model, therefore the quantization factor... The selection of the quantization factor is very important. To ensure that the quantized result is represented within a specified bit range (e.g., 16 bits), the quantization factor here... The selection is based on the following formula:

[0091]

[0092] in and These are the maximum and minimum values ​​of the object before quantization, respectively.

[0093] 4) Lightweight model of adaptive width self-attention mechanism: The training dataset, i.e., the log-Mel-cepstral coefficients of the synthesized audio, is fed into the lightweight model of adaptive width self-attention mechanism built in module 3). The initial values ​​of the weights of each layer in the model are randomly given by PyTorch, and the output is obtained. Where C is the total number of event categories and T is the total number of frames, calculate the true positive prediction label loss. ,Will and The output is obtained by element-wise multiplication. Finally, the following binary cross-entropy loss function is calculated:

[0094]

[0095] Gradient backpropagation is performed using the Adam gradient descent method with a learning rate of 0.001. The weight parameters are updated, and training is iterated until the loss is minimized. The model parameters are then saved to obtain an adaptive width self-attention mechanism detection model.

[0096] (4) Use the trained detection model to detect the model to be predicted: The audio to be detected with unknown labels undergoes the same preprocessing and feature extraction as the self-attention mechanism model, and is then fed into the trained lightweight model of the adaptive width self-attention mechanism to obtain the neural network probability output, which is then saved. The optimal decision threshold is searched using the f1-score as the benchmark. According to the judgment threshold Binarization yields the prediction results from the labels. Specifically, it involves determining the start and end time frames of the sound event based on the label prediction output matrix, and calculating the cosine similarity between adjacent frames corresponding to the frame nodes in the neural network probability output matrix. If the similarity is greater than 0.5, the frame is extended, i.e., the timestamp in the label matrix is ​​extended. Finally, the extended prediction matrix is ​​obtained, yielding the recognition result and completing the prediction.

[0097] Note: The extension on the left and right sides must not exceed the preset hyperparameter collar value, which is generally (250ms-50ms).

Claims

1. A lightweight anomalous acoustic event detection method based on an adaptive width self-attention mechanism, characterized by: Includes the following steps: (1) Construct a synthetic audio dataset and label and classify each audio file containing multiple anomalous sound events; (2) Preprocess and extract features from the dataset, and feed it into the built adaptive width self-attention mechanism model for iterative training of the network until the model reaches its optimal state; (3) The model is compressed using a lightweight method to obtain a lightweight detection model with an adaptive width self-attention mechanism; the method for building the lightweight detection model with the adaptive width self-attention mechanism includes: 1) Pre-trained model: The self-attention mechanism model network was built using the Python framework as follows: the model consists of 3 convolutions, 3 pooling operations, 1 gated recurrent unit (GRU), 1 layer of adaptive width self-attention mechanism, and 1 layer of temporal distribution; where: the first layer is the input layer, with 40-dimensional log-Melch cepstral coefficients as input; the second layer is a 64*5*5 convolutional layer followed by max pooling; the third layer is a 64*5*5 convolutional layer followed by 4*1 max pooling; the fourth layer is a 64*5*5 convolutional layer followed by 2*1 max pooling; the fifth layer consists of reshape and permute operations to reduce the dimensionality and transpose the output of the fourth layer; the sixth layer uses a GRU with 64 neurons and tanh activation; the seventh layer is an adaptive width self-attention mechanism using add-attention and sigmoid activation; the eighth layer uses sigmoid activation and Dense is the temporal distribution of the number of sound event categories; each convolutional layer uses a size of The convolutional kernel has a stride of 1, and each convolutional layer is followed by a normalized layer, all of which are activated by the 'ReLU' function, and dolopout is also added; Then, the output of the sixth layer is used as input to the attention layer. The input is multiplied by the three attention weight matrices of the attention layer to obtain three attention matrices: query Q, key K, and value V. These are then used to calculate the attention weights. By continuously training and iterating the attention weights, the loss function is minimized, resulting in the optimal model. Simultaneously, an adaptive width self-attention mechanism is employed, allowing the width to be controlled in each training iteration to achieve optimization. Where: 2) Self-attention mechanism model: Feature sequences read and processed from audio files Then multiply by the corresponding attention matrix. Obtain the attention input matrix ; ； in , This is the dimension of the attention mechanism's output; then the following calculation is performed: ； ； in It is a pre-set value. Indicates the time position; the final output is: ； 3) Adaptive width self-attention mechanism model: The attention width is also included as a training parameter and fed into the model for training, adaptively selecting the attention width; in practice, a mask function is introduced. : ； This function is a distance... The non-increasing function mapped to [0,1] is given by b. Parameterization, where It is the maximum width of the attention set. This represents the slope of the decrease in attention span; the attention score at this point is: ； The adaptive width self-attention mechanism improves the effectiveness and reliability of urban road abnormal sound event detection methods by sacrificing some sequence information to a certain extent, saving computation time, filtering interference information, and improving computational efficiency. Lightweighting: The trained adaptive width self-attention mechanism detection model uses 16-bit low-precision floating-point numbers instead of 32-bit high-precision floating-point numbers for storage and prediction. The general form of lightweighting is expressed as: ； in and These are the numbers before and after quantization, respectively. It is a quantification factor. It is the value of 0 in the original range after quantization. Because there will be many 0s in the weights and inputs during padding and ReLU, the real number 0 needs to be accurately represented during quantization. Among them, quantization factor The choice of quantization factor determines the error between the quantized model and the original model, therefore the quantization factor... The selection of the quantization factor is crucial to ensure that the quantized result falls within the 16-bit representation range. The selection is based on the following formula: ； in and These are the maximum and minimum values ​​of the object before quantization; 4) Lightweight model with adaptive width self-attention mechanism: The log-Melch cepstral coefficients of the training dataset are fed into a lightweight adaptive width self-attention mechanism model. The initial values ​​of the weights in each layer of the model are randomly given by PyTorch, and the output is obtained. Where C is the total number of event categories and T is the total number of frames, calculate the true positive prediction label loss. ,Will and The output is obtained by element-wise multiplication. Finally, the following binary cross-entropy loss function is calculated: ； Gradient backpropagation is performed using the Adam gradient descent method with a learning rate of 0.

001. The weight parameters are updated, and training is iterated until the loss is minimized. The model parameters are then saved to obtain an adaptive width self-attention mechanism detection model. (4) The audio to be detected is preprocessed and its features are extracted, and then fed into the compressed detection model for detection to obtain the prediction result.

2. The lightweight anomalous acoustic event detection method based on adaptive width self-attention mechanism according to claim 1, characterized in that: The labeling and classification in step (1) are as follows: First, obtain a certain number of labeled single-sound event audios, number each type of sound event, and obtain the total number of sound event categories. Then, some sound events are randomly synthesized to obtain synthesized audio, and the audio is then labeled. ,in This indicates that the first [unit / item] was used during synthesis. Classified sound events; finally, export a tag file, which records the audio file name and the category of each type of sound event that occurred under each audio file name.

3. The lightweight anomalous acoustic event detection method based on adaptive width self-attention mechanism according to claim 1, characterized in that: The preprocessing and feature extraction described in steps (2) and (4) involve resampling the speech at a sampling rate of 16kHz, then standardizing the audio waveform and mapping the audio waveform data uniformly to... Above, max-normalization is used: ,in: This is data obtained by reading audio files (.wav) using the Python (wav) package; 40-dimensional log-Mel frequency cepstral coefficients are extracted from all audio data using Short Time Fourier Transform (STFT), with the following parameters: Sampling rate Frame overlap Extract the 40-dimensional log-Mel spectrum coefficients and normalize them using z-score: Assumption The log-Mel cepstral coefficients obtained by passing the STFT for seconds are ,in, yes Frames per second , ; ； The mapped log-Mel cepstral coefficients are obtained as follows: Its mean is 0 and its variance is 1.

4. The lightweight anomalous acoustic event detection method based on adaptive width self-attention mechanism according to claim 1, characterized in that: The audio tags in step (1) are converted from tags in seconds to tags in frames. For each tag file, the following steps are performed to transform the audio tag encoding matrix in frames. The tag encoding consists of 0 and 1 elements. The number of columns in the matrix is ​​the total number of frames, and the number of rows in the matrix is... Total number of sound event categories; where: one contains The process of converting the audio tag encoding matrix of abnormal sound events (sound-like events) from seconds to frames is as follows: First, generate one OK The zero matrix of columns, the audio duration is seconds, number of rows in the matrix For the number of sound event categories; when the label is number 1 Sound-like events occur Time will make the first one The first sound-like event corresponding to the The duration of a row vector is converted to the length of a frame unit, and the corresponding zero vector is replaced with a 1 vector. Finally, the matrix formed by combining the vectors of each individual sound event is the audio tag encoding matrix of the synthesized anomalous sound event.

5. The lightweight anomalous acoustic event detection method based on adaptive width self-attention mechanism according to claim 1, characterized in that: The method for using the trained detection model to detect the model to be predicted in step (4) is as follows: the audio to be detected with unknown label is preprocessed and feature extracted in the same way as the adaptive width self-attention mechanism model, and then fed into the trained lightweight model of the adaptive width self-attention mechanism to obtain the neural network probability output and save it. The optimal decision threshold is searched using the f1-score as the benchmark. According to the judgment threshold Binarization yields the prediction results from the labels. Based on the label prediction output matrix, the start and end time frame nodes of the sound event are determined. The cosine similarity between adjacent frames corresponding to the frame nodes of the neural network probability output matrix is ​​calculated. If the similarity is greater than 0.5, the frame is extended, and adjacent frames are also considered as having the sound event. Finally, the prediction matrix after label extension is obtained, the recognition result is obtained, and the prediction is completed.

Images