An audio content security detection system based on deep learning

Abstract

The present invention relates to the technical field of content detection, and discloses an audio content security detection system based on deep learning. The system obtains audio data from public data sets, self-recorded audio and online platforms through a data acquisition module, a data preprocessing module cleans the audio data, and calculates the audio signal-to-noise ratio for evaluating the audio quality, a feature extraction module extracts effective features from the preprocessed audio, including Mel spectrum coefficients, short-time Fourier transform, zero crossing rate, signal amplitude and audio feature variance, a model training module selects a convolutional neural network as a deep learning architecture, uses the features extracted by the feature extraction module and creates a label data set, uses a cross entropy loss function for model training, a model verification module creates a verification data set for model verification, and detects the security of its content, and a result output module generates a comprehensive report and outputs it to a display terminal.

Description

Technical Field

[0001] The present invention relates to the technical field of content detection, and in particular to an audio content security detection system based on deep learning. Background Art

[0002] With the continuous development of technology and the widespread use of social media, audio content has become easier to spread and obtain. Audio content such as audio news, podcasts, music, etc. can quickly spread false information or inflammatory remarks, which has a negative impact on social stability and public safety. Ensuring the security of audio content can effectively prevent the spread of false information and rumors. At the same time, social media and online platforms rely on user-generated content. If this content is not effectively tested for security, it may lead to a decline in user experience and affect the platform's reputation and user loyalty. Therefore, if audio content is not tested, false and misleading information may be widely disseminated, leading to public misunderstanding of important events.

[0003] Existing audio detection technologies may not be sensitive enough to accents, dialects or noisy background sounds when processing speech recognition and audio content review, making it impossible to accurately identify the security of audio content, and may also cause false positives or negatives in audio content. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides an audio content security detection system based on deep learning, which has a data acquisition module for acquiring audio data from public data sets, self-recorded audio and online platforms, including annotated audio samples and environmental noise samples; a data preprocessing module for cleaning audio data, normalizing the audio length by cropping and padding, performing noise elimination and signal enhancement, and calculating the audio signal-to-noise ratio Ypxz for evaluating audio quality; a feature extraction module for extracting effective features from the preprocessed audio, including Mel spectrum coefficients Mexs, short-time Fourier transform Dsbh, zero crossing rate JCll, signal amplitude Xhfd and audio feature variance Ypfc; the above features are used as model inputs; a model training module selects a convolutional neural network as a deep learning architecture, The features extracted by the feature extraction module are used to create a label data set, and the cross entropy loss function is used for model training. The model verification module creates a verification data set for model verification, and performs inference on new audio samples to detect the security of their content. The new audio samples are preprocessed and feature extracted, and the features are input into the trained model for classification and regression, and the verification results are output. The result output module calculates the unsafe sample rate Baql, the false alarm rate Wblz, and the missed alarm rate Lblz according to the total number of detected samples, and generates a comprehensive report to output to the display terminal. By cleaning and denoising the collected audio data, extracting relevant features to train the audio security detection model, the security of the audio content can be accurately identified, while reducing the false alarm rate and missed alarm rate of the audio content, etc., thereby solving the above problems.

[0006] (II) Technical solution

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: an audio content security detection system based on deep learning, comprising a data acquisition module, a data preprocessing module, a feature extraction module, a model training module, a model verification module and a result output module;

[0008] The data acquisition module acquires audio data through public data sets, self-recorded audio, and online platforms, including annotated audio samples and environmental noise samples, and transmits the acquired data to the data preprocessing module;

[0009] The data preprocessing module cleans the audio data, normalizes the audio length by cutting and padding, performs noise elimination and signal enhancement, and calculates the audio signal-to-noise ratio Ypxz for evaluating the audio quality;

[0010] The feature extraction module extracts effective features from the preprocessed audio, including Mel spectrum coefficient Mexs, short-time Fourier transform Dsbh, zero crossing rate Jcll, signal amplitude Xhfd and audio feature variance Ypfc, and the above features are used as model input;

[0011] The model training module selects a convolutional neural network as a deep learning architecture, uses the features extracted by the feature extraction module to create a label data set, and uses a cross entropy loss function for model training;

[0012] The model verification module creates a verification data set for model verification, performs reasoning on new audio samples, detects the security of their content, preprocesses and extracts features of the new audio samples, inputs the features into the trained model, performs classification and regression, and outputs verification results;

[0013] The result output module calculates the unsafe sample rate Baql, the false alarm rate Wblz and the missed alarm rate Lblz according to the total number of detected samples, and generates a comprehensive report to output to the display terminal.

[0014] Preferably, the data preprocessing module cleans the audio data, normalizes the audio length by cutting and padding, performs noise elimination and signal enhancement, and calculates the audio signal-to-noise ratio Ypxz, and the calculation formula is as follows:

[0015]

[0016] In the formula, Ypxz represents the audio signal-to-noise ratio, P signal Represents the power of the audio signal, P noise Represents the power of audio noise.

[0017] Preferably, the formula for extracting the Mel spectrum coefficient Mexs from the preprocessed audio by the feature extraction module is as follows:

[0018]

[0019] In the formula, Mexs represents the Mel spectrum coefficient, S(k) represents the energy of the kth Mel frequency channel, A(k) represents the weight of the Mel filter bank, K represents the total number of Mel frequency channels, and n represents the latitude number of the Mel spectrum coefficient.

[0020] Preferably, the formula for extracting the short-time Fourier transform Dsbh from the preprocessed audio by the feature extraction module is as follows:

[0021]

[0022] In the formula, Dsbh(y, k) represents short-time Fourier transform, x(m) represents the discrete sample of the input audio signal, m is the time index, w(my) represents the window function, which is a weighted function for time m and is used for localization of the time signal, y represents the center position of the current time frame window, represents the complex exponential term, represents the fundamental frequency component at frequency k, and N represents the length of the window.

[0023] Preferably, the formula for extracting the zero crossing rate Jcll from the preprocessed audio by the feature extraction module is as follows:

[0024]

[0025] In the formula, Jcll represents the zero crossing rate, L represents the total number of signal samples, δ(p(t)*p(t-1) represents the unit cross-order function, and t represents the time index.

[0026] Preferably, the formula for extracting the signal amplitude Xhfd from the preprocessed audio by the feature extraction module is as follows:

[0027]

[0028] In the formula, Xhfd represents the signal amplitude, and X(a) represents the audio signal.

[0029] Preferably, the formula for extracting the audio feature variance Ypfc from the preprocessed audio by the feature extraction module is as follows:

[0030]

[0031] In the formula, Ypfc 2 represents the characteristic variance, L represents the total number of signal samples, x[i] represents the eigenvalue of the i-th sample, and μ represents the mean of the sample characteristics.

[0032] Preferably, the result output module calculates the unsafe sample rate Baql according to the total number of detected samples, and the calculation formula is as follows:

[0033]

[0034] In the formula, Baql represents the unsafe sample rate, U represents the number of samples marked as unsafe, and L represents the total number of signal samples.

[0035] Preferably, the result output module calculates the false alarm rate Wblz according to the total number of detected samples, and the calculation formula is as follows:

[0036]

[0037] In the formula, Wblz represents the false alarm rate, FP represents the number of false alarm samples, and TN represents the number of truly safe samples.

[0038] Preferably, the result output module calculates the underreporting rate Lblz according to the total number of detected samples, and the calculation formula is as follows:

[0039]

[0040] In the formula, Lblz represents the false negative rate, FN represents the number of false negative samples, and TP represents the number of truly unsafe samples.

[0041] Compared with the prior art, the present invention provides an audio content security detection system based on deep learning, which has the following beneficial effects:

[0042] The present invention obtains audio data from public data sets, self-recorded audio and online platforms through a data acquisition module, including annotating audio samples and environmental noise samples. The data preprocessing module cleans the audio data, normalizes the audio length by cropping and padding, performs noise elimination and signal enhancement, and calculates the audio signal-to-noise ratio Ypxz for evaluating audio quality. The feature extraction module extracts effective features from the preprocessed audio, including Mel spectrum coefficient Mexs, short-time Fourier transform Dsbh, zero crossing rate Jcll, signal amplitude Xhfd and audio feature variance Ypfc. The above features are used as model inputs. The model training module selects convolutional neural network as the deep learning architecture, and uses the features extracted by the feature extraction module to extract the audio data. The model verification module creates a verification data set for model verification, and performs inference on new audio samples to detect the security of their content. The new audio samples are preprocessed and feature extracted, and the features are input into the trained model for classification and regression, and the verification results are output. The result output module calculates the unsafe sample rate Baql, false alarm rate Wblz, and missed alarm rate Lblz according to the total number of detected samples, and generates a comprehensive report to output to the display terminal. By cleaning and denoising the collected audio data, relevant features are extracted to train the audio security detection model, the security of the audio content can be accurately identified, and the false alarm rate and missed alarm rate of the audio content can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In view of the fact that the existing audio detection technology may not be sensitive enough to accents, dialects or noisy background sounds when processing speech recognition and audio content review, and thus cannot accurately identify the security of audio content, and may also cause false positives or false negatives of audio content, a deep learning-based audio content security detection system is proposed. Figure 1 ,The system includes a data acquisition module, a data preprocessing module, a feature extraction module, a model training module, a model verification module and a result output module;

[0046] The data collection module automatically accesses multiple open audio databases and uses API interfaces to download well-labeled audio samples. These databases usually contain audio in different scenarios, languages, and environmental conditions, providing rich data information. To ensure the diversity and representativeness of the data, the system will filter and classify the data, select audio clips related to the project goals, and automatically capture public audio content on social media, podcasts, and audio sharing platforms through web crawler technology. Use natural language processing technology to extract tags and descriptions related to the topic for effective screening and classification. During the collection process, the module will simultaneously record and filter out audio samples with multiple labels such as sound type, emotional color, and background noise, and ensure the accuracy and consistency of the labels for subsequent supervised learning applications. After the collection is completed, the audio data is uploaded to the central server using the HTTPS data transmission protocol;

[0047] The data preprocessing module standardizes the format of the collected audio data, supports the conversion of multiple audio formats, and uses the Librosa audio processing library for file import and initialization. For missing, repeated or abnormal audio clips, they will be identified and eliminated through a threshold-based filtering algorithm to ensure that only high-quality audio samples are retained. The dynamic cropping technology is used to automatically analyze the audio, identify the effective part of each audio, and crop the silent segments and irrelevant parts to ensure that each audio clip only retains important information. If the length of an audio is less than the preset standard, the system will use zero padding or loop padding to supplement the audio clip to the standard length to ensure the uniformity of the data and prepare for further analysis and modeling. At the same time, the module will separate the signal and noise for each audio clip, and calculate the audio signal-to-noise ratio based on the power of the original signal and noise. The calculation formula is as follows:

[0048]

[0049] Calculating the audio signal-to-noise ratio helps to evaluate the quality of audio samples and ensure that the audio data input into the deep learning model meets the expected standards in terms of sound quality. This can reduce the negative impact of low-quality audio on model training and reasoning. In the formula, Ypxz represents the audio signal-to-noise ratio, P signal Represents the power of the audio signal, P noise Represents the power of audio noise. By focusing on the signal-to-noise ratio, the system can better handle audio inputs of different qualities. Even under low signal-to-noise ratio conditions, the model can still adapt and effectively identify potential safety hazards.

[0050] The feature extraction module extracts effective features from the preprocessed audio, including Mel spectrum coefficient Mexs, short-time Fourier transform Dsbh, zero crossing rate Jcll, signal amplitude Xhfd and audio feature variance Ypfc. The above features are used as model inputs, where:

[0051] The calculation formula of Mel spectrum coefficient is as follows:

[0052]

[0053] Mel spectrum coefficients are highly robust to environmental noise, which enables the model to better handle various background noises in practical applications and improve the ability to recognize sound content. In the formula, Mexs represents the Mel spectrum coefficient, S(k) represents the energy of the kth Mel frequency channel, A(k) represents the weight of the Mel filter group, K represents the total number of Mel frequency channels, and n represents the latitude number of the Mel spectrum coefficient. By reducing the dimension, important information in the audio data is retained, which helps to reduce the computational burden and improve the training efficiency and prediction accuracy of the model.

[0054] The short-time Fourier transform formula is as follows:

[0055]

[0056] Short-time Fourier transform can analyze signals in two dimensions: time and frequency, so that the system can capture the changes of audio signals in the time domain, helping to more accurately identify and classify audio content. In the formula, Dsbh(y, k) represents short-time Fourier transform, x(m) represents the discrete samples of the input audio signal, m is the time index, w(my) represents the window function, which is a weighted function for time m and is used for localization of time signals. y represents the center position of the current time frame window. represents the complex exponential term, represents the fundamental frequency component at frequency k, N represents the length of the window, and the result of short-time Fourier transform can be visualized using a spectrogram, allowing researchers to more intuitively understand and analyze the characteristics of audio signals, thereby providing a basis for model tuning;

[0057] The formula for calculating the zero crossing rate is as follows:

[0058]

[0059] The zero crossing rate is a feature that describes the frequency of the signal crossing zero points. It is suitable for detecting noise and sound boundaries in audio and helps determine whether there is a specific type of content in the audio. In the formula, Jcll represents the zero crossing rate, L represents the total number of signal samples, δ(p(t)*p(t-1) represents the unit cross-order function, and t represents the time index. By analyzing the change in the zero crossing rate, the system can perform audio segmentation and event detection and support the processing of structured audio.

[0060] The signal amplitude calculation formula is as follows:

[0061]

[0062] Signal amplitude information can reveal the dynamic range and volume changes of audio signals, helping the model analyze the intensity and loudness of audio content. In the formula, Xhfd represents the signal amplitude, and X(a) represents the audio signal. In security detection, abnormal signal amplitudes such as sudden increases or decreases may indicate abnormal audio activities. Calculating signal amplitudes can help detect such changes.

[0063] The formula for calculating the audio feature variance is as follows:

[0064]

[0065] Feature variance can help the system evaluate the consistency and stability of audio features. A higher variance may indicate the diversity and complexity of the signal, which helps to understand the characteristic changes of the signal. In the formula, Ypfc 2 represents the feature variance, L represents the total number of signal samples, x[i] represents the feature value of the i-th sample, and μ represents the mean of the sample features. In the process of training the model, the feature variance information can help the system reduce the dimension or perform feature selection, retain important features, and remove noise and redundant information, thereby improving the efficiency of the model.

[0066] The model training module uses convolutional neural networks as the core deep learning architecture. This is because convolutional neural networks have significant advantages in processing high-dimensional data with spatial structures. They can effectively and automatically extract multi-level feature information. By calculating multiple audio features such as Mel spectrum coefficients, short-time Fourier transform, zero crossing rate, signal amplitude, and audio feature variance, the original audio signal is converted into feature vectors suitable for model training. These feature vectors are used as input data and create a labeled data set with corresponding labels. The labels indicate the specific category of the audio or whether there is unsafe content. During the model training process, the cross-entropy loss function is used to quantify the gap between the model output and the true label. The cross-entropy loss function can effectively evaluate the performance of classification tasks, especially when dealing with multi-category classification, it can improve the model's ability to distinguish between each category. Through stochastic gradient descent or Adam optimization algorithm, the network parameters are gradually iteratively updated to minimize the cross-entropy loss, thereby optimizing the performance and accuracy of the model. During the training process, batch training and data enhancement techniques such as time jitter and volume change are used to improve the generalization ability of the model, ultimately enabling the system to accurately perform security detection and classification when facing unknown or noise-interfered audio content;

[0067] An independent validation dataset is created in the model validation module. The dataset contains a variety of new audio samples to ensure that it covers a wide range of content types and environmental conditions to comprehensively test the generalization ability of the model. Before reasoning about new audio samples, a series of preprocessing steps are taken, including denoising, normalization, and framing of audio signals. These steps ensure the quality of the audio signals and make subsequent feature extraction more efficient and accurate. The feature extraction module is then used to extract features from the preprocessed audio data and calculate key features such as Mel spectrum coefficients, short-time Fourier transform, zero crossing rate, signal amplitude, and audio feature variance. These feature vectors are organized into input formats and then input into the trained convolutional neural network model for reasoning. During the model operation, the extracted audio features are classified and regressed using its deep network structure to determine the security of the audio content.

[0068] The model outputs the prediction results generated for each audio sample, including classification labels and confidence scores, indicating whether the audio contains unsafe content and its probability distribution in each category. Finally, the verification results are summarized and analyzed, and the performance of the model is evaluated by comparing with the actual labels, including accuracy, recall and F1-scor e This comprehensive verification process not only ensures the effectiveness of the model, but also provides important feedback for the continuous optimization of the system, thereby improving the reliability of overall audio content security detection;

[0069] The result output module calculates the unsafe sample rate Baql, the false alarm rate Wblz and the missed alarm rate Lblz according to the total number of detected samples, and generates a comprehensive report to output to the display terminal, where:

[0070] The formula for calculating the unsafe sample rate is as follows:

[0071]

[0072] The unsafe sample rate provides the proportion of potentially dangerous audio that the system can identify. In the formula, Baql represents the unsafe sample rate, U represents the number of samples marked as unsafe, and L represents the total number of signal samples. By understanding this ratio, developers can evaluate the performance of the model in identifying unsafe content and clarify the effectiveness of the model.

[0073] The formula for calculating the false alarm rate is as follows:

[0074]

[0075] By calculating the false alarm rate, developers can adjust and optimize the model's hyperparameters to further improve its accuracy, which provides a specific reference for model improvement and performance enhancement. In the formula, Wblz represents the false alarm rate, FP represents the number of false alarm samples, and TN represents the number of truly safe samples. A high false alarm rate may indicate that the model has deviations in certain specific features. Understanding this helps to deeply analyze the data set and take corrective measures.

[0076] The formula for calculating the false negative rate is as follows:

[0077]

[0078] Monitoring the false negative rate enables the system to detect unidentified dangerous samples in a timely manner, laying the foundation for further research and model improvement, and ensuring the adaptability of the system in a rapidly changing threat environment. In the formula, Lblz represents the false negative rate, FN represents the number of false negative samples, and TP represents the number of truly unsafe samples. Evaluating the false negative rate can provide direction for subsequent data training, helping developers effectively locate the shortcomings of the model in identifying unsafe audio and achieve targeted optimization.

[0079] By cleaning and denoising the collected audio data and extracting relevant features to train the audio security detection model, the security of audio content can be accurately identified, while reducing the false alarm rate and missed alarm rate of audio content.

[0080] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An audio content security detection system based on deep learning, characterized in that: It includes data acquisition module, data preprocessing module, feature extraction module, model training module, model verification module and result output module; The data acquisition module acquires audio data through public data sets, self-recorded audio, and online platforms, including annotated audio samples and environmental noise samples, and transmits the acquired data to the data preprocessing module; The data preprocessing module cleans the audio data, normalizes the audio length by cutting and padding, performs noise elimination and signal enhancement, and calculates the audio signal-to-noise ratio Ypxz for evaluating the audio quality; The feature extraction module extracts effective features from the preprocessed audio, including Mel spectrum coefficient Mexs, short-time Fourier transform Dsbh, zero crossing rate Jcll, signal amplitude Xhfd and audio feature variance Ypfc, and the above features are used as model input; The model training module selects a convolutional neural network as a deep learning architecture, uses the features extracted by the feature extraction module and creates a label data set, and uses a cross entropy loss function for model training; The model verification module creates a verification data set for model verification, performs reasoning on new audio samples, detects the security of their content, preprocesses and extracts features from new audio samples, inputs the features into the trained model, performs classification and regression, and outputs verification results; The result output module calculates the unsafe sample rate Baql, the false alarm rate Wblz and the missed alarm rate Lblz according to the total number of detected samples, and generates a comprehensive report to output to the display terminal.

2. According to claim 1, a deep learning-based audio content security detection system is characterized in that: The data preprocessing module cleans the audio data, normalizes the audio length by cropping and padding, performs noise elimination and signal enhancement, and calculates the audio signal-to-noise ratio Ypxz. The calculation formula is as follows: In the formula, Ypxz represents the audio signal-to-noise ratio, P signal Represents the power of the audio signal, P noise Represents the power of audio noise.

3. The audio content security detection system based on deep learning according to claim 2 is characterized in that: The formula for extracting the Mel spectrum coefficient Mexs from the preprocessed audio by the feature extraction module is as follows: In the formula, Mexs represents the Mel spectrum coefficient, S(k) represents the energy of the kth Mel frequency channel, A(k) represents the weight of the Mel filter bank, K represents the total number of Mel frequency channels, and n represents the latitude number of the Mel spectrum coefficient.

4. The audio content security detection system based on deep learning according to claim 3 is characterized in that: The formula for extracting the short-time Fourier transform Dsbh from the preprocessed audio by the feature extraction module is as follows: In the formula, Dsbh(y,k) represents short-time Fourier transform, x(m) represents the discrete sample of the input audio signal, m is the time index, w(my) represents the window function, which is a weighted function for time m and is used for localization of the time signal, y represents the center position of the current time frame window, represents the complex exponential term, represents the fundamental frequency component at frequency k, and N represents the length of the window.

5. The audio content security detection system based on deep learning according to claim 4 is characterized in that: The formula for extracting the zero crossing rate Jcll from the preprocessed audio by the feature extraction module is as follows: In the formula, Jcll represents the zero crossing rate, L represents the total number of signal samples, δ(p(t)*p(t-1) represents the unit cross-order function, and t represents the time index.

6. The audio content security detection system based on deep learning according to claim 5, characterized in that: The formula for extracting the signal amplitude Xhfd from the preprocessed audio by the feature extraction module is as follows: In the formula, Xhfd represents the signal amplitude, and X(a) represents the audio signal.

7. The audio content security detection system based on deep learning according to claim 6 is characterized in that: The formula for extracting the audio feature variance Ypfc from the preprocessed audio by the feature extraction module is as follows: In the formula, Ypfc 2 represents the characteristic variance, L represents the total number of signal samples, x[i] represents the eigenvalue of the i-th sample, and μ represents the mean of the sample characteristics.

8. The audio content security detection system based on deep learning according to claim 7, characterized in that: The result output module calculates the unsafe sample rate Baql according to the total number of detected samples, and the calculation formula is as follows: In the formula, Baql represents the unsafe sample rate, U represents the number of samples marked as unsafe, and L represents the total number of signal samples.

9. The audio content security detection system based on deep learning according to claim 8, characterized in that: The result output module calculates the false alarm rate Wblz according to the total number of detected samples. The calculation formula is as follows: In the formula, Wblz represents the false alarm rate, FP represents the number of false alarm samples, and TN represents the number of truly safe samples.

10. The audio content security detection system based on deep learning according to claim 9, characterized in that: The result output module calculates the false negative rate Lblz according to the total number of detected samples. The calculation formula is as follows: In the formula, Lblz represents the false negative rate, FN represents the number of false negative samples, and TP represents the number of truly unsafe samples.

Images