A noise monitoring digital audio tamper detection method based on power grid frequency signal

Abstract

The application discloses a noise monitoring digital audio tampering detection method based on a power grid frequency signal, and steps include: obtaining a noise monitoring digital audio and extracting a power grid frequency signal; signal pretreatment is performed on the extracted power grid frequency signal; the pretreated power grid frequency signal is input into a trained deep learning model to obtain the two-classification probability of each data in the power grid frequency signal being normal or tampered; the model output is normalized to calculate the tampering probability of each data in the power grid frequency signal, compared with a preset threshold, and a tampering mark is set to obtain a prediction sequence; the prediction sequence is subjected to connected domain analysis to identify continuous tampering segments, and the sample index of the tampering segments is mapped back to the timestamp of the original noise monitoring digital audio to output a corresponding detection report. The application automatically learns a tampering feature mode through a deep neural network, realizes point-by-point labeling of a tampering position, and realizes high-precision tampering detection while keeping lightweight.

Description

Technical Field

[0001] This invention relates to audio tampering detection technology, specifically to a method for detecting digital audio tampering based on power grid frequency signals and noise monitoring. Background Technology

[0002] Power grid frequency signals, as environmental features passively embedded during audio recording, possess spatiotemporal uniqueness and are difficult to forge, thus serving as a "fingerprint" for digital audio authentication. Existing audio tampering detection methods based on power grid frequency signals mainly include: I. Tampering Detection Methods Based on Signal Matching. This type of method extracts the power grid frequency signal from the audio being tested and compares it with a real-world reference power grid frequency signal. The continuity and consistency of the match determine whether tampering has occurred. Typical schemes include: extracting the power grid frequency signal using short-time Fourier transform and extremum search methods, then performing point-by-point comparisons using Pearson similarity and Euclidean distance; any abrupt changes are identified as tampering points. Another scheme uses a dynamic matching algorithm, setting a frequency resolution threshold, correcting the power grid frequency with a correction signal, calculating the minimum mean square error to determine the optimal matching position, and thus inferring the timestamp information and tampering status of the audio. This method has high computational complexity and low processing efficiency. In practical applications, the window size, filter parameters, and threshold settings of the short-time Fourier transform need to be manually adjusted, resulting in insufficient automation. The accuracy of this method in locating tampering boundaries depends on the selection of signal processing parameters. When facing complex tampering types, such as multiple short-time tamperings or small-span segment replacements, it is prone to missed detections or inaccurate location. Furthermore, the quality of the power grid frequency signal extracted by this method decreases under low signal-to-noise ratio environments, leading to reduced matching accuracy and requiring improvement in robustness.

[0003] II. Location Method Based on Power Grid Frequency Reference Signal Network. This scheme extracts audio from online real-time multimedia data from multiple regions and performs downsampling processing. It detects the presence of power grid frequencies in the audio, fuses the power grid frequency signals of multiple audio recordings from the same region to obtain a reference signal for a single region, and then constructs a power grid frequency reference signal network covering multiple regions. By calculating the correlation coefficient between the power grid frequency signal of the audio to be located and each node of the reference network, the geographical location and recording time of the audio can be determined, and the authenticity of the audio can be judged accordingly. This method requires the collection and processing of a large amount of multimedia data from different regions and time periods, resulting in high data acquisition and maintenance costs. Furthermore, the coverage and temporal resolution of the reference signal network are limited. This method is mainly used for spatiotemporal location and macroscopic authenticity verification of audio, but it is insufficient for accurately locating the start and end times of tampered areas and cannot achieve point-by-point tamper detection. In actual environmental monitoring scenarios, law enforcement officers are more concerned with which specific time period the data was tampered with, rather than simply judging whether the audio is authentic as a whole. Therefore, the applicability of this scheme in noise monitoring data review is limited.

[0004] III. A Deep Learning-Based Power Grid Frequency Enhancement Method. This scheme addresses the difficulty of extracting power grid frequency signals under low signal-to-noise ratio conditions by employing a generative adversarial network (GAN) to enhance the extracted power grid frequency signals. The generator uses an encoder-decoder structure, employing multiple one-dimensional convolutional layers to extract signal features. A discriminator distinguishes the enhanced signal from the real signal, and adversarial training improves the quality of the power grid frequency signal, thereby enhancing the accuracy of subsequent forensic applications. However, it does not directly perform tamper detection; it still needs to be used in conjunction with other detection algorithms. Training the GAN requires a large amount of labeled data and computational resources, making the training process complex and with a slow convergence speed. The enhancement effect of this scheme depends on the network structure design and the representativeness of the training data; the model's generalization ability may be affected by different recording devices and environmental noise conditions. Furthermore, this scheme focuses on the signal preprocessing stage and lacks sufficient automatic learning and recognition capabilities for tamper features, failing to achieve end-to-end tamper detection.

[0005] In summary, existing technical solutions generally suffer from the following shortcomings when dealing with the problem of detecting digital audio tampering in noise monitoring: the detection process relies on manual parameter adjustment and feature engineering, resulting in low automation; the accuracy of tamper boundary localization is not high, making it difficult to achieve precise point-by-point annotation; the detection capability for complex tampering types and minor tampering is limited; robustness is insufficient under low signal-to-noise ratio conditions; and computational efficiency is low, making it difficult to meet the real-time requirements of large-scale data auditing. These shortcomings limit the practical application effectiveness of existing technologies. Summary of the Invention

[0006] The technical problem to be solved by this invention is as follows: Addressing the aforementioned problems in existing technologies, this invention provides a noise monitoring method for digital audio tampering detection based on power grid frequency signals. By introducing image segmentation concepts from deep learning, the tampering detection problem of power grid frequency signals is transformed into a one-dimensional time series segmentation task. Employing a lightweight U-shaped visual Transformer architecture and combining a local-global-local attention mechanism, it achieves automatic and accurate labeling of tampering locations, overcoming the shortcomings of existing technologies and exhibiting significant advantages in detection accuracy, automation, computational efficiency, and robustness.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for detecting digital audio tampering based on power grid frequency signals during noise monitoring includes the following steps: Acquire digital audio from noise monitoring and extract power grid frequency signals; The extracted power grid frequency signal is preprocessed. The preprocessed power grid frequency signal is input into a trained deep learning model. The deep learning model first uses an input embedding layer to map the power grid frequency signal to a multi-dimensional feature space. Then, it uses cascaded encoders to progressively extract features at different time scales and downsample them. Next, it uses a bottleneck layer with local-global-local attention and Transformer attention mechanisms to model local and global dependencies. Then, it uses cascaded decoders to progressively upsample the output features of the bottleneck layer to gradually restore the signal resolution. At the same time, it fuses the skip connection features of the corresponding encoders. Finally, it uses an output layer to map the output features of the last stage decoder into binary classification probabilities of normal and tampered classes. The model output is normalized to calculate the tampering probability of each data in the power grid frequency signal. The tampering probability of each data is compared with a preset threshold and a tampering flag is set to obtain the prediction sequence. Connectivity analysis is performed on the predicted sequence to identify continuous tampered segments. The sample index of the tampered segment is mapped back to the timestamp of the original noise monitoring digital audio, and the corresponding detection report is output.

[0008] Furthermore, when using the input embedding layer to map the power grid frequency signal to a multidimensional feature space, the power grid frequency signal is processed sequentially through one-dimensional convolution, batch normalization, and activation functions, thereby mapping the single-channel power grid frequency signal to a multidimensional feature space.

[0009] Furthermore, when using cascaded encoders to progressively extract features at different time scales and perform downsampling, the specific steps include: The first-level encoder obtains the output features of the input embedding layer, and after processing by the ConvUtr1D block, the corresponding output features are obtained. The second-stage encoder acquires the output features of the first-stage encoder, and then processes them sequentially through the ConvUtr1D block and downsampling to obtain the corresponding output features. The third-stage encoder acquires the output features of the second-stage encoder, and after sequential processing by cascaded ConvUtr1D blocks and downsampling, obtains the corresponding output features, which are then input into the bottleneck layer for processing.

[0010] Furthermore, the ConvUtr1D block includes cascaded depthwise convolutional branches and inverted bottleneck branches, wherein: The depthwise convolutional branch extracts features from the local receptive field, and its mathematical expression is as follows:

[0011] in, The output features of the depthwise convolution branch, The input features are the depthwise convolutional branch, DWConv represents depthwise separable convolution, k is the kernel size, and GELU is the Gaussian error linear unit activation function. The inverted bottleneck branch first expands the number of channels and then compresses them to enhance feature representation capabilities. The mathematical expression is as follows:

[0012] in, The output characteristics of the inverted bottleneck branch, Indicates batch normalization, and For different 1×1 convolutions, C represents the number of input channels, and the superscript indicates the channel number transformation relationship. This means increasing the number of channels by 4. This means compressing the number of channels back to 1 / 4 of the original number and then restoring it.

[0013] Furthermore, by using local-global-local attention mechanisms and Transformer attention mechanisms in the bottleneck layer, when modeling local and global dependencies, the output features of the last encoder are input into the cascaded LGL attention blocks for processing. Each LGL attention block first uses a local aggregation module to capture local dependencies and then uses a global sparse attention module to model global dependencies. Then, the processing results of the cascaded LGL attention blocks are input into the cascaded Transformer blocks for processing to model global long-distance dependencies and understand the temporal consistency of the entire audio sequence.

[0014] Furthermore, when using the local aggregation module to capture local dependencies, a three-stage gating modulation is employed to enhance the expressive power of local features, as shown in the following mathematical expression:

[0015]

[0016]

[0017] in, These are the input features of the local aggregation module. These are the output features of the local aggregation module. This represents element-wise multiplication. This indicates a position encoding operation. Represents a 1×1 convolution. This indicates a depthwise separable convolution with a kernel size of 9. Indicates batch normalization, It is the Sigmoid activation function. For position encoding.

[0018] Furthermore, when modeling global dependencies using a global sparse attention module, a spatial reduction attention mechanism is first employed to spatially reduce the input features before multi-head self-attention computation. The reduced attention result is then restored to its original resolution via linear interpolation, followed by a nonlinear transformation using a multilayer perceptron. The mathematical expression is as follows:

[0019]

[0020]

[0021] in, These are the input features of the global sparse attention module. These are the output features of the global sparse attention module. This indicates a position encoding operation. Representation space reduction attention mechanism, This represents a multilayer perceptron. Representation layer normalization.

[0022] Furthermore, the cascaded decoders and cascaded encoders are one-to-one and both are three-stage. The signal resolution is gradually restored by upsampling the output features of the bottleneck layer through the cascaded decoders. At the same time, when fusing the skip connection features of the corresponding encoders, the mathematical expression is as follows:

[0023]

[0024]

[0025] in, This represents the output features of the first-level decoder. This represents the output characteristics of the bottleneck layer. This represents the skip connection feature after downsampling of the third-level encoder. This represents the output features of the second-level decoder. This represents the skip connection feature after downsampling in the second-stage encoder. This represents the output characteristics of the third-level encoder. This represents the skip connection feature after downsampling from the first-level encoder. The term "upsampled convolutional block" indicates an upsampled convolutional block, "Concat" indicates a channel concatenation operation, "Up" indicates upsampling, and "Down" indicates downsampling. Specifically, the downsampled skip connection features are the features obtained by max pooling the skip connection features of the corresponding encoder.

[0026] Furthermore, when the output features of the last-stage decoder are mapped to binary classification probabilities of the normal class and the tampered class through the output layer, the mathematical expression is as follows:

[0027] in, This represents the input features of the output layer. , This represents a 1×1 convolution, and the output is... Channel 0 represents the probability of a normal sample, channel 1 represents the probability of a tampered sample, B is the batch size, and L is the signal length.

[0028] Furthermore, the mathematical expression for the loss function during the training of the deep learning model is as follows:

[0029] in, , These are weight hyperparameters; The mathematical expression for cross-entropy loss is as follows:

[0030] in, For the true labels of i data points in the power grid frequency signal, Let c be the predicted probability of i data points in the power grid frequency signal, where c = 0 represents a normal sample and c = 1 represents a tampered sample, and L is the signal length. The mathematical expression for Dice loss is as follows:

[0031] in, Let i data points in the power grid frequency signal be the true labels for those belonging to the tampering category. Let be the predicted probability that i data points in the power grid frequency signal belong to the tampering class. To prevent smoothing terms from being divided by zero.

[0032] Compared with the prior art, the advantages of the present invention are as follows: This invention utilizes a deep learning model to automatically learn tampering feature patterns, exhibiting significant advantages in detection accuracy, automation, computational efficiency, and robustness. Regarding detection accuracy, the model employs a local-global-local attention mechanism to address the low information density and high semantic differences in power grid frequency signals. Combined with a Transformer attention mechanism to model global long-range dependencies, it understands the temporal consistency of the entire audio sequence, achieving precise point-by-point labeling of tampering locations. In terms of automation, the entire process from signal extraction, preprocessing, model inference to tamper location can be completed automatically without manual parameter adjustment. Regarding computational efficiency, the cascaded design of the input embedding layer, encoder, bottleneck layer, and decoder, along with skip connection feature fusion, achieves efficient inference while maintaining a lightweight architecture. In terms of robustness, multi-scale feature extraction and dependency modeling at different time scales enable the model to adapt to complex tampering detection under low signal-to-noise ratio conditions. Attached Figure Description

[0033] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0034] Figure 2 This is an overall architecture diagram of the Mobile U-ViT 1D model in an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0036] Example 1 like Figure 1 As shown, this embodiment proposes a digital audio tampering detection method based on power grid frequency signals for noise monitoring. The method transforms the digital audio tampering detection problem into a segmentation task of the power grid frequency signal time series. A deep neural network automatically learns tampering feature patterns to achieve point-by-point labeling of tampering locations. The core technical approach employs a lightweight U-shaped visual Transformer one-dimensional version (Mobile U-ViT 1D) neural network model. This model draws inspiration from medical image segmentation techniques, innovatively transferring two-dimensional image segmentation methods to the field of one-dimensional time-series signal processing. The model adopts a U-shaped architecture of encoder-bottleneck layer-decoder, combined with a local-global-local attention mechanism, achieving high-precision tampering detection while maintaining lightweight design.

[0037] like Figure 1As shown, the method includes the following steps: S1: Audio input to be tested. Specifically, this is obtained from the ecological environment monitoring center or a third-party environmental protection service organization for noise monitoring.

[0038] S2: Power Grid Frequency Signal Extraction. Specifically, this involves extracting the power grid frequency signal sequence from the digital audio file used for noise monitoring. A short-time Fourier transform (SFT) method is employed for time-frequency analysis of the audio signal. A bandpass filter (49-51 Hz) is designed (for my country's nominal power grid frequency of 50 Hz). Spectral peaks are detected within each time window to obtain the time-varying sequence of the power grid frequency, which is then resampled to 10 Hz. Mathematically, this is expressed as:

[0039] Where s(t) is the audio signal, STFT is the short-time Fourier transform, and ENF(t) is the extracted power grid frequency signal.

[0040] S3: Signal Preprocessing. Specifically, the extracted power grid frequency signal undergoes signal preprocessing through normalization and window segmentation. First, the Z-score normalization method is used to eliminate the influence of signal dimensions.

[0041] in, and These are the mean and standard deviation of the signal, respectively.

[0042] The long sequence is then divided into fixed-length windows, with each window having 600 sampling points (corresponding to 60 seconds of audio). The windows have a 50% overlap, and multiple windows are assembled into batches for model input.

[0043] S4: Deep Learning Model Inference. The preprocessed power grid frequency signal is input into the trained Mobile U-ViT1D model to obtain point-by-point tamper probability predictions. The model input is... Where B is the batch size and L is the signal length; the model output is Channel 0 represents the probability of a normal sample, and channel 1 represents the probability of a tampered sample.

[0044] The Mobile U-ViT 1D model in this embodiment adopts a U-shaped encoder-decoder architecture, such as... Figure 2 As shown in the diagram, dashed arrows represent jump connections. The Mobile U-ViT 1D model consists of the following parts: (1) Input embedding layer (marked as 1): Receives the power grid frequency signal and maps the power grid frequency signal to a multi-dimensional feature space; (2) Encoder (labeled as 2-1, 2-2, 2-3): A three-layer hierarchical structure is adopted, and the cascaded encoders gradually extract features at different time scales and perform downsampling; (3) Bottleneck layer (labeled as 3-1, 3-2): A two-layer design is adopted. The bottleneck layer uses local-global-local attention mechanism and Transformer attention mechanism to model local and global dependencies; (4) Decoder (labeled as 4-1, 4-2, 4-3): It adopts a three-layer structure. The cascaded decoders upsample the output features of the bottleneck layer in turn to gradually restore the signal resolution, while fusing the skip connection features of the corresponding encoder (labeled as 5-1, 5-2, 5-3). (5) Output layer (labeled as 6): The output layer maps the output features of the last-stage decoder to binary classification probabilities of normal class and tampered class through 1×1 convolution, thereby generating a tampering probability map.

[0045] S5: Post-processing. First, the model output is normalized using Softmax, and the tampering probability of each data point in the power grid frequency signal is calculated:

[0046] in, Let be the predicted probability that i data points in the power grid frequency signal belong to the tampering class. Let be the predicted probability that i data points in the power grid frequency signal belong to the normal class.

[0047] Then, the tampering probability of each data point is compared with a preset threshold, and a tampering flag is set:

[0048] Where the threshold The value is set to 0.5. Finally, a median filter with a window size of 5 is applied to eliminate isolated false positives, thus obtaining the predicted sequence.

[0049] S6: Tampered Region Location. Connectivity analysis is performed on the predicted sequence to identify continuous tampered segments. Short, falsely detected segments shorter than 0.5 seconds are filtered out. The sample index of the tampered segment is mapped back to the timestamp of the original noise monitoring digital audio. The start and end times of the tampered region are calculated as follows:

[0050] in and These are the start and end sample indices of the tampered region, respectively, and fs = 10Hz is the sampling rate of the power grid frequency signal.

[0051] S7: Output Detection Report. Generate and output a detection report that includes the start and end times of the tampered area, average confidence level, tampering type judgment, and visualization charts.

[0052] In this embodiment, the hierarchical feature extraction table of the Mobile U-ViT 1D model is shown in Table 1. The following section, in conjunction with the appendix... Figure 2 Table 1 provides a detailed explanation of the structure of each layer of the Mobile U-ViT 1D neural network model.

[0053]

[0054] In this embodiment, the input embedding layer (Stem) has 1 input channel, 16 output channels, a kernel size of 3, a stride of 1, padding of 1, and the activation function is ReLU. Batch normalization is performed before the activation function. When using the input embedding layer to map the power grid frequency signal to a multi-dimensional feature space, the power grid frequency signal is processed sequentially through one-dimensional convolution, batch normalization, and the activation function, thereby using a 3×3 one-dimensional convolution to map the single-channel power grid frequency signal to a 16-dimensional feature space.

[0055] in, Indicates batch normalization, This is the activation function.

[0056] like Figure 2 As shown, in this embodiment, the encoder adopts a multi-scale hierarchical structure, containing three ConvUtr1D layers that are downsampled layer by layer: Encoder Layer 0: As shown in Table 1, it has 16 input channels and 16 output channels, contains one ConvUtr1D block, has a depthwise separable convolution kernel size of 3, and does not perform downsampling. The inverted bottleneck branch of the ConvUtr1D block has an expansion factor of 4, meaning it has 64 intermediate channels.

[0057] Encoder Layer 1: As shown in Table 1, it has 16 input channels and 32 output channels. It contains one ConvUtr1D block, a depthwise separable convolution kernel of size 3, and performs 2x downsampling through convolution with a stride of 2.

[0058] Encoder Layer 2: As shown in Table 1, it has 32 input channels and 64 output channels. It contains 3 cascaded ConvUtr1D blocks, with a depthwise separable convolution kernel size of 7, and performs 2x downsampling through convolution with a stride of 2.

[0059] Therefore, when using a cascaded encoder to progressively extract features at different time scales and perform downsampling, the specific steps include: The first-level encoder Layer0 obtains the output features of the input embedding layer Stem, and then processes them through the ConvUtr1D block to obtain the corresponding output features; The second-stage encoder Layer1 obtains the output features of the first-stage encoder Layer0, and then processes them sequentially through the ConvUtr1D block and downsampling to obtain the corresponding output features. The third-level encoder Layer2 acquires the output features of the second-level encoder Layer1, and after being processed by cascaded ConvUtr1D blocks and downsampling, the corresponding output features are obtained and input into the bottleneck layer for processing.

[0060] In this embodiment, the ConvUtr1D block is the basic building block of the encoder, employing a design combining depthwise separable convolution and an inverted bottleneck structure. Each ConvUtr1D block includes two cascaded residual branches—a depthwise convolution branch and an inverted bottleneck branch, wherein: The depthwise convolutional branch extracts features from the local receptive field, and its mathematical expression is as follows:

[0061] in, The output features of the depthwise convolution branch, The input features are the depthwise convolutional branch, DWConv represents depthwise separable convolution, k is the kernel size, and GELU is the Gaussian error linear unit activation function. The inverted bottleneck branch first expands the number of channels and then compresses them to enhance feature representation capabilities. The mathematical expression is as follows:

[0062] in, The output characteristics of the inverted bottleneck branch, Indicates batch normalization, and For different 1×1 convolutions, C represents the number of input channels, and the superscript indicates the channel number transformation relationship. This means increasing the number of channels by 4. This means compressing the number of channels back to 1 / 4 of the original number and then restoring it.

[0063] With the above structure, in the three encoders, Layer 0 uses a convolution with a kernel size of 3 to maintain a resolution of L; Layer 1 uses a convolution with a kernel size of 3, downsampling the resolution to L / 2; and Layer 2 uses a convolution with a kernel size of 7, downsampling the resolution to L / 4. The deep use of large convolution kernels enables the capture of tampering transition patterns in the power grid frequency signal that span a long time span.

[0064] like Figure 2As shown, the bottleneck layer in this embodiment is the core innovative part of the model, employing a two-layer design to process semantic information at different levels. Its structure includes: Phase 1: LGL Bottleneck Layer In this embodiment, the first stage of the bottleneck layer (LGL Bottleneck) is shown in Table 1. The number of input and output channels is 64, and it contains 3 LGL (local-global-local) attention blocks cascaded together.

[0065] The Local-Global-Local Attention Block is specifically designed to address the low information density and high semantic discrepancies of power grid frequency signals. This module comprises two key components: local aggregation and global sparse attention. The local aggregation module enhances the expressive power of local features through a gating mechanism, thereby capturing local dependencies. This module employs triple-gating modulation: The first time the gating is used, position coding is used:

[0066] The second gating uses depthwise convolution:

[0067] The third gate uses a channel-level MLP:

[0068] in, These are the input features of the local aggregation module. These are the output features of the local aggregation module. This indicates a position encoding operation. This represents element-wise multiplication. It is the Sigmoid activation function. For position encoding.

[0069] In this embodiment, the location encoding of the local aggregation module uses a depthwise convolution with a kernel size of 9, and the gating mechanism has an offset of -0.5. The -0.5 operation adjusts the gating range from [0, 1] to [-0.5, 0.5], achieving bidirectional modulation of the features. Using a depthwise convolution with a kernel size of 9 can capture a relatively long range of local dependencies in the power grid frequency signal, which is important for detecting transition regions at tamper boundaries.

[0070] To model long-distance dependencies while maintaining computational efficiency, the global sparse attention module employs a space-reduced attention mechanism.

[0071] First, reduce the input space:

[0072] Where r is the reduction ratio, set r=2.

[0073] Then, multi-head self-attention calculation is performed. For the query vector Q and reduced key-value pairs K , V :

[0074]

[0075] in For each dimension of attention head, , , This is a learnable weight matrix.

[0076] Finally, linear interpolation is used to restore the reduced attention result to the original resolution:

[0077] The global sparse attention module finally performs a nonlinear transformation on the result of the spatially reduced attention mechanism using a multilayer perceptron:

[0078] The dimension of the hidden layer is 4 times that of the input dimension.

[0079] The complete calculation process of the global sparse attention module is as follows:

[0080]

[0081]

[0082] in, These are the input features of the global sparse attention module. These are the output features of the global sparse attention module. Representation layer normalization, Representation space reduction attention mechanism, This represents a multilayer perceptron. The global sparse attention module in this embodiment uses four attention heads, each with a dimension of 16 (total dimension 64), a space reduction ratio of 2, and linear interpolation as the upsampling method. The MLP hidden layer has a dimension of 256 (4 times the input dimension).

[0083] Phase Two: Transformer Bottleneck Layer In this embodiment, the second stage of the bottleneck layer (Transformer Bottleneck) uses the standard Transformer structure to model global long-range dependencies and understand the temporal consistency of the entire audio sequence. This stage contains three Transformer blocks, as shown in Table 1, with 128 channels and a spatial reduction ratio of 1 (i.e., global attention). First, the number of channels is expanded from 64 to 128 using 1×1 convolutions, followed by a cascade of three Transformer blocks. Each Transformer block uses eight attention heads, each with a dimension of 16 (total dimension 128), and a spatial reduction ratio of 1 (no reduction). The MLP hidden layer dimension is 512. Finally, the number of channels is compressed back from 128 to 64 using 1×1 convolutions.

[0084] Therefore, in this embodiment, when modeling local and global dependencies using the local-global-local attention mechanism and the Transformer attention mechanism at the bottleneck layer, the output features of the last encoder are specifically input into the cascaded LGL attention blocks for processing. Each LGL attention block first uses a local aggregation module to capture local dependencies and then uses a global sparse attention module to model global dependencies. The processing results of the cascaded LGL attention blocks are then input into the cascaded Transformer blocks to model global long-range dependencies and understand the temporal consistency of the entire audio sequence. The dimensionality transformation process is as follows:

[0085]

[0086]

[0087]

[0088] in, This represents the input features of the bottleneck layer, i.e., the output features of the last encoder layer, Layer 2. This represents the input features of the bottleneck layer. This indicates a stack of 3 LGL blocks. This represents a stack of 3 Transformer blocks. The two-layer bottleneck design allows the model to optimize the extraction of local and global information separately in different feature spaces.

[0089] In this embodiment, the decoder adopts the skip connection concept of the U-Net architecture, fusing multi-scale features from the encoder into the decoding process to achieve accurate point-by-point segmentation. For example... Figure 2 As shown, the cascaded decoders and cascaded encoders correspond one-to-one and are both three-stage, including: The first-level decoder, Layer 3, as shown in Table 1, has 64+64=128 input channels (64 channels from the bottleneck layer output + 64 channels after downsampling from the skip connections of the encoder Layer 2), and 64 output channels. It contains one upsampling convolution block with an upsampling factor of 2, implemented using transposed convolution.

[0090] The second-level decoder, Layer 2, as shown in Table 1, has 64+32=96 input channels (64 output channels from the previous layer + 32 channels after downsampling from the skip connection of the encoder Layer 1) and 32 output channels, including one upsampled convolutional block with an upsampling factor of 2.

[0091] The third-level decoder, Layer 1, as shown in Table 1, has 32+16=48 input channels (32 output channels from the previous layer + 16 channels after downsampling from the encoder Layer 0 jump connection) and 16 output channels, including one upsampled convolutional block with an upsampling factor of 1 (no upsampling).

[0092] This embodiment employs an innovative downsampling skip connection strategy. Unlike the direct connections of traditional U-Net, additional downsampling processing is applied to the skip connection features of each encoder. In this embodiment, the downsampling operation for skip connections uses max pooling with a kernel size of 2 and a stride of 2.

[0093] This design has two advantages: first, it reduces the size difference of feature maps in the decoder, thus reducing the amount of computation; second, it performs spatial abstraction on skip connection features, preserving the main structural information and reducing noise interference.

[0094] Therefore, when the output features of the bottleneck layer are upsampled sequentially by cascaded decoders to gradually restore the signal resolution, and the skip connection features of the corresponding encoder are fused, the mathematical expression is as follows:

[0095]

[0096]

[0097] in, This represents the output features of the first-level decoder. This represents the output characteristics of the bottleneck layer. This represents the skip connection feature after downsampling of the third-level encoder. This represents the output features of the second-level decoder. This represents the skip connection feature after downsampling in the second-stage encoder. This represents the output characteristics of the third-level encoder. This represents the skip connection feature after downsampling from the first-level encoder. The term "upsampled convolutional block" indicates an upsampled convolutional block, "Concat" indicates a channel concatenation operation, "Up" indicates upsampling, and "Down" indicates downsampling. Specifically, the downsampled skip connection features are the features obtained by max pooling the skip connection features of the corresponding encoder.

[0098] As shown in Table 1, the output layer in this embodiment has 16 input channels and 2 output channels (normal class and tampered class), using 1×1 convolutions without activation functions. In this embodiment, batch normalization and GELU activation functions are applied after all convolutional layers outside the model's output layer. When the output features of the last-stage decoder are mapped to the binary classification probabilities of the normal and tampered classes through the output layer, the mathematical expression is as follows:

[0099]

[0100] in, This represents the input features of the output layer. This represents a 1×1 convolution, and the output is... Channel 0 represents the probability of a normal sample, channel 1 represents the probability of a tampered sample, B is the batch size, and L is the signal length.

[0101] Through the above design, the model has a total of 992,690 parameters and a computational complexity of 0.8 GFLOPs (with an input length of 600). Furthermore, the Mobile U-ViT 1D model utilizes various techniques to achieve lightweight design, making it suitable for resource-constrained devices. The main lightweighting techniques include: 1. Depthwise Separable Convolution: Decomposes standard convolution into depthwise convolution and pointwise convolution. The number of parameters in standard convolution is... The number of parameters for depthwise separable convolution is Compression ratio is .when When the value is large, the number of parameters decreases significantly.

[0102] 2. Sparse attention: The computational complexity of attention is reduced by 50% by a space reduction ratio r=2.

[0103] 3. Channel number optimization: A smaller base channel number configuration (16, 32, 64, 128) is adopted, which significantly reduces the number of parameters compared to common deep learning models (which usually use 64, 128, 256, 512).

[0104] The performance comparison table of the models is shown in Table 2. As can be seen from Table 2, the Mobile U-ViT 1D model has less than 1M parameters, which is only 1 / 9 to 1 / 26 of the comparison models. It also has significant advantages in terms of computational load and inference speed.

[0105]

[0106] In this embodiment, the Mobile U-ViT 1D model employs a combined loss function during training, simultaneously optimizing pixel-level accuracy and class balance. The loss function is a weighted combination of cross-entropy loss and Dice loss, expressed mathematically as follows:

[0107] in, , The weight hyperparameter is set to [value] based on experiments. =1.0, =1.0; The mathematical expression for cross-entropy loss is as follows:

[0108] in, For the true labels of i data points in the power grid frequency signal, Let c be the predicted probability of i data points in the power grid frequency signal, c = 0 for normal samples and c = 1 for tampered samples, and L be the signal length. In this embodiment, the cross-entropy loss uses the torch.nn.CrossEntropyLoss class of PyTorch, with weights set to [1.0, 2.5], and a higher weight is given to the tampered class to balance the class imbalance. The mathematical expression for Dice loss is as follows:

[0109] in, Let i data points in the power grid frequency signal be the true labels for those belonging to the tampering category. Let be the predicted probability that i data points in the power grid frequency signal belong to the tampering class. To prevent smoothing terms from being divided by zero.

[0110] In this embodiment, the training strategy for the Mobile U-ViT 1D model includes: using the AdamW optimizer with an initial learning rate of 1e-4 and a cosine annealing scheduling strategy; data augmentation including random pruning, time stretching, and Gaussian noise injection; and using DropPath regularization with a dropout probability of 0.1 to prevent overfitting.

[0111] The following experiment verifies the effectiveness of the Mobile U-ViT 1D model: Hardware environment: The workstation is equipped with an NVIDIA GeForce RTX 3090 graphics card (24GB VRAM), an Intel Xeon Gold 6248R processor (48 cores), and 128GB of DDR4 memory. The operating system is Ubuntu 20.04 LTS 64-bit.

[0112] Software environment: Python version 3.8.10, deep learning framework PyTorch 1.12.0, CUDA version 11.3, and cuDNN version 8.3.2. Other dependencies include NumPy 1.21.2 for numerical computation, SciPy 1.7.3 for signal processing, librosa 0.9.2 for audio processing, scikit-learn 1.0.2 for data preprocessing, and matplotlib 3.4.3 for result visualization.

[0113] First, prepare the data, including: 1. Audio Data Acquisition: A total of 2,000 digital audio data segments for environmental noise monitoring were obtained from a provincial ecological and environmental monitoring center. The sampling rate was 48kHz, mono, WAV format, and each segment ranged in length from 120 to 300 seconds. These audio recordings were made by a third-party environmental service agency during noise monitoring at the boundaries of industrial enterprises and included noise samples under various operating conditions.

[0114] 2. Data Tampering Generation: 1600 audio segments were randomly selected from 2000 original audio segments for the training set, 200 for the validation set, and 200 for the test set. The audio segments in the training and validation sets were manually tampered with to generate tampered samples. Tampering types included: (1) Segment deletion: randomly deleting consecutive segments of 5-30 seconds; (2) Segment insertion: randomly selecting segments of 5-30 seconds from other audio segments and inserting them into the current audio; (3) Segment replacement: randomly selecting segments of 10-40 seconds and replacing them with segments of the same length from other audio segments; (4) Multi-segment tampering: performing a combination of the above operations 2-4 times in the same audio segment. The sample proportions for each tampering type were 30%, 25%, 25%, and 20%, respectively. The tampering operations were performed using Adobe Audition CC 2021 audio editing software.

[0115] 3. Power Grid Frequency Signal Extraction: Power grid frequency signals are extracted from all audio files. The specific steps are as follows: (1) The audio is subjected to a short-time Fourier transform using the scipy.signal.stft function. The window type is a Hanning window with a window length of 4096 sampling points and a window overlap rate of 75%. (2) A bandpass filter of 49-51Hz is designed. The filter type is a 4th-order Butterworth filter. The scipy.signal.butter and scipy.signal.filtfilt functions are used to implement the filter. (3) The frequency corresponding to the maximum amplitude in the 49-51Hz range is searched in the spectrum of each time window and used as the power grid frequency estimate at that moment. (4) The extracted power grid frequency signal is resampled from the original sampling rate to 10Hz. The resampling is performed using the scipy.signal.resample function. The extracted power grid frequency signal is saved as a .npy format file.

[0116] 4. Tag Creation: For tampered audio, create point-by-point tags based on the start and end times of the tampering. The tags are one-dimensional arrays of the same length as the power grid frequency signal; normal sample positions are marked as 0, and tampered sample positions are marked as 1. Considering the transition region at the tampering boundary, positions ±2 sampling points (corresponding to ±0.2 seconds) before and after the tampering start and end points are also marked as 1. The tag file is saved in .npy format.

[0117] 5. Data Preprocessing: Z-score normalization was performed on the power grid frequency signals in the training set. The validation and test sets were normalized using the statistics from the training set. The normalized signals were then segmented into fixed-length windows of 600 sampling points (corresponding to 60 seconds of audio) with a window step of 300 sampling points (50% overlap). Each window and its corresponding label constituted a training sample. A total of 32,480 samples were generated for the training set, of which 38.5% were tampered with; 4,160 samples were generated for the validation set; the test set retained the complete audio without segmentation and was used to test the complete detection process.

[0118] Then, model training is performed, including: 1. Training parameter settings: Batch size is set to 32, and the number of training epochs is 100. The optimizer used is AdamW, with an initial learning rate of 1e-4 and a weight decay coefficient of 0.01. The learning rate scheduling strategy uses cosine annealing, with a minimum learning rate of 1e-6 and a restart cycle of 100 epochs (without restarting).

[0119] 2. Data Augmentation Strategies: During the training phase, data augmentation is applied to each sample with a 50% probability. Augmentation operations include: (1) Random cropping: with a 30% probability, a 600-point window is randomly cropped to 540-600 points, and then restored to 600 points through linear interpolation; (2) Time stretching: with a 30% probability, the signal is stretched or compressed by 0.95-1.05 times on the time axis, and then restored to 600 points through resampling; (3) Gaussian noise injection: with a 40% probability, Gaussian noise with a standard deviation of 0.05 is added; (4) Amplitude scaling: with a 30% probability, the signal amplitude is scaled by 0.9-1.1 times. Each augmentation operation is independent of the others and can be combined.

[0120] 3. Regularization Techniques: DropPath regularization is employed, with the dropout probability set to 0.1 in both the encoder and decoder, and 0.15 in the bottleneck layer. Additionally, Dropout is used in the LGL attention block and Transformer block, with a dropout rate of 0.1. The momentum parameter for batch normalization is set to 0.1, and epsilon is set to 1e-5.

[0121] 4. Training Process: Single-card training was used, with each training round lasting approximately 15 minutes. During training, model performance was evaluated on the validation set after each round, recording the loss, accuracy, precision, recall, and F1 score. An early stopping strategy was employed, monitoring the validation set F1 score; training was stopped if there was no improvement after 20 consecutive rounds. The model reached its optimal validation performance in the 68th round, with an F1 score of 95.8%, corresponding to a training loss of 0.142 and a validation loss of 0.168. The model parameters for this round were saved as the final model.

[0122] 5. Training Results: The final model's performance metrics on the validation set are: accuracy 96.3%, precision 94.7%, recall 97.0%, and F1 score 95.8%. The average error for tampered boundary localization is ±0.12 seconds. The total training time for the model was approximately 17 hours.

[0123] Finally, application testing is performed, including: 1. Preparation of the audio to be tested: A noise monitoring audio segment submitted by a company at the factory boundary was selected from the test set. The file name is "test_audio_001.wav", the duration is 180 seconds, and the sampling rate is 48kHz. The audio was found to be abnormal upon manual inspection and is suspected of being tampered with.

[0124] 2. Power Grid Frequency Signal Extraction: The power grid frequency signal is extracted using the same method as in the training phase. A short-time Fourier transform is performed on the audio, with a window length of 4096, an overlap rate of 75%, a bandpass filter range of 49-51Hz, and resampling to 10Hz. This yields a power grid frequency signal sequence of length 1800 (corresponding to 180 seconds of audio).

[0125] 3. Signal Preprocessing: The extracted power grid frequency signal was Z-score normalized using statistics from the training set. Then, a sliding window segmentation was used, with a window length of 600, a step size of 300, and an overlap rate of 50%. A total of 5 windows were generated, corresponding to time periods of 0-60 seconds, 30-90 seconds, 60-120 seconds, 90-150 seconds, and 120-180 seconds, respectively.

[0126] 4. Model Inference: Load the trained model parameters and set the model to evaluation mode (disable Dropout and DropPath). Input the model with 5 windows in batches, with a batch size of 5. The model output shape is (5, 2, 600), corresponding to the probabilities of 5 windows, 2 classes, and 600 time points. Apply Softmax normalization to the second dimension of the output (class dimension) to obtain the probability of each time point belonging to the tampering class.

[0127] 5. Post-processing: Threshold binarization is applied to the tampering probability of each window, with a threshold set to 0.5. Positions with a probability greater than the threshold are marked as 1 (tampered), otherwise marked as 0 (normal). Then, a median filter with a window size of 5 is applied to the binarized result to eliminate isolated false positives. Since there is overlap between windows, they need to be stitched together. For overlapping areas, if the predictions of two windows are inconsistent, the prediction with the higher probability is taken; if they are consistent, the prediction is retained. After stitching, a complete 1800-point prediction sequence is obtained.

[0128] 6. Tampered Region Location: Connectivity analysis is performed on the spliced ​​predicted sequence to identify continuous tampered segments. Two tampered regions were detected: Region 1, from sample index 456 to sample index 612, corresponding to times 45.6 seconds to 61.2 seconds, lasting 15.6 seconds; Region 2, from sample index 1203 to sample index 1378, corresponding to times 120.3 seconds to 137.8 seconds, lasting 17.5 seconds. Short segments with a length of less than 5 sampling points (0.5 seconds) are filtered out. In this example, both regions exceeded the threshold and were retained.

[0129] 7. Test Report Output: Generate a test report containing the following information: Audio file name: test_audio_001.wav Audio duration: 180 seconds Detection time: January 15, 2024, 14:32:05 Number of tampered areas: 2 Region 1: Start time 45.6 seconds, end time 61.2 seconds, duration 15.6 seconds, average confidence level 0.87, classified as fragment replacement. Region 2: Start time 120.3 seconds, end time 137.8 seconds, duration 17.5 seconds, average confidence level 0.92, classified as fragment insertion. Visualization charts: including the original power grid frequency signal curve, tampering probability curve, and binarized detection result bar chart. 8. Result Verification: The detection results were compared with the manually marked actual tampering locations. The actual tampering locations were: Region 1 from 45.0 seconds to 60.8 seconds (a 15.8-second segment was deleted), and Region 2 from 120.0 seconds to 138.2 seconds (an 18.2-second segment was inserted). The errors between the detection results and the actual situation were: Region 1: starting point error +0.6 seconds, ending point error +0.4 seconds; Region 2: starting point error +0.3 seconds, ending point error -0.4 seconds. All errors were within ±0.6 seconds, meeting the accuracy requirements.

[0130] 9. Performance Statistics: Detection was performed on all 200 audio segments in the test set. The total detection time was 38 minutes, with an average detection time of 11.4 seconds per audio segment (including power grid frequency extraction and model inference). The detection accuracy was 96.7%, precision was 95.2%, recall was 97.8%, and F1 score was 96.5%. The average error for tamper boundary localization was ±0.11 seconds, and the maximum error was ±0.42 seconds.

[0131] Example 2 This embodiment proposes another method for noise monitoring and digital audio tampering detection based on power grid frequency signals, which is basically the same as the first embodiment, except that the steps for extracting the power grid frequency signals are different: Continuous wavelet transform (CWT) is used for time-frequency analysis to replace short-time Fourier transform (STFT) for extracting power grid frequency signals. Wavelet transform, through an adjustable time-frequency window, can provide different time-frequency resolutions across different frequency ranges, and has a better ability to capture minute fluctuations in power grid frequency.

[0132] In another specific embodiment, Hilbert-Huang transform (HHT) can also be used for adaptive decomposition. The audio signal is decomposed into multiple intrinsic mode functions through empirical mode decomposition (EMD), and the instantaneous frequency is extracted through Hilbert transform. This method has good adaptability to non-stationary signals.

[0133] In another specific embodiment, a phase-locked loop (PLL) or a Kalman filter can also be used for grid frequency tracking. These methods can estimate frequency change trends in real time and are more robust to noise interference.

[0134] In another specific embodiment, the Iterative Discrete Fourier Transform (DFT) method can also be used to gradually refine the frequency estimation accuracy through multiple iterations, thereby reducing the impact of spectral leakage.

[0135] Example 3 This embodiment proposes another method for detecting digital audio tampering based on power grid frequency signals and noise monitoring. It is basically the same as the first embodiment, except that the signal preprocessing steps are different: Using Min-Max normalization to scale the signal to the [0, 1] or [-1, 1] interval, or using Robust normalization based on the median and interquartile range to scale, can replace Z-score normalization and is more robust to outliers.

[0136] In another specific embodiment, a fixed window, a sliding window, or an adaptive window can also be used for window segmentation. The window length can be adjusted to different durations such as 300 points (30 seconds) or 900 points (90 seconds) according to actual needs. The window overlap rate can be set to 25%, 50%, or 75%; the higher the overlap rate, the more complete the boundary information is preserved.

[0137] In another specific embodiment, a filtering step can also be added, such as using median filtering, Gaussian filtering, or bilateral filtering to smooth the extracted power grid frequency signal and remove high-frequency noise.

[0138] Example 4 This embodiment proposes another method for detecting digital audio tampering based on power grid frequency signals and noise monitoring. It is basically the same as the first embodiment, except that the deep learning model architecture is different. The ConvUtr1D blocks in the encoder are replaced with residual blocks (ResBlock) or dense blocks (DenseBlock). Residual blocks alleviate the gradient vanishing problem through skip connections, while dense blocks improve parameter efficiency through feature reuse.

[0139] In another specific embodiment, the kernel size can be adjusted. The kernel size of each layer of the encoder can be set to different combinations such as (3, 3, 5), (5, 5, 9), or (3, 5, 7). The depthwise kernel size of the bottleneck layer local aggregation module can be set to 5, 7, or 11, selected according to the required time span of capture.

[0140] In another specific embodiment, the attention mechanism can employ different variations. Local-global-local attention can be replaced with pure convolutional attention (CBAM), adaptive attention, or deformable attention. The spatial reduction ratio of global sparse attention can be set to 1 (no reduction), 2, 4, or 8, depending on the trade-off between computational resources and accuracy requirements.

[0141] In another specific embodiment, the two-layer design of the bottleneck layer can be simplified to a single layer, using only LGL attention blocks or only Transformer blocks. Alternatively, it can be extended to three layers by adding an intermediate transition layer. The number of bottleneck layers can be adjusted to 2, 4, or 5 layers.

[0142] In another specific embodiment, the skip connection method of the decoder can be direct concatenation (without downsampling), additive fusion, or attention-weighted fusion. The downsampling operation can be implemented using average pooling, max pooling, or strided convolution.

[0143] In another specific embodiment, the channel count configuration can be adjusted. The base number of channels can be set to (8, 16, 32, 64) for ultra-lightweight models, or to (32, 64, 128, 256) for high-precision models. The channel count scaling factor can be set to 2x, 3x, or 4x.

[0144] In summary, this invention proposes a noise monitoring method for digital audio tampering detection based on power grid frequency signals. It employs a lightweight one-dimensional version of the U-shaped visual Transformer (Mobile U-ViT 1D) neural network model and, drawing on medical image segmentation techniques, innovatively transfers two-dimensional image segmentation methods to the field of one-dimensional time-series signal processing, achieving high-precision tampering detection while maintaining lightweight design. Key points are as follows: 1. The core innovation of this invention is to transform the problem of detecting tampering in power grid frequency signals into a one-dimensional time series segmentation task. By drawing on medical image segmentation technology, it achieves a cross-domain transfer from two-dimensional image segmentation to one-dimensional time series signal segmentation, enabling tampering detection to accurately locate the tampering boundary through point-by-point annotation.

[0145] 2. A lightweight U-shaped visual Transformer one-dimensional version (Mobile U-ViT 1D) network architecture is adopted for tampering detection of power grid frequency signals. This architecture includes a U-shaped structure of encoder-bottleneck layer-decoder. Through multi-scale feature extraction and cross-layer feature fusion, high-precision detection (F1 score > 95%) is achieved while maintaining a lightweight (parameter count < 1M).

[0146] 3. Application of Local-Global-Local (LGL) Attention Mechanism in Time Series Signal Analysis. This mechanism includes a local aggregation module and a global sparse attention module. It captures local dependencies through large-kernel deep convolution (kernel size of 9) and models global dependencies through spatially reduced attention (reduction ratio of 2). This solves the problems of low information density and high semantic differences in power grid frequency signals and can effectively identify transitional features of tampered boundaries.

[0147] 4. Design of a dual-bottom bottleneck architecture. The first stage uses LGL attention blocks to process mid-level features, and the second stage uses Transformer blocks to model global long-range dependencies. The two stages optimize local and global information extraction in 64-dimensional and 128-dimensional feature spaces, respectively. Compared with a single bottleneck design, this architecture can better handle complex tampering patterns.

[0148] 5. Downsampling Skip Connection Strategy. Unlike the direct connections in traditional U-Net, this invention performs additional downsampling processing on the skip connection features from the encoder to the decoder. This reduces noise interference and computational load while preserving the main structural information, thus improving the robustness and efficiency of the model.

[0149] 6. Hierarchical feature embedding design. The encoder uses three ConvUtr1D layers and uses convolutional kernels of different scales (3, 3, 7) to extract features across different time spans layer by layer. The deep layers use large convolutional kernels (7) to capture tampering transition patterns that span a long time, which is crucial for detecting complex tampering types.

[0150] 7. Design of the ConvUtr1D block. This module combines depthwise separable convolution and an inverted bottleneck structure. Through two residual branches (depthwise convolution branch and inverted bottleneck branch), it achieves powerful feature representation capabilities while maintaining lightweight design. It is a key technology for achieving a model with less than 1M parameters.

[0151] 8. Design of the combined loss function. A weighted combination of cross-entropy loss and Dice loss is adopted. Cross-entropy loss optimizes the pointwise classification accuracy, while Dice loss handles the class imbalance problem. The combination of the two enables the model to maintain high accuracy while having good detection ability for the minority class (tampered samples).

[0152] 9. End-to-end automated detection process. The entire process, from power grid frequency signal extraction, preprocessing, model inference, post-processing to tamper location, can be completed automatically without manual parameter adjustment, significantly improving detection efficiency. The inference time for a single 60-second audio window is less than 30 milliseconds, supporting real-time and batch detection.

[0153] 10. Specialized optimization for noise monitoring audio. The model design fully considers the characteristics of power grid frequency signals (low information density, slow changes, and susceptibility to noise interference). Through techniques such as large kernel convolution, sparse attention, and data augmentation, the robustness in actual noise monitoring scenarios is improved, and the detection accuracy can still be maintained at over 90% under low signal-to-noise ratio (10dB) conditions.

[0154] 11. High-precision tamper-proof boundary location capability. Achieving a positioning accuracy of ±0.1 seconds through point-by-point segmentation, which is 3-10 times higher than traditional methods (±0.5-1 seconds), with a minimum detectable tamper length of 0.5 seconds, meeting the needs of environmental regulatory enforcement for accurate evidence collection.

[0155] 12. The lightweight design allows the model to be deployed on edge devices and mobile terminals. Employing techniques such as depthwise separable convolution, sparse attention, and channel number optimization, the model has 992,690 parameters, a computational complexity of 0.8 GFLOPs, and a memory footprint of less than 50 MB. It supports offline operation on resource-constrained devices and is suitable for application scenarios such as on-site inspections by law enforcement agencies.

[0156] The key technical indicators of this invention are summarized in Table 3, and the quantitative effect comparison table is shown in Table 4.

[0157]

[0158]

[0159] As can be seen from Tables 3 and 4, this invention achieves high-precision, high-efficiency, and lightweight digital audio tampering detection for noise monitoring, providing a practical and reliable technical means for environmental regulatory enforcement, and possessing significant social value and application prospects. It has the following beneficial effects: 1. Significantly Improved Detection Accuracy. This invention employs a deep learning model to automatically learn tampering feature patterns. Compared to traditional signal processing methods that rely on manually set thresholds and parameters, it achieves a detection accuracy of over 96% and an F1 score exceeding 95%. Existing methods based on Pearson similarity and Euclidean distance typically achieve accuracy between 85% and 90% in complex tampering scenarios and are sensitive to parameter settings. This invention, through an end-to-end learning approach, can adapt to different tampering patterns, significantly reducing false positive and false negative rates.

[0160] 2. High accuracy in locating tampered boundaries. This invention employs a point-by-point segmentation method, enabling precise marking of the start and end times of the tampered location with a positioning accuracy of ±0.1 seconds. Traditional methods primarily rely on detecting abrupt changes or sliding window matching, typically achieving positioning accuracy between ±0.5 and ±1 seconds, making it difficult to accurately define the tampered boundaries. The high-precision positioning capability of this invention is of significant importance for the forensic identification of environmental noise monitoring data, providing more reliable technical support for law enforcement evidence collection.

[0161] 3. High degree of automation, reducing manual intervention. The entire detection process of this invention, from extracting the power grid frequency signal to locating the tampered area, can be completed automatically without manual parameter adjustment. Existing technologies require adjusting multiple parameters such as the window size of the short-time Fourier transform, filter parameters, and matching thresholds for different audio quality and noise environments, which is time-consuming and relies on professional experience. This invention adopts a data-driven deep learning method. The model learns the optimal parameter configuration during the training phase, and only needs to input the audio to be detected during the inference phase to output the result, which greatly improves the detection efficiency. Based on the average processing time per audio file, this invention only requires about 30 milliseconds to process a 60-second audio file (excluding the power grid frequency extraction time), while traditional methods usually require hundreds of milliseconds to several seconds.

[0162] 4. High computational efficiency and lightweight model. The Mobile U-ViT 1D model of this invention has only 992,690 parameters, less than 1M, and a computational complexity of 0.8 GFLOPs. Compared with the ResNet-50 1D model (25.6M parameters, 4.1 GFLOPs), the number of parameters is reduced by 96% and the computational cost is reduced by 80%. The model's memory usage during inference is less than 50 MB, which can be deployed on edge devices and mobile terminals, supporting offline detection. This has significant practical value for on-site inspections by law enforcement agencies and applications on portable devices. Existing generative adversarial network-based methods typically require tens or even hundreds of megabytes of parameters, making them difficult to run on resource-constrained devices.

[0163] 5. Strong robustness and adaptability to various scenarios. This invention leverages the powerful representational capabilities of deep neural networks to maintain stable detection performance under varying signal-to-noise ratios (SNR). Experiments show that even with an SNR reduced to 10 dB, the detection accuracy remains above 90%. Existing methods experience significant performance degradation under low SNR conditions, with accuracy potentially dropping to 70%-80%. The data augmentation strategies employed in this invention (random pruning, time stretching, and Gaussian noise injection) expose the model to diverse scenarios during training, improving generalization ability. Furthermore, this invention demonstrates good adaptability to different recording devices and audio samples with varying sampling rates.

[0164] 6. Supports detection of multiple tampering types. This invention can detect not only simple segment insertion and deletion tampering, but also complex multi-segment tampering, short-term tampering, and segment replacement operations. It can effectively detect tampering segments as short as 0.5 seconds, while traditional methods are weak in detecting tampering segments shorter than 2 seconds. The local-global-local attention mechanism employed in this invention captures the transition features of tampering boundaries through deep convolution with a large kernel (kernel size of 9), and understands the overall temporal consistency through global attention. The combination of these two approaches gives the model a strong ability to recognize various tampering patterns.

[0165] 7. Highly innovative technical solution. This invention is the first to apply medical image segmentation technology to the detection of tampering in power grid frequency signals, achieving a cross-domain migration from two-dimensional image segmentation to one-dimensional time-series signal segmentation. It innovatively proposes a dual-layer bottleneck architecture and a downsampling skip connection strategy, achieving high accuracy while maintaining lightweight design. These technological innovations are not only applicable to noise monitoring and audio tampering detection, but can also be extended to other fields of time-series signal anomaly detection, such as electrocardiogram signal analysis and industrial equipment vibration monitoring.

[0166] 8. High Practical Application Value. This invention provides a highly efficient and reliable technical solution to the practical problem of data falsification by third-party environmental service agencies in the field of environmental noise monitoring. This method can be integrated into the data auditing system of environmental regulatory departments to achieve automated batch detection of reported audio data, significantly improving regulatory efficiency. According to existing regulatory processes, manually auditing a single audio segment typically takes 5-10 minutes, while this invention can process dozens of audio segments within one minute, increasing efficiency by dozens of times. Simultaneously, the detection results include the start and end times of the tampered area, confidence level, and visual charts, facilitating law enforcement personnel to quickly locate problems and conduct subsequent investigations.

[0167] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting digital audio tampering based on power grid frequency signals during noise monitoring, characterized in that, Includes the following steps: Acquire digital audio from noise monitoring and extract power grid frequency signals; The extracted power grid frequency signal is preprocessed. The preprocessed power grid frequency signal is input into a trained deep learning model. The deep learning model first uses an input embedding layer to map the power grid frequency signal to a multi-dimensional feature space, and then uses cascaded encoders to progressively extract features at different time scales and perform downsampling. Then, through a bottleneck layer, a local-global-local attention mechanism and a Transformer attention mechanism are used to model local and global dependencies. Specifically, the output features of the last stage encoder are input into cascaded LGL attention blocks for processing. Each LGL attention block first uses a local aggregation module to capture local dependencies and then uses a global sparse attention module to model global dependencies. Then, the processing results of the cascaded LGL attention blocks are input into cascaded Transformer blocks to model global long-range dependencies and understand the temporal consistency of the entire audio sequence. Finally, through cascaded decoders, the output features of the bottleneck layer are upsampled sequentially to progressively restore the signal resolution, while fusing the skip connection features of the corresponding encoder. Finally, the output features of the last-stage decoder are mapped to binary classification probabilities of the normal class and the tampered class through the output layer; The model output is normalized to calculate the tampering probability of each data in the power grid frequency signal. The tampering probability of each data is compared with a preset threshold and a tampering flag is set to obtain the prediction sequence. Connectivity analysis is performed on the predicted sequence to identify continuous tampered segments. The sample index of the tampered segment is mapped back to the timestamp of the original noise monitoring digital audio, and the corresponding detection report is output.

2. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, When using an input embedding layer to map the power grid frequency signal to a multidimensional feature space, the power grid frequency signal is processed sequentially through one-dimensional convolution, batch normalization, and activation functions, thereby mapping the single-channel power grid frequency signal to a multidimensional feature space.

3. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, When using a cascaded encoder to progressively extract features at different time scales and perform downsampling, the specific steps include: The first-level encoder obtains the output features of the input embedding layer, and after processing by the ConvUtr1D block, the corresponding output features are obtained. The second-stage encoder acquires the output features of the first-stage encoder, and then processes them sequentially through the ConvUtr1D block and downsampling to obtain the corresponding output features. The third-stage encoder acquires the output features of the second-stage encoder, and after sequential processing by cascaded ConvUtr1D blocks and downsampling, obtains the corresponding output features, which are then input into the bottleneck layer for processing.

4. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 3, characterized in that, The ConvUtr1D block includes cascaded depthwise convolutional branches and inverted bottleneck branches, wherein: The depthwise convolutional branch extracts features from the local receptive field, and its mathematical expression is as follows: in, The output features of the depthwise convolution branch, The input features are the depthwise convolutional branch, DWConv represents depthwise separable convolution, k is the kernel size, and GELU is the Gaussian error linear unit activation function. The inverted bottleneck branch first expands the number of channels and then compresses them to enhance feature representation capabilities. The mathematical expression is as follows: in, The output characteristics of the inverted bottleneck branch, Indicates batch normalization, and For different 1×1 convolutions, C represents the number of input channels, and the superscript indicates the channel number transformation relationship. This means increasing the number of channels by 4. This means compressing the number of channels back to 1 / 4 of the original number and then restoring it.

5. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, When capturing local dependencies using the local aggregation module, a three-stage gating modulation is employed to enhance the expressive power of local features. The mathematical expression is as follows: in, These are the input features of the local aggregation module. These are the output features of the local aggregation module. This represents element-wise multiplication. This indicates a position encoding operation. Represents a 1×1 convolution. This indicates a depthwise separable convolution with a kernel size of 9. Indicates batch normalization, It is the Sigmoid activation function. For position encoding.

6. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, When modeling global dependencies using a global sparse attention module, a spatial reduction attention mechanism is first employed to spatially reduce the input features before multi-head self-attention computation. Linear interpolation is then used to restore the reduced attention result to its original resolution. Finally, a nonlinear transformation is performed using a multilayer perceptron. The mathematical expression is as follows: in, These are the input features of the global sparse attention module. These are the output features of the global sparse attention module. This indicates a position encoding operation. Representation space reduction attention mechanism, This represents a multilayer perceptron. Representation layer normalization.

7. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, The cascaded decoders and cascaded encoders are one-to-one and both are three-stage. The signal resolution is gradually restored by upsampling the output features of the bottleneck layer through the cascaded decoders. At the same time, when the skip connection features of the corresponding encoders are fused, the mathematical expression is as follows: in, This represents the output features of the first-level decoder. This represents the output characteristics of the bottleneck layer. This represents the skip connection feature after downsampling from the third-level encoder. This represents the output features of the second-level decoder. This represents the skip connection feature after downsampling by the second-level encoder. This represents the output characteristics of the third-level encoder. This represents the skip connection features after downsampling from the first-level encoder. The term "upsampled convolutional block" indicates an upsampled convolutional block, "Concat" indicates a channel concatenation operation, "Up" indicates upsampling, and "Down" indicates downsampling. Specifically, the downsampled skip connection features are the features obtained by max pooling the skip connection features of the corresponding encoder.

8. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 7, characterized in that, When the output features of the last-stage decoder are mapped to binary classification probabilities of the normal class and the tampered class through the output layer, the mathematical expression is as follows: in, This represents the input features of the output layer. , Represents a 1×1 convolution, output Channel 0 represents the probability of a normal sample, channel 1 represents the probability of a tampered sample, B is the batch size, and L is the signal length.

9. The method for detecting digital audio tampering based on power grid frequency signals for noise monitoring according to claim 1, characterized in that, The mathematical expression for the loss function during the training of the deep learning model is as follows: in, , These are weight hyperparameters; The cross-entropy loss is expressed mathematically as follows: in, For the true labels of i data points in the power grid frequency signal, Let c be the predicted probability of i data points in the power grid frequency signal, where c = 0 represents a normal sample and c = 1 represents a tampered sample, and L is the signal length. The mathematical expression for Dice loss is as follows: in, Let i data points in the power grid frequency signal be the true labels for those belonging to the tampering category. Let be the predicted probability that i data points in the power grid frequency signal belong to the tampering class. To prevent smoothing terms from being divided by zero.

Images