Deep power grid frequency enhancement method, device and equipment oriented to audio evidence collection

Abstract

The invention relates to the technical field of digital signal detection and processing, in particular to a deep power grid frequency enhancement method, device and equipment oriented to audio forensics, and the method comprises the steps: obtaining audio data needed by a forensics application; preprocessing the audio data, and estimating a power grid frequency signal of the preprocessed audio data; a power grid frequency signal is input into a generative adversarial network, the generative adversarial network outputs an enhanced power grid frequency signal, the generative adversarial network comprises a generator and a discriminator, the number of input channels of the generator and the number of input channels of the discriminator are different, and an encoder part of the generator uses a plurality of one-dimensional convolutional layers of the same structure; an encoder part and a decoder part of the generator are mirror images of each other; and performing target evidence obtaining based on the enhanced power grid frequency signal and the reference signal database. Therefore, the problems of difficult signal extraction, insufficient quality, high calculation complexity, limited application range, insufficient standardization and interoperability and the like in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the technical field of digital signal detection and processing, and in particular to a method, device and equipment for deep power grid frequency enhancement for audio forensics. Background Art

[0002] With the rapid advancement of digital technology, the recording, dissemination and use of audio have become more convenient and efficient than ever before, greatly broadening the application areas of digital audio files and increasing their number. However, this convenience is also accompanied by hidden dangers. The popularity of audio and video editing tools has given users powerful processing capabilities, making it easy to modify audio files, thereby exacerbating the risk of audio information security and providing a breeding ground for criminals to forge and spread false information. In key areas such as military intelligence analysis, case investigation, and judicial evidence collection, the authenticity and integrity of audio files are extremely high. Therefore, in-depth research on digital audio forensics technology is of great significance to maintaining national information security and promoting social stability.

[0003] As one of the basic characteristics of the power distribution network, the stability of the power grid frequency is determined by the turbine speed. It is generally maintained near the nominal value of 50Hz or 60Hz, and exhibits small random fluctuations. This feature makes the power grid frequency signal widely present in multimedia content such as audio, video and pictures, becoming a key passive indicator in the field of digital multimedia forensics. With its unique fluctuation pattern and regional consistency, the power grid frequency forensics technology has shown great application potential in geographic location tracing, timestamp verification and tampering detection.

[0004] At present, the forensic research based on power grid frequency mainly focuses on the accurate estimation of power grid frequency and its subsequent application exploration in forensic practice. However, there is still a lack of research on whether the quality of power grid frequency signals extracted from complex multimedia environments is sufficient to support high-standard forensic requirements. The enhancement processing of power grid frequency signals, especially noise suppression technology, has gradually become a research hotspot in recent years. The existing enhancement methods are mainly RFA (Robust Filtering Algorithm), HRFA ​​(Harmonic Robust Filtering Algorithm) and robust media timestamp method, all of which are rooted in traditional signal processing technology. Although they can effectively reduce additive noise interference and improve the quality of power grid frequency signals in low signal-to-noise ratio environments, they generally have the problems of high computational complexity and long processing time. In particular, the de-fragmentation algorithm, although it optimizes the signal quality by removing noise fragments, sacrifices the integrity and tampering detection capabilities of the signal, limits its extensive application in forensic applications, and is prone to misleading results when processing short-duration signals. Summary of the invention

[0005] The present application provides a deep power grid frequency enhancement method, device and equipment for audio forensics to solve the problems of difficult signal extraction, insufficient quality, high computational complexity, limited application scope, and lack of standardization and interoperability in the prior art.

[0006] The first aspect of the present application provides a deep power grid frequency enhancement method for audio forensics, comprising the following steps: acquiring audio data required for forensic applications; preprocessing the audio data and estimating the power grid frequency signal of the preprocessed audio data; inputting the power grid frequency signal into a generative adversarial network, and the generative adversarial network outputs an enhanced power grid frequency signal, wherein the generative adversarial network includes a generator and a discriminator, the generator and the discriminator have different numbers of input channels, the encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and decoder part of the generator are mirror images of each other; and performing target forensics based on the enhanced power grid frequency signal and a reference signal database.

[0007] Optionally, before the power grid frequency signal is input into the generative adversarial network, it also includes: obtaining a training data set, wherein the training data set includes a reference power grid frequency signal and a noisy power grid frequency signal extracted from the actual recording data; inputting the reference power grid frequency signal and the noisy power grid frequency signal into a discriminator, the discriminator learning a first data distribution of the reference power grid frequency signal and the noisy power grid frequency signal, performing binary classification based on the first data distribution to obtain real samples, and updating the parameters of the discriminator through back propagation; inputting the noisy power grid frequency signal and the potential sample vector into a generator, the generator outputs a denoised power grid frequency signal, inputting the noisy power grid frequency signal and the denoised power grid frequency signal into a discriminator, the discriminator learning a second data distribution of the noisy power grid frequency signal and the denoised power grid frequency signal, performing binary classification based on the second data distribution to obtain false samples, and updating the parameters of the discriminator through back propagation; freezing the parameters of the discriminator, calculating a training loss value based on the discriminator output and the real sample, and allowing the generator to back propagate and learn the data distribution of the reference power grid frequency signal.

[0008] Optionally, the target forensics based on the enhanced grid frequency signal and the reference signal database includes: calculating the mean square error or correlation coefficient between the grid frequency signal and the reference signal in the reference signal database; and determining a reference signal matching the grid frequency signal based on the minimum mean square error in the mean square error or the maximum correlation coefficient in the correlation coefficient.

[0009] Optionally, the calculation formula of the mean square error is:

[0010]

[0011] Wherein, N is the length, x(n) is the power grid frequency signal extracted from the recording file to be tested, n is the nth frequency point of the signal, r(n) is the power grid frequency reference signal within the search range, and k is the kth reference signal; the calculation formula of the correlation coefficient is:

[0012]

[0013] in, is the mean value of the grid frequency reference signal within the search time range, is the mean value of the power grid frequency extracted from the audio file to be tested.

[0014] Optionally, before performing target forensics based on the enhanced power grid frequency signal and reference signal database, it also includes: synchronously collecting the power grid frequency signal and the corresponding time information; storing the power grid frequency signal and the corresponding time information in different storage locations; and generating the reference signal database based on the data stored in the different storage locations.

[0015] Optionally, estimating the power grid frequency signal of the preprocessed audio data includes: performing short-time Fourier transform on the preprocessed audio data; and estimating the power grid frequency signal of the short-time Fourier transformed data using a quadratic interpolation method to obtain the power grid frequency signal.

[0016] The second aspect of the present application provides a deep power grid frequency enhancement device for audio forensics, including: an acquisition module, used to acquire audio data required for forensic applications; a processing module, used to preprocess the audio data and estimate the power grid frequency signal of the preprocessed audio data; an output module, used to input the power grid frequency signal into a generative adversarial network, and the generative adversarial network outputs an enhanced power grid frequency signal, wherein the generative adversarial network includes a generator and a discriminator, the generator and the discriminator have different numbers of input channels, the encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and decoder part of the generator are mirror images of each other; an forensics module, used to perform target forensics based on the enhanced power grid frequency signal and a reference signal database.

[0017] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep power grid frequency enhancement method for audio forensics as described in the above embodiment.

[0018] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the deep power grid frequency enhancement method for audio forensics as described in the above embodiments.

[0019] The fifth aspect of the present application provides a computer program product, including a computer program or instructions, which, when executed, are used to implement the deep power grid frequency enhancement method for audio forensics as described in the above-mentioned embodiment.

[0020] Therefore, this application includes the following beneficial effects:

[0021] The embodiment of the present application uses deep learning technology, especially the bidirectional conditional least squares generative adversarial network framework, to effectively process large-scale power grid frequency data sets by optimizing the generator structure and loss function, significantly enhancing the quality of power grid frequency signals, especially in complex environments with low signal-to-noise ratios. It not only simplifies the calculation process and realizes real-time processing, but also ensures the integrity of power grid frequency signals, further expanding the application boundaries of audio forensics technology. As a result, technical problems such as difficulty in signal extraction, insufficient quality, high computational complexity, limited application scope, and insufficient standardization and interoperability in the prior art are solved.

[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 A flowchart of a deep power grid frequency enhancement method for audio forensics provided according to an embodiment of the present application;

[0025] Figure 2 A schematic diagram of a generator structure of a generative adversarial network for power grid frequency enhancement provided according to an embodiment of the present application;

[0026] Figure 3 A training flow chart of a generative adversarial network for grid frequency enhancement provided according to an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the result of enhancing the power grid frequency signal provided according to an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a timestamp matching result after a power grid frequency signal is enhanced according to an embodiment of the present application;

[0029] Figure 6 A flowchart of a deep power grid frequency enhancement method for audio forensics provided according to an embodiment of the present application;

[0030] Figure 7 A schematic diagram of a deep power grid frequency enhancement system for audio forensics provided according to an embodiment of the present application;

[0031] Figure 8 This is an example diagram of a deep power grid frequency enhancement device for audio forensics according to an embodiment of the present application;

[0032] Fig. 9 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the deep power grid frequency enhancement method, device and equipment for audio forensics of the embodiments of the present application with reference to the accompanying drawings. In response to the problem of inability to accurately obtain evidence mentioned in the above background technology, the present application provides a deep power grid frequency enhancement method for audio forensics. In this method, deep learning technology, especially the bidirectional conditional least squares generative adversarial network framework, is used to effectively process large-scale power grid frequency data sets by optimizing the generator structure and loss function, and significantly enhance the quality of power grid frequency signals, especially in complex environments with low signal-to-noise ratios. It not only simplifies the calculation process and realizes real-time processing, but also ensures the integrity of the power grid frequency signal, further expanding the application boundaries of audio forensics technology. As a result, the problems of difficult signal extraction, insufficient quality, high computational complexity, limited application scope, and lack of standardization and interoperability in the prior art are solved.

[0035] Specifically, Figure 1 A flowchart of a deep power grid frequency enhancement method for audio forensics provided in an embodiment of the present application.

[0036] like Figure 1 As shown, the deep power grid frequency enhancement method for audio forensics includes the following steps:

[0037] In step S101, audio data required by the forensic application is obtained.

[0038] Among them, forensic application can be the process of obtaining and analyzing evidence using technical means in fields such as law, security or investigation, and audio data can be sound information recorded by recording equipment.

[0039] It can be understood that the embodiments of the present application not only provide key evidence for case investigation by acquiring the audio data required for forensic applications, but also serve as the basis for deep learning model processing, facilitating subsequent support for the estimation and enhancement of power grid frequency signals.

[0040] Specifically, the recorded audio file is read to obtain the read audio data and sampling frequency. In general, the audio sampling frequency is 8000Hz or 44100Hz. In order to reduce the required computing resources, the audio recording is downsampled, where the downsampled audio data is S and the downsampled sampling frequency is f. s , f s Can be set to 400Hz.

[0041] In step S102, the audio data is preprocessed, and a power grid frequency signal of the preprocessed audio data is estimated.

[0042] The grid frequency signal may be the power supply frequency of the power distribution network in the grid.

[0043] It can be understood that the embodiments of the present application can reduce noise and interference in the audio data, remove some unnecessary redundant information, improve data quality, and facilitate more accurate estimation of the power grid frequency signal by preprocessing the audio data. Through preprocessing and power grid frequency signal estimation, the authenticity and integrity of the audio data can be further verified.

[0044] Specifically, since the power grid frequency is a narrowband signal, the out-of-band audio content and noise will affect the power grid frequency estimation, so a bandpass filter is selected to filter the downsampled result, with the center frequency set to 100 Hz and the passband width set to 4 Hz, to obtain the audio data S after bandpass filtering and preprocessing. down .

[0045] In an embodiment of the present application, estimating the power grid frequency signal of the preprocessed audio data includes: performing short-time Fourier transform on the preprocessed audio data; and using quadratic interpolation to estimate the power grid frequency signal of the short-time Fourier transformed data to obtain the power grid frequency signal.

[0046] Among them, the short-time Fourier transform can be a mathematical transformation related to the Fourier transform, which is used to determine the frequency and phase of the sine wave in the local area of ​​the time-varying signal; the quadratic interpolation method can be a polynomial approximation method established by utilizing the property of a function as a quadratic function near the extreme point.

[0047] It can be understood that the embodiment of the present application applies short-time Fourier transform to the preprocessed audio data to extract spectral features, and uses quadratic interpolation to accurately estimate the power grid frequency signal, thereby improving the efficiency and accuracy of audio signal processing.

[0048] Specifically, the preprocessed data is subjected to a short-time Fourier transform to obtain a time-frequency domain sequence X(k,n) corresponding to the audio data, where k represents the kth frame of the short-time Fourier transform, n represents the nth frequency point of the short-time Fourier transform, and the total number of short-time Fourier transform frequency points is N;

[0049] Use the quadratic interpolation method to estimate the power grid frequency signal and obtain the power grid frequency signal; find the point m corresponding to the maximum value β of the fast Fourier transform within a window length, and then find the points m-1 and m+1 adjacent to m within the window length respectively, and calculate the value of its fast Fourier transform. The number of points selected is N FFT Assuming that the fast Fourier transform corresponding to point m-1 is a, and the fast Fourier transform corresponding to point m+1 is λ, the relative position δ of the top of the quadratic model interpolation is:

[0050]

[0051] The best estimated position of the instantaneous frequency of the power grid frequency signal obtained by the interpolation estimation method is m+δ, so the best estimated value of the instantaneous frequency of the power grid frequency signal is:

[0052]

[0053] The frequencies within a certain fluctuation range are retained to obtain a preliminary estimate. Since the actual fluctuation range of the power grid frequency signal is very limited, in order to reduce the impact of outliers during estimation, the amplitude of the fluctuation exceeding the specified threshold is truncated and replaced with the corresponding boundary value.

[0054] It should be noted that, considering the characteristics of frequency signal fluctuations in the central China power grid, the specified threshold may be 50 Hz ± 0.05 Hz.

[0055] In step S103, the power grid frequency signal is input into a generative adversarial network, and the generative adversarial network outputs a strengthened power grid frequency signal.

[0056] Among them, the generative adversarial network can include a generator and a discriminator. The generator and the discriminator have different numbers of input channels. The encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and the decoder part of the generator are mirror images of each other.

[0057] It is understandable that the embodiment of the present application enhances the power grid frequency signal through a generative adversarial network. The generative adversarial network includes a generator and a discriminator. The generator generates a pseudo signal using an encoder and decoder structure composed of multiple one-dimensional convolutional layers, while the discriminator is responsible for distinguishing between real signals and pseudo signals. Through adversarial training of the two, the generative adversarial network can output a clearer and closer to real power grid frequency signal, thereby achieving purposes such as signal enhancement, noise suppression, data amplification and anomaly detection.

[0058] Specifically, a bidirectional conditional least squares generative adversarial network is used, and an improved generator structure and loss function are used for training. When improving the generator, the architectural idea of ​​the autoencoder is borrowed to achieve an efficient mapping from the latent space to the data space. Figure 2 As shown in the figure, the encoder part of the generator cleverly uses four one-dimensional convolutional layers, each of which is configured with a convolution kernel of size 4 and a stride of 2. This not only effectively reduces the spatial dimension of the data, but also increases the number of channels of the feature map, thereby achieving effective feature extraction and compression.

[0059] For example, if m is the number of data samples, l is the length of a single sample, and m×l is used to represent the size of each convolutional layer, the resulting sizes of all 8 layers are: 250×64, 125×128, 63×256, 32×512. Therefore, in the encoder stage of the generator, the intermediate thought vector c guided by the latent vector z has a size of 32×512.

[0060] The decoder part of the generator is a mirror image of the encoder structure. Through a series of deconvolution operations, the spatial dimensions of the data are gradually restored until an output close to the original data is generated. This symmetrical encoder-decoder structure not only ensures the effective transmission of information, but also enhances the robustness of the network in complex data transformations.

[0061] Unlike the generator, the discriminator needs to process both the grid frequency reference signal and the noisy grid frequency signal at the same time, so its input channel is designed to be two to receive these two signals respectively. In terms of structure, the discriminator uses a convolutional layer stacking method similar to the generator encoder, but it is appropriately adjusted according to the task requirements to ensure that it can accurately distinguish between real samples and generated samples, providing a strong feedback signal for the generator.

[0062] In an embodiment of the present application, before the power grid frequency signal is input into the generative adversarial network, it also includes: obtaining a training data set, wherein the training data set includes a reference power grid frequency signal and a noisy power grid frequency signal extracted from the actual recording data; inputting the reference power grid frequency signal and the noisy power grid frequency signal into the discriminator, the discriminator learns the first data distribution of the reference power grid frequency signal and the noisy power grid frequency signal, performs binary classification based on the first data distribution to obtain real samples, and updates the parameters of the discriminator through back propagation; inputting the noisy power grid frequency signal and the potential sample vector into the generator, the generator outputs the denoised power grid frequency signal, inputting the noisy power grid frequency signal and the denoised power grid frequency signal into the discriminator, the discriminator learns the second data distribution of the noisy power grid frequency signal and the denoised power grid frequency signal, performs binary classification based on the second data distribution to obtain false samples, and updates the parameters of the discriminator through back propagation; freezing the parameters of the discriminator, calculating the training loss value based on the discriminator output and the real sample, and allowing the generator to back propagate and learn the data distribution of the reference power grid frequency signal.

[0063] The reference grid frequency signal may be a clean grid frequency signal sample that is not interfered by noise.

[0064] It can be understood that, in the embodiment of the present application, by alternately training the generator and the discriminator, in their mutual confrontation, the generator gradually learns to generate more realistic denoised signals, while the discriminator continuously improves its ability to distinguish between real signals and false signals. By continuously optimizing the generator and the discriminator, effective denoising of the power grid frequency signal is achieved.

[0065] For example, Figure 3 As shown in Figure 1, the training process can be divided into two steps. In the first step, the training data consists of a reference grid frequency signal x and a noisy grid frequency signal y extracted from an actual recording. These two types of signals are input into the discriminator for preliminary learning. The discriminator distinguishes between real samples and noisy samples through a binary classification mechanism, and continuously optimizes its parameters using a back-propagation algorithm to enhance the ability to distinguish.

[0066] The second step can be divided into two stages to fine-tune the denoising ability of the generator. In the first stage, the discriminator faces new challenges: the discriminator again uses its learned knowledge of data distribution to try to classify the data pairs of noisy and denoised signals as fake samples, and continues to update its own parameters through backpropagation to cope with the generator's ever-improving deception ability.

[0067] Entering the second stage, in order to stabilize the training process and ensure that the generator focuses on learning the essential characteristics of the reference signal, the parameters of the discriminator are frozen. At this point, the generator has the opportunity to optimize its parameters alone through the back-propagation mechanism, and can generate a denoised version that is closer to the reference grid frequency signal. As the number of iterations increases, the generator gradually masters the data distribution of the reference signal, and its output quality is significantly improved.

[0068] like Figure 4 As shown in the figure, the results of the signal before enhancement, the original reference signal and three existing advanced enhancement algorithms are compared intuitively. In order to observe the results more clearly, the amplitudes of different signals are shifted up and down to highlight the significant improvements in clarity, signal-to-noise ratio, etc. of the enhanced signal.

[0069] In step S104, target forensics is performed based on the enhanced grid frequency signal and the reference signal database.

[0070] The reference signal database may include known, standard or verified grid frequency signals.

[0071] It can be understood that the embodiments of the present application utilize the enhanced grid frequency signal and reference signal database to perform specific forensic analysis or verification work, such as determining the timestamp of the audio recording, verifying the authenticity of the audio, etc., which improves the accuracy, capability and application scope of the forensic process and effectively ensures the authenticity of the evidence.

[0072] In an embodiment of the present application, target forensics is performed based on the enhanced grid frequency signal and the reference signal database, including: calculating the mean square error or correlation coefficient between the grid frequency signal and the reference signal in the reference signal database; determining the reference signal that matches the grid frequency signal based on the minimum mean square error in the mean square error or the maximum correlation coefficient in the correlation coefficient.

[0073] It can be understood that the embodiments of the present application improve the accuracy of signal recognition by calculating the mean square error or maximum correlation coefficient between the grid frequency signal and the reference signal database to determine the most matching reference signal, thereby supporting grid monitoring and analysis, helping to optimize grid operation, predict and prevent faults, and promote the development of smart grids.

[0074] In the embodiment of the present application, the calculation formula of the mean square error is:

[0075]

[0076] Wherein, N is the length, x(n) is the power grid frequency signal extracted from the recording file to be tested, n is the nth frequency point of the signal, r(n) is the power grid frequency reference signal within the search range, and k is the kth reference signal;

[0077] The calculation formula for the correlation coefficient is as follows:

[0078]

[0079] where, is the mean value of the grid frequency reference signal within the search time range, is the mean value of the grid frequency extracted from the audio file to be measured.

[0080] Specifically, data matching is performed between the enhanced grid frequency signal and the reference signal. It is required that the time-frequency sequence of the obtained reference signal is longer than the time-frequency sequence extracted from the audio file to be measured, so as to achieve search within the possible audio recording time range. Therefore, in order to determine the part of the reference signal corresponding to the extracted grid frequency signal, the minimum mean square error or the maximum correlation coefficient can be found between all possible reference signals and the extracted grid frequency signal. If the grid frequency reference signal within the search range is r(n) with a length of L, and the grid frequency signal extracted from the audio file to be measured is x(n) with a length of N, where N < L, the mean square error formula is:

[0081]

[0082] where, r k (n) = r(n + k), k = 0, 1, 2... L - N.

[0083] The corresponding optimal matching position at this time is:

[0084]

[0085] The calculation formula for the correlation coefficient is as follows:

[0086]

[0087] where, is the mean value of the grid frequency reference signal within the search time range, is the mean value of the grid frequency extracted from the audio file to be measured.

[0088] The corresponding optimal matching position at this time is:

[0089]

[0090] In the embodiment of the present application, the matching criterion of the maximum correlation coefficient is selected, and the timestamp matching result after enhancing a certain grid frequency signal is obtained. As Figure 5 shown, it shows the comparison of the grid frequency signal before and after enhancement, as well as the effects of three advanced enhancement algorithms. Each signal is shown by a dotted line representing the reference signal and the upper and lower offsets, clearly comparing the improvement of the signal quality, which provides strong support for evaluating the enhancement algorithm.

[0091] In an embodiment of the present application, before performing target forensics based on the strengthened power grid frequency signal and reference signal database, it also includes: synchronously collecting the power grid frequency signal and the corresponding time information; storing the power grid frequency signal and the corresponding time information in different storage locations; and generating a reference signal database based on the data stored in different storage locations.

[0092] It can be understood that the embodiment of the present application ensures the accuracy and completeness of the data, optimizes data management, supports real-time monitoring and early warning, and improves the stability and reliability of the power grid by synchronously collecting the power grid frequency signal and its time information, storing them separately, and then generating a reference signal database.

[0093] Specifically, a reference signal acquisition and storage system for real-time processing of grid frequency signals is constructed to realize the archiving of grid frequency reference signals. When collecting grid frequency reference signals, the step-down transformer captures grid frequency fluctuations and transmits them to the computer, and the computer sound card and control program read the signal and save it to the buffer. In order to facilitate the self-setting of short-time Fourier transform parameters and accuracy, the time domain reference signal is directly stored.

[0094] The Python program based on the Windows operating system was carefully written to ensure that the system can run uninterruptedly for 24 hours, and the collected power grid frequency signal values ​​and their corresponding time information are periodically stored in two different storage locations.

[0095] In addition, the embodiment of the present application arranges a power grid frequency collector for synchronous collection, which can reduce the impact of local power grid activity instability, obtain a more accurate and stable power grid frequency reference signal, effectively avoid interference with data collection caused by special circumstances such as local power outages and equipment failures, and ensure data continuity and reliability.

[0096] The grid frequency reference signal used in the embodiment of the present application is stored using the following parameters: the sampling rate is set to 400 Hz, the quantization accuracy is 16 bits, the channel is selected as mono, and the storage interval is 1 hour. Among them, all audio files are saved in pulse code modulation waveform format, which helps to ensure the quality and accuracy of audio data. The naming rule of audio files is "year_month_day_week_hour_minute_second.wav", for example, 2024_06_26_Wed_07_00_00.wav, to facilitate intuitive organization and search of audio files.

[0097] According to the deep power grid frequency enhancement method for audio forensics proposed in the embodiment of the present application, deep learning technology, especially the bidirectional conditional least squares generative adversarial network framework, is used to effectively process large-scale power grid frequency data sets by optimizing the generator structure and loss function, and significantly enhance the quality of power grid frequency signals, especially in complex environments with low signal-to-noise ratios. It not only simplifies the calculation process and realizes real-time processing, but also ensures the integrity of power grid frequency signals, further expanding the application boundaries of audio forensics technology. As a result, the problems of difficult signal extraction, insufficient quality, high computational complexity, limited application scope, and insufficient standardization and interoperability in the prior art are solved.

[0098] The following will be combined Figure 6 The deep power grid frequency enhancement method for audio forensics is described in detail as follows:

[0099] Step 1: Downsample the recorded audio file and pass it through a bandpass filter to obtain preprocessed data;

[0100] Step 2: Perform short-time Fourier transform on the preprocessed data and use quadratic interpolation to estimate the grid frequency signal, obtain the grid frequency signal, retain the frequency within a certain fluctuation range, and obtain a preliminary estimate;

[0101] Step 3: construct and train a grid frequency-generative adversarial network for grid frequency enhancement, input the estimated grid frequency signal into the neural network, and obtain an enhanced grid frequency signal;

[0102] Step 4: Build a grid frequency reference signal database and use the enhanced grid frequency signal for forensic applications such as timestamp verification, geolocation estimation, and tamper detection.

[0103] In addition, if Figure 7 As shown, the deep power grid frequency enhancement system 10 for audio forensics provided in the embodiment of the present application includes: an audio recording module 100, a PC (Personal Computer) data processing module 200 and a power grid frequency reference signal database 300.

[0104] Among them, the audio recording module 100 is used to record the audio content to be collected, save it and transmit it to the PC data processing module; the PC processing module 200 is mainly used to save and process the audio file, and use the grid frequency generation adversarial network to enhance the extracted grid frequency signal for forensic application; the grid frequency reference signal database 300 mainly stores reference signals to provide reference values ​​for audio forensics.

[0105] In summary, the embodiments of the present application utilize deep learning technology to process large-scale power grid frequency data sets, thereby achieving efficient real-time enhancement of power grid frequency signals of different qualities, especially improving the quality of low signal-to-noise ratio signals, while ensuring signal integrity and broadening the scope of audio analysis and application.

[0106] Next, a deep power grid frequency enhancement device for audio forensics proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0107] Figure 8 It is a block diagram of a deep power grid frequency enhancement device for audio forensics according to an embodiment of the present application.

[0108] like Figure 8 As shown, the deep power grid frequency enhancement device 20 for audio forensics includes: an acquisition module 210, a processing module 220, an output module 230 and a forensics module 240.

[0109] Among them, the acquisition module 210 is used to acquire the audio data required for the forensic application; the processing module 220 is used to preprocess the audio data and estimate the power grid frequency signal of the preprocessed audio data; the output module 230 is used to input the power grid frequency signal into the generative adversarial network, and the generative adversarial network outputs the enhanced power grid frequency signal, wherein the generative adversarial network includes a generator and a discriminator, the generator and the discriminator have different numbers of input channels, the encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and the decoder part of the generator are mirror images of each other; the forensics module 240 is used to perform target forensics based on the enhanced power grid frequency signal and the reference signal database.

[0110] It should be noted that the aforementioned explanation of the embodiment of the deep power grid frequency enhancement method for audio forensics is also applicable to the deep power grid frequency enhancement device for audio forensics of this embodiment, and will not be repeated here.

[0111] According to the deep power grid frequency enhancement device for audio forensics proposed in the embodiment of the present application, deep learning technology, especially the bidirectional conditional least squares generative adversarial network framework, is used to effectively process large-scale power grid frequency data sets by optimizing the generator structure and loss function, and significantly enhance the quality of power grid frequency signals, especially in complex environments with low signal-to-noise ratios. It not only simplifies the calculation process and realizes real-time processing, but also ensures the integrity of power grid frequency signals, further expanding the application boundaries of audio forensics technology. As a result, the problems of difficult signal extraction, insufficient quality, high computational complexity, limited application scope, and insufficient standardization and interoperability in the prior art are solved.

[0112] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0113] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0114] When the processor 902 executes the program, the deep power grid frequency enhancement method for audio forensics provided in the above embodiment is implemented.

[0115] Furthermore, the electronic device further comprises:

[0116] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0117] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0118] The memory 901 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0119] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0120] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0121] The processor 902 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0122] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned deep power grid frequency enhancement method for audio forensics.

[0123] The present application also provides a computer program product, including a computer program or instructions, which, when executed, are used to implement the deep power grid frequency enhancement method for audio forensics as described in the above embodiment.

[0124] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0125] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0126] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0127] It should be understood that the various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0128] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0129] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A deep power grid frequency enhancement method for audio forensics, It is characterized in that The following steps are involved: Acquire audio data required for forensic applications; preprocessing the audio data and estimating a power grid frequency signal of the preprocessed audio data; Inputting the power grid frequency signal into a generative adversarial network, the generative adversarial network outputs an enhanced power grid frequency signal, wherein the generative adversarial network includes a generator and a discriminator, the generator and the discriminator have different numbers of input channels, the encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and the decoder part of the generator are mirror images of each other; Target forensics is performed based on the enhanced power grid frequency signal and the reference signal database.

2. The deep power grid frequency enhancement method for audio forensics according to claim 1, It is characterized in that Before the power grid frequency signal is input into the generative adversarial network, the method further includes: Acquire a training data set, wherein the training data set includes a reference power grid frequency signal and a noisy power grid frequency signal extracted from actual recording data; Inputting the reference grid frequency signal and the noisy grid frequency signal into a discriminator, the discriminator learning a first data distribution of the reference grid frequency signal and the noisy grid frequency signal, performing binary classification based on the first data distribution to obtain a real sample, and updating the parameters of the discriminator by back propagation; Inputting the noisy grid frequency signal and the potential sample vector into a generator, the generator outputting a de-noised grid frequency signal, inputting the noisy grid frequency signal and the de-noised grid frequency signal into a discriminator, the discriminator learning a second data distribution of the noisy grid frequency signal and the de-noised grid frequency signal, performing binary classification based on the second data distribution to obtain a false sample, and updating the parameters of the discriminator by back propagation; The parameters of the discriminator are frozen, a training loss value is calculated based on the discriminator output and the real sample, and the generator is allowed to back-propagate and learn the data distribution of the reference grid frequency signal.

3. The deep power grid frequency enhancement method for audio forensics according to claim 1, It is characterized in that The target forensics based on the enhanced power grid frequency signal and the reference signal database includes: Calculating a mean square error or a correlation coefficient between the grid frequency signal and a reference signal in a reference signal database; A reference signal matching the grid frequency signal is determined according to a minimum mean square error among the mean square errors or a maximum correlation coefficient among the correlation coefficients.

4. The deep power grid frequency enhancement method for audio forensics according to claim 3, It is characterized in that The calculation formula of the mean square error is: Wherein, N is the length, x(n) is the power grid frequency signal extracted from the recording file to be tested, n is the nth frequency point of the signal, r(n) is the power grid frequency reference signal within the search range, and k is the kth reference signal; The calculation formula of the correlation coefficient is: in, is the mean value of the grid frequency reference signal within the search time range, is the mean value of the power grid frequency extracted from the audio file to be tested.

5. The deep power grid frequency enhancement method for audio forensics according to claim 1 or 3, It is characterized in that Before performing target forensics based on the enhanced power grid frequency signal and reference signal database, the method further includes: Synchronously collect power grid frequency signals and corresponding time information; Storing the grid frequency signal and the corresponding time information in different storage locations; The reference signal database is generated according to the data stored in the different storage locations.

6. The deep power grid frequency enhancement method for audio forensics according to claim 1, It is characterized in that The estimating the power grid frequency signal of the preprocessed audio data comprises: Perform short-time Fourier transform on the preprocessed audio data; The grid frequency signal is estimated by using the quadratic interpolation method on the data after short-time Fourier transformation to obtain the grid frequency signal.

7. A deep power grid frequency enhancement device for audio forensics, It is characterized in that include: An acquisition module, used to acquire audio data required for forensic applications; A processing module, used for preprocessing the audio data and estimating a power grid frequency signal of the preprocessed audio data; An output module, used for inputting the power grid frequency signal into a generative adversarial network, and the generative adversarial network outputs an enhanced power grid frequency signal, wherein the generative adversarial network includes a generator and a discriminator, the generator and the discriminator have different numbers of input channels, the encoder part of the generator uses multiple one-dimensional convolutional layers with the same structure, and the encoder part and the decoder part of the generator are mirror images of each other; The forensics module is used to perform target forensics based on the enhanced power grid frequency signal and the reference signal database.

8. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep power grid frequency enhancement method for audio forensics as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, It is characterized in that When the computer program or instruction is executed, the deep power grid frequency enhancement method for audio forensics described in any one of claims 1-6 is implemented.

10. A computer program product comprising a computer program or instructions, It is characterized in that When the computer program or instruction is executed, the deep power grid frequency enhancement method for audio forensics described in any one of claims 1-6 is implemented.

Images