Audio authentication method and device, computer device and storage medium

Abstract

The application discloses an audio authentication method and device, computer equipment and a storage medium. The method comprises: obtaining audio data to be detected; extracting mel spectrum features of the audio data to be detected; inputting the mel spectrum features into a flow matching model to perform reverse operations on the mel spectrum features, and reversely mapping the mel spectrum features back to a prior noise distribution space to obtain corresponding potential noise variables; calculating a log-likelihood value of the audio data to be detected according to the potential noise variables; and comparing the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio. The method of the application can avoid misjudgment due to local feature distortion, effectively detect various counterfeit audios, break through the generalization bottleneck, and be applied to the fields of finance and medical health.

Description

Technical Field

[0001] This invention relates to the field of audio authentication technology, and more specifically to audio authentication methods, devices, computer equipment, and storage media. Background Technology

[0002] With the deep penetration of artificial intelligence technology into various fields, the application of deep generative models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Diffusion Models in audio synthesis has become increasingly mature, driving the leapfrog development of audio generation technology. These models can accurately simulate the speech content, prosodic features, and timbre details of real audio, and the generated fake audio is so different from real audio at the auditory perception level that it is difficult to distinguish them intuitively.

[0003] However, the sophistication of audio spoofing technology has also triggered serious security risks and a crisis of trust, particularly in critical sectors such as finance and healthcare, where its negative impact is even more pronounced. In the financial sector, forged audio can be used to impersonate account holders to issue transfer instructions and forge customer authorization recordings, posing a direct threat to fund security and the order of financial transactions. In healthcare, fake medical voice recordings and remote consultation audio can interfere with diagnostic results, mislead treatment plans, and even trigger medical disputes and safety incidents. Furthermore, in areas such as news dissemination, forged audio can undermine the authenticity of information, leading to the spread of fake news and severely impacting social trust systems and public order. Audio security protection has become a crucial issue that urgently needs to be addressed.

[0004] To address the aforementioned challenges of audio spoofing, various voice authentication methods have been developed in related technical fields, which can be mainly summarized into the following two technical approaches:

[0005] Firstly, there's the method of using hand-crafted features combined with classifiers for audio authentication. This approach involves manually designing acoustic features for audio (such as Mel-frequency cepstral coefficients and linear prediction coefficients), and then using traditional classifiers like support vector machines and random forests to learn and classify these features, thus distinguishing between real and fake audio. Its core advantage lies in its simple implementation logic and low computational and time costs for model training, leading to its application in early audio authentication scenarios. However, this method has significant drawbacks: it relies excessively on human experience for feature design, making it difficult to cover high-dimensional and complex audio features. Especially for fake audio generated by novel deep generative models, whose feature distribution differs fundamentally from traditional fake audio, this method suffers from extremely poor generalization ability, failing to effectively capture subtle differences between real and novel fake audio, resulting in a significant drop in authentication accuracy.

[0006] Secondly, there are deep learning-based anti-counterfeiting methods. These methods use deep learning models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers as their core. They adaptively learn high-dimensional feature representations directly from the original audio signal or its transformed feature maps (such as Mel spectrograms), and then classify the audio as genuine or fake based on the learned features. Compared to manual feature methods, their advantage lies in the fact that they do not require manual intervention in feature design and can autonomously mine potential, complex discriminative features in the audio. Their anti-counterfeiting performance on specific datasets is superior to traditional methods. However, these methods still have key technical bottlenecks: on the one hand, the complex model structure leads to high computational complexity and high hardware computing power requirements, making it difficult to meet the needs of real-time anti-counterfeiting scenarios; on the other hand, the model is extremely sensitive to interference factors in the audio signal. When the audio is compressed, cropped, or mixed with environmental noise, the learned features will be distorted, and the anti-counterfeiting performance will significantly decrease. More importantly, the generalization ability of this type of method has inherent limitations—the model performance is highly dependent on the distribution of the training dataset. If the training set does not contain fake audio samples of a certain type of new generative model, when faced with such unknown fake audio, the model cannot effectively adapt to its feature distribution, and the fake detection accuracy will drop sharply, making it difficult to cope with the constantly iterative and updated audio fake technology.

[0007] In summary, existing voice authentication methods have insurmountable shortcomings in terms of generalization ability, robustness, and adaptability, and cannot meet the high reliability requirements of audio authentication technology in key fields such as finance and healthcare. There is an urgent need for an audio authentication technology that can break through the current technical bottlenecks and has strong generalization ability and high robustness to effectively address the audio forgery challenges brought about by deep generative models. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide an audio authentication method, device, computer equipment, and storage medium.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] Audio authentication methods include:

[0011] Acquire the audio data to be detected;

[0012] Extract the mel spectrum features of the audio data to be detected;

[0013] The mel spectrum features are input into the flow matching model to perform an inverse operation on the mel spectrum features and to map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables.

[0014] The log-likelihood value of the audio data to be detected is calculated based on the potential noise variables.

[0015] The log-likelihood value is compared with a preset threshold to determine whether the audio data to be detected is real audio.

[0016] The present invention also provides an audio authentication device, comprising:

[0017] The acquisition unit is used to acquire the audio data to be detected;

[0018] The extraction unit is used to extract the mel spectrum features of the audio data to be detected;

[0019] The input execution unit is used to input the mel spectrum features into the flow matching model to perform inverse operations on the mel spectrum features and to reverse map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables.

[0020] The calculation unit is used to calculate the log-likelihood value of the audio data to be detected based on the potential noise variables;

[0021] The comparison and determination unit is used to compare the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio.

[0022] The present invention also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method.

[0023] The present invention also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0024] The advantages of this invention compared to existing technologies are as follows: By selecting MEL spectrum features as model input, the MEL spectrum features themselves possess nonlinear mapping characteristics to audio frequencies, which can preserve the core acoustic features of the audio while suppressing redundant interference caused by irrelevant noise and compression distortion. Compared to the original audio signal or other features, it has stronger anti-interference capabilities against interference factors. The flow matching model accurately captures the probability distribution of real audio through training. Its inverse mapping and log-likelihood calculation process is essentially based on the overall distribution law of real audio to judge whether the audio to be detected belongs to the distribution, rather than relying on local, easily interfered details. Even if the audio to be detected is compressed or mixed with noise, as long as its core acoustic features still partially retain the distribution trend of real audio, the model can still accurately identify its "real distribution belonging" through the calculation of the log-likelihood value. If interference causes the audio to deviate significantly from the real distribution, it will be judged as a forgery with a lower log-likelihood value, avoiding the limitations of existing methods due to local features. The problem of distortion and misjudgment is addressed. Furthermore, by learning only the probability distribution of real audio through the flow matching model, the anti-spoofing process does not need to distinguish between the "real" and "fake" binary categories. Instead, it transforms the problem into an out-of-distribution detection problem: "Does the audio to be detected conform to the real audio distribution?" This involves extracting the mel spectrum features of the audio to be detected, inputting them into the flow matching model to perform an inverse operation, mapping them back to the prior noise distribution space to obtain a latent noise variable, and then calculating the log-likelihood value based on this variable to characterize the probability that the audio to be detected belongs to the real distribution. This technical approach allows the model to be trained without relying on any fake audio samples, covering all fake audio that deviates from the real audio distribution using only the real audio distribution. Regardless of whether the fake audio is generated by a generative adversarial network, variational autoencoder, or diffusion model, as long as its distribution differs from the real audio, it will show a significantly lower log-likelihood value, thus achieving effective detection of various types of fake audio and overcoming the generalization bottleneck of existing methods that "cannot detect unseen fake samples."

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating an application scenario of the audio authentication method provided in this embodiment of the invention.

[0028] Figure 2 This is a flowchart illustrating the audio authentication method provided in an embodiment of the present invention.

[0029] Figure 3 A schematic block diagram of an audio authentication device provided in an embodiment of the present invention;

[0030] Figure 4 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0034] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram illustrating an application scenario of the audio authentication method provided in this embodiment of the invention. Figure 2 This is a schematic flowchart illustrating the audio authentication method provided in an embodiment of the present invention. The audio authentication method is applied to a server that interacts with a terminal, avoiding the misjudgment problem caused by local feature distortion in existing methods. Furthermore, it can effectively detect various types of fake audio, overcoming the generalization bottleneck of existing methods that cannot detect unseen fake samples.

[0035] Figure 2 This is a flowchart illustrating the audio authentication method provided in an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S150.

[0036] S110. Obtain the audio data to be detected;

[0037] Specifically, the audio data to be detected originates from target application scenarios such as finance and healthcare. Examples include customer call recordings and transfer instruction voices in the financial sector, and remote consultation audios and medical order recordings in the medical field. The data format supports common audio formats (such as WAV and MP3). If it is a compressed format (such as MP3), it must first be converted to uncompressed PCM format using conventional decoding tools to ensure that the original audio signal information is not lost. Furthermore, the acquired audio data undergoes preprocessing, including removing silent segments at the beginning and end of the audio (by setting an audio energy threshold to filter out silent portions with energy below the threshold) and standardizing the audio sampling rate (set to 16kHz; if the original audio sampling rate is different, it is adjusted using interpolation or resampling techniques) to ensure the consistency of the input data and avoid affecting the accuracy of subsequent feature extraction due to differences in format or sampling rate.

[0038] S120. Extract the mel spectrum features of the audio data to be detected;

[0039] Specifically, the librosa audio processing library or professional audio analysis tools are used to extract mel spectrum features according to industry standard procedures.

[0040] The first step is to perform frame segmentation on the preprocessed audio signal, with the frame length set to 20-30ms (e.g., 25ms) and the frame shift set to 10-15ms (e.g., 10ms). The Hanning window function is used to reduce the spectral leakage of the signal between frames.

[0041] The second step is to perform a Fast Fourier Transform (FFT) on each frame of signal to convert the time-domain signal into a frequency-domain signal and obtain the spectral information in the frequency range of 20Hz-8kHz (the main frequency band of human speech).

[0042] The third step is to filter the frequency domain signal based on the MEL scale filter bank (usually 40-80 filters, such as 64), converting the linear frequency axis into a MEL frequency axis that conforms to the characteristics of human hearing, and thus obtaining the MEL spectrum.

[0043] The fourth step is to take the logarithm of the mel spectrum to obtain the logarithmic mel spectrum (i.e., mel spectrum feature). This feature can effectively compress the dynamic range of the spectrum, highlight the key acoustic information of speech (such as timbre and prosodic features), and suppress irrelevant noise interference.

[0044] In one embodiment, extracting the mel spectrum features of the audio data to be detected includes:

[0045] The audio data to be detected is pre-emphasized, framed, and windowed to obtain a pre-processed audio signal;

[0046] Specifically, the purpose of pre-emphasis processing is to compensate for the attenuation of high-frequency components in the audio signal during propagation or acquisition (the high-frequency energy of human speech signals is usually lower than that of low frequencies and is easily affected by noise), enhance the acoustic features of high-frequency bands (such as key identification information such as fricatives and voiceless consonants in speech), and improve the accuracy of subsequent feature extraction.

[0047] Preemphasis is achieved using a first-order linear high-pass filter, with the filter transfer function being H(z) = 1 - αz. -1 The attenuation coefficient α ranges from 0.93 to 0.97 (preferably 0.95, which is the optimal empirical value in the field of speech signal processing, effectively enhancing high frequencies while preserving low-frequency information). The discrete signal x(n) of the audio data to be detected is input into the filter, and the pre-emphasized signal y(n) = x(n) - αx(n-1) is output, where n is the discrete time point.

[0048] The purpose of frame segmentation is to divide a continuous audio signal into a short frame sequence. Since the audio signal is a non-stationary signal, it can be approximated as a stationary signal within a short time range (such as 20-30ms). After framing, the time-frequency domain variation characteristics of the signal can be captured more accurately, avoiding feature distortion caused by the failure of the long-term signal stationarity assumption.

[0049] An overlapping framing strategy is adopted, with a frame length set to 20-30ms (preferably 25ms, balancing time and frequency resolution: 25ms can completely cover one pitch cycle of the speech while avoiding excessive signal changes within the frame); the frame shift (the overlapping portion of two adjacent frames) is set to 10-15ms (preferably 10ms, with an overlap rate of approximately 40%, reducing inter-frame information loss and avoiding redundant calculations). If the sampling rate of the audio to be detected is 16kHz, then each frame contains 16000 × 0.025 = 400 sampling points, and the number of sampling points corresponding to the frame shift is 16000 × 0.01 = 160. The pre-emphasized signal y(n) is segmented by segment through a sliding window to obtain the frame sequence y1(n), y2(n), ..., y K (n) (K is the total number of frames).

[0050] The purpose of windowing is to suppress spectral leakage caused by signal truncation during framing (i.e., energy diffusion of the signal in the frequency domain, resulting in blurred spectral characteristics), so as to make the two ends of the signal within the frame transition smoothly and improve the spectral accuracy of the subsequent short-time Fourier transform.

[0051] The Hanning window is chosen as the window function, and its expression is w(n) = 0.5 - 0.5cos(2πn / (N-1)) (where N is the number of sampling points per frame, such as 400 points). The signal y of each frame... kThe frame signal z is obtained by multiplying n (k=1,2,...,K) with the Hanning window function w(n) point by point. k (n)=y k (n)×w(n), all windowed frame signals together constitute the preprocessed audio signal.

[0052] The preprocessed audio signal is subjected to a short-time Fourier transform to obtain a spectrogram;

[0053] Specifically, the preprocessed time-domain frame signal is converted into a frequency-domain signal to obtain the frequency components and corresponding energy distribution of each frame signal, laying the foundation for subsequent extraction of MEL frequency scale features.

[0054] For each frame signal z in the preprocessed audio signal k (n) Perform Short-Time Fourier Transform (STFT):

[0055] The FFT (Fast Fourier Transform) points are set to 512 (greater than the 400 sampling points per frame, and zero-padding is used to ensure that the frequency domain resolution reaches 16000 / 512≈31.25Hz, which can clearly distinguish the frequency components of speech).

[0056] For each frame z k (n) Calculate the FFT to obtain the frequency domain signal Z in complex form. k (m) (m=0,1,...,255, taking the first half of the FFT result, since the FFT result of the real signal is conjugate symmetric, the second half has no new information).

[0057] The amplitude spectrum of the frequency domain signal for each frame is calculated using the formula |Z k (m)|= (Re is the real part, Im is the imaginary part), and then the power spectrum P is obtained by squaring the amplitude spectrum. k (m)=|Z k (m)| 2 ;

[0058] Arrange the power spectra of all frames in chronological order to form a two-dimensional spectrum (the horizontal axis represents the time frame number, the vertical axis represents the frequency point m, and the pixel value is the power value of the corresponding time-frequency point).

[0059] The spectrum is filtered through a MEL filter bank to extract energy features at the MEL frequency scale, forming MEL spectrum features.

[0060] Specifically, the spectrum of the linear frequency scale is converted into the mel frequency scale, which conforms to the characteristics of the human auditory system. Human hearing has high frequency resolution for low-frequency signals and low resolution for high-frequency signals. The mel frequency scale can simulate this characteristic, highlighting the low-frequency features in speech that are more important to human perception, while compressing high-frequency redundant information and reducing the input dimension and computational complexity of the subsequent stream matching model.

[0061] Construct a MEL filter bank: Design 40-80 (ideally 64, balancing feature dimensionality and information preservation: 64 filters can fully cover the key frequency range of speech without increasing the model training burden due to excessive dimensionality) MEL filters in the form of triangular window functions. The center frequencies of the filters are uniformly distributed according to the MEL frequency scale. The conversion formula between MEL frequency and linear frequency is mel(f) = 2595log 10 (1+f / 700) (f is the linear frequency in Hz), the frequency range of the filter covers 20Hz-8kHz (the main frequency range of human speech), and the 3dB bandwidth of adjacent filters overlaps (the overlap rate is about 50%, which ensures smooth frequency transition and no information loss).

[0062] Filtering and energy calculation: Calculate the power spectrum P of each frame in the spectrogram. k (m) is multiplied point-by-point by each filter in the mel filter bank, and then the product results of each filter are summed to obtain the energy value E of each frame at each mel frequency point. k (l)=∑ m P k (m)×H l (m)(l=1,2,...,64,H l (m) represents the frequency response of the l-th mel filter.

[0063] Logarithmic compression and mel spectrum formation: mel energy value E for each frame k (l) Take the natural logarithm (the formula is log(E)). k (l)+ϵ),ϵ=10 -8 To avoid the logarithm being meaningless when the energy value is 0, the logarithmic MEL energy feature of each frame is obtained. The logarithmic MEL energy features of all frames are arranged in chronological order to form a two-dimensional MEL spectrum feature (the horizontal axis is the time frame number, the vertical axis is the MEL frequency point number, and the pixel value is the logarithmic MEL energy value). This feature is the core feature of the subsequent input stream matching model.

[0064] In other words, pre-emphasis processing enhances the discriminative features of high-frequency segments of speech (such as the high-frequency energy of voiceless consonants), which are key to distinguishing real audio from fake audio. Deep generative models (such as GANs and Diffusion Models) often suffer from high-frequency detail distortion due to limitations in their generation mechanisms when synthesizing audio. Pre-emphasis amplifies this difference. Framing and windowing ensure the time-frequency domain accuracy of the features, avoiding feature ambiguity caused by long-term signal non-stationarity. This allows the mel spectrum to accurately capture the spectral differences between real and fake audio within short frames (such as the stability of the fundamental frequency of real speech and the irregular spectral fluctuations of fake speech). The nonlinear frequency mapping of the mel filter bank simulates human auditory characteristics, highlighting low-frequency features that are more important for speech discrimination (such as the fundamental frequency of vowels and the position of formants). These features have stable physiological acoustic laws in real audio, which are difficult to reproduce accurately in fake audio. This gives the mel spectrum features a stronger ability to distinguish between real and fake audio, laying the foundation for subsequent stream matching models to achieve high-precision fake detection through inverse operations and log-likelihood calculations. Furthermore, pre-emphasis processing reduces the masking of high-frequency features by environmental noise (such as background noise in financial settings and equipment noise in medical settings) by compensating for high-frequency attenuation, making high-frequency identification information clearer; the windowing process of the Hanning window suppresses spectral leakage and avoids spectral distortion caused by frame truncation. Even if the audio to be detected is compressed (such as MP3 compression, which will cause some frequency components to be lost), the windowed spectrum can still retain the core frequency features; the logarithmic compression operation of the mel spectrum features reduces the dynamic range of the signal and reduces the energy fluctuation of the audio signal caused by differences in acquisition equipment (such as different microphone sensitivity) or transmission loss, making the features more stable.

[0065] S130. Input the mel spectrum features into the flow matching model to perform the inverse operation on the mel spectrum features and reverse map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables.

[0066] Specifically, the prerequisite for the stream matching model is that the model must be trained in advance using real audio data. The training process is as follows:

[0067] Data preparation: Collect massive amounts of real audio data (such as compliant recordings in the financial field and standard medical consultation audio), and divide them into training set, validation set and test set in a 7:2:1 ratio; use only the training set data, and extract MEL spectral features as training samples according to the above steps.

[0068] Model training: Randomly sample real audio mel spectrum samples, time points t (0≤t≤1), and standard Gaussian noise samples from the training set. Construct the midpoint of time t through linear interpolation. Train the neural network to predict the velocity field, making the predicted velocity field approximate the target velocity field (the target velocity field is a constant, defined by the difference between the real sample and the noise sample). Optimize the model using the mean squared error loss function (calculating the average error between the predicted velocity field and the target velocity field) until the model can accurately describe the flow path of "standard Gaussian distribution → real audio distribution". Then, select the best model through the validation set and verify the model performance through the test set.

[0069] Reverse operation implementation:

[0070] Input the mel spectrum features of the audio to be detected into the trained stream matching model, and use the mel spectrum features as the starting point for the reverse operation (corresponding to the state at time t=0).

[0071] Discretize the time interval [0, 1]: Set N time steps (e.g., 100 time steps) to obtain the time series t0=0, t1..., t N =1;

[0072] Numerical integration to solve the inverse ODE: For each time step t1, the model predicts the velocity field at the current time and uses the Euler numerical method to solve the inverse ordinary differential equation (ODE). After N steps of inverse calculation, the state at time t=1 is obtained. This state is the potential noise variable z (z follows a standard Gaussian distribution) that is mapped back from the mel spectrum features of the audio data to be detected to the prior noise distribution space.

[0073] In one embodiment, the training process of the flow matching model includes:

[0074] Collect a certain amount of real audio data and preprocess the collected real audio data to obtain audio frames suitable for training the stream matching model.

[0075] Specifically, the focus is on target application scenarios such as finance and healthcare, collecting authentic audio data that matches the characteristics of those scenarios. For example, in the financial sector, compliant customer call recordings and account operation voice commands can be collected; in the healthcare sector, standard remote consultation audio and medical order recordings can be collected. The data needs to cover different speakers (such as human speech of different ages, genders, and accents), different collection devices (such as mobile phone microphones and professional recording equipment), and different environments (such as quiet offices and slightly noisy business halls) to ensure data diversity and avoid model overfitting. The total amount of data needs to meet the training requirements of the stream matching model, preferably collecting more than 100,000 hours of labeled authentic audio (the labeling information only needs to confirm the "authentic" attribute, without distinguishing between forged types).

[0076] Format uniformity: The collected audio data is uniformly converted into uncompressed WAV format, the sampling rate is set to 16kHz (the standard sampling rate for speech signal processing, which can completely preserve the core frequency range of human speech from 20Hz to 8kHz), and the quantization bit depth is set to 16bit to ensure data format consistency.

[0077] Audio data preprocessing:

[0078] Silence Removal: The energy threshold method is used to remove silent segments at the beginning and end of the audio. The energy threshold is set to 1 / 20 of the overall average energy of the audio. When the energy of the audio is lower than this threshold for 100ms, it is judged as silent and is cut off to avoid interference from silent segments to feature extraction.

[0079] Pre-emphasis, framing, and windowing: Referring to the preprocessing logic of mel spectrum feature extraction, pre-emphasis is performed using a first-order linear high-pass filter (attenuation coefficient α=0.95) to compensate for high-frequency attenuation; an overlapping framing strategy is adopted, with a frame length of 25ms (corresponding to 400 sampling points at a sampling rate of 16kHz) and a frame shift of 10ms (corresponding to 160 sampling points); Hanning windowing is used for windowing to suppress spectral leakage, ultimately resulting in a discrete audio frame sequence, with each audio frame serving as an independent basic unit for subsequent feature extraction.

[0080] Extract mel spectrum features from audio frames to form a mel spectrum feature dataset;

[0081] Specifically, a short-time Fourier transform (STFT) and mel filtering operation are performed on each preprocessed audio frame:

[0082] STFT processing: Set the FFT point count to 512 points (zero-filled to 512 points, frequency domain resolution approximately 31.25Hz), and calculate the power spectrum of each audio frame;

[0083] Mel filtering: Construct a Mel filter bank of 64 triangular windows, covering the frequency range of 20Hz-8kHz. Pass the power spectrum through the Mel filter bank and sum it to obtain the energy value of each Mel frequency point.

[0084] Logarithmic compression: taking the natural logarithm of the mel energy value (plus ε=10). -8 (To avoid logarithmic inconsistency), we obtain single-frame mel spectrum features with a dimension of 64.

[0085] Dataset Construction: The mel spectrum features of all audio frames are arranged in order to form a mel spectrum feature dataset. Each data sample consists of "single frame mel spectrum feature + real label" (the label is only marked as "real", and there is no need to distinguish other categories). The dataset format adopts HDF5 or TFRecord to facilitate efficient reading and training of the model.

[0086] The extracted mel spectrum feature dataset is divided into training set, validation set and test set according to a certain ratio;

[0087] Specifically, a 7:2:1 partitioning principle is followed, meaning the training set accounts for 70%, the validation set for 20%, and the test set for 10%. This ratio ensures that the training set is large enough to support the model learning the real audio distribution, while the validation and test sets effectively evaluate the model's generalization ability. A stratified random sampling method is used to ensure that the sample distribution of the three datasets is consistent with the original mel spectral feature dataset (e.g., the proportion of audio frames from different speakers and environments is the same), avoiding distortion of validation or test results due to sampling bias. For example, the datasets are first grouped by speaker, and then 70% are randomly selected from each group as training samples, 20% as validation samples, and 10% as test samples, ensuring a balanced distribution of samples across groups.

[0088] Initialize the flow matching model parameters and train the flow matching model using the training set;

[0089] Specifically, the core of the flow matching model is the velocity field prediction network, which employs a 3-layer fully connected network or a lightweight convolutional neural network (CNN). The input dimension is 64 (mel spectrum feature dimension), and the output dimension is 64 (velocity field vector dimension). The activation function is ReLU (to avoid gradient vanishing). The number of network layers and neurons is adjusted according to the amount of data; the preferred settings are 256, 128, and 64 neurons per layer to ensure moderate model complexity. Parameter initialization: The network weights are initialized using a He normal distribution, and the bias term is initialized to 0. The optimizer is set to Adam, with an initial learning rate of 1e. -4 The weight decay coefficient is 1e -5 (To prevent overfitting); Set the training batch size to 256 and the training epochs to 100 (which can be dynamically adjusted based on the performance of the validation set).

[0090] The model training process is as follows:

[0091] In each round of training, N real MEL spectral feature samples x (N=batch size) are randomly sampled from the training set.

[0092] Random sampling time point t (following a uniform distribution in [0,1]) and standard Gaussian noise sample z0 (with the same dimension as x);

[0093] According to the linear interpolation formula x t =t·x+(1-t)·z0, construct the midpoint xt of time t;

[0094] Target velocity field calculation: based on formula v t ∗=x-z0 (the target velocity field is a constant), determine the target velocity field for each sample;

[0095] Model prediction and loss calculation: Input xt and t into the velocity field prediction network to obtain the predicted velocity field vt; use the mean square error loss function L=E x,z0,t [||vt-v t ∗ || 2 ] Calculate the error between the predicted velocity field and the target velocity field;

[0096] Backpropagation and parameter update: The gradient is calculated through backpropagation, and the network weights are updated using the Adam optimizer to minimize the loss function until the end of the training rounds or the loss function converges (e.g., the loss decreases by less than 1e for 10 consecutive rounds). -6 ).

[0097] At the end of each training round, the detection accuracy of the model is evaluated using a validation set in order to select the target model;

[0098] Specifically, the core validation metric is "authentication accuracy," with auxiliary metrics including "True Positive Rate (TPR, the proportion of correctly identified genuine audio)" and "False Positive Rate (FPR, the proportion of fake audio incorrectly identified as genuine)." The authentication accuracy is calculated as follows: Genuine MEL spectrogram features from the validation set are input into the model, and the log-likelihood value is calculated. Simultaneously, a small number of known fake audio MEL spectrogram features (such as GAN-generated audio features) are introduced as negative samples. A temporary threshold is set (based on the mean of the log-likelihood values ​​of the genuine samples in the validation set minus one standard deviation), and the proportion of correctly identified genuine samples plus correctly identified fake samples out of the total validation samples is calculated.

[0099] Target model selection: After each round of training, calculate the validation set's fake detection accuracy, TPR, and FPR; if the fake detection accuracy of the current round model is higher than the historical best value (and TPR≥99%, FPR≤1%, meeting the high reliability requirements of key domains), then save the model parameters of that round as a candidate model; after training, select the model with the "highest validation set fake detection accuracy and best generalization" from all candidate models as the target model to avoid model overfitting.

[0100] The target model is evaluated using a test set; if the test passes, the final flow matching model is obtained.

[0101] Specifically, the test set is used only for final performance validation and does not participate in any model parameter tuning. The real mel spectrum features and fake audio mel spectrum features (novel fake audio that did not appear in the training and validation phases, such as audio generated by the Diffusion Model) in the test set are input into the target model, the log-likelihood value of all samples is calculated, and the optimal threshold determined in the validation phase is used for judgment.

[0102] The test pass criteria are as follows: The core indicators for passing the test are "test set fake detection accuracy ≥ 98%, TPR ≥ 99%, FPR ≤ 1.5%", while the model is required to have a fake detection accuracy drop of no more than 3% for compressed audio (MP3 128kbps) and noisy audio (signal-to-noise ratio 10dB) (to verify robustness); if all indicators meet the requirements, the test is considered passed, and the target model is the final stream matching model; if it fails, the test returns to the model training stage, the network structure or training parameters are adjusted (such as increasing the learning rate or increasing the number of training rounds), and the model is retrained and evaluated.

[0103] In other words, the data collection phase covers real audio from multiple scenarios and dimensions, while the preprocessing phase removes interference information and unifies the data format, enabling the training data to fully reflect the essential characteristics of real audio. The MEL spectrum feature extraction focuses on key speech identification information, avoiding redundant features from interfering with model learning. The flow matching model, through the training logic of "constructing intermediate points - predicting velocity fields - optimizing loss", accurately depicts the flow path of "standard Gaussian distribution → real audio distribution", which can more clearly and accurately capture the high-dimensional probability distribution pattern of real audio, providing a reliable model foundation for the subsequent counterfeit detection logic of "determining distribution attribution by log-likelihood value".

[0104] In one embodiment, a large amount of real audio data is collected and organized to construct a training set, a test set, and a validation set in a 7:2:1 ratio. The training set is used for model training; the validation set is used to evaluate the detection accuracy of the model after each round of training to select the target model; and the test set is used for the final evaluation of the selected best model after training is complete. Furthermore, training the model using only real audio data aims to teach the model to gradually transform from a simple Gaussian noise distribution to the distribution of real audio data. During training, the mel spectrum features of the audio are first extracted as training samples.

[0105] Flow matching is a generative model whose goal is to learn a velocity field related to time t. (The velocity field is a function that assigns a velocity vector to each point in spacetime; it can be viewed as a wind map, indicating the direction and speed at which a point should move at any given time.) This ensures that at any time point t, the data evolves along the correct trajectory. Through multiple transformations, a simple prior distribution, such as a standard Gaussian distribution, is ultimately transformed into a complex target data distribution. Any fake audio is generated by a different probability distribution. As long as it is generated So, fake samples in The probability under the distribution is very low.

[0106] The specific training process for the stream matching model is as follows:

[0107] Sample a real sample from the training set. Randomly sample a time point A noise sample that follows a standard Gaussian distribution. Construct a linear interpolation operation in Midpoint of time

[0108] (1)

[0109] Then train the model to predict the output. Approaching the real velocity field ,in:

[0110] (2)

[0111] Depend on The formula shows that the target velocity field is constant at all times. Constructed data points At any moment The magnitude and direction of the changes should be as close as possible to the target velocity field, so that a neural network Predicted velocity field During training, the system continuously tries to fit the target velocity field. The loss function used during training is...

[0112] (3)

[0113] in The model needs to learn to predict the velocity field. It is the target velocity field. This represents the error between the predicted velocity field and the target velocity field. Indicates different times Different training samples The error is averaged. This loss function calculates the mean square error between the model's predicted velocity field and the actual velocity field. During training, this loss function is continuously optimized so that the predicted velocity field gets closer and closer to the target velocity field. Finally, the model can accurately describe the "flow path" from the initial noise distribution to the target distribution.

[0114] Once the model is trained, a model that can accurately predict the velocity field is obtained. To generate a new audio sample, the model only needs to set the number of sampling steps. Get a list of times, in Randomly sample a noise sample at any time. The model predicts the velocity field at the current moment. Then solve the ODE equation. The sample state at the next time step is obtained, and then the sample state at the next time step is used as input to predict the velocity field and solve the ODE equation. The new sample is finally obtained from the calculation of the steps. It comes from the distribution .

[0115] In this method, audio authentication using stream matching does not utilize the audio's generation capabilities during the authentication process; instead, it calculates test samples. Exact log-likelihood under the true distribution defined by the flow matching model This log-likelihood represents the probability that the test sample is a real sample. Calculating this value allows us to distinguish between real and fake audio. However, this value cannot be directly calculated. Instead, we use the inverse transformation capability of the stream matching model, integrating the ODE in the reverse direction, to obtain the latent noise variable z corresponding to this sample. The inverse process is carried out by the same velocity field. definition.

[0116] (4)

[0117] At time t=0, there are initial conditions. Integrate up to time t=1 to obtain the noise code corresponding to the sample. (noise).

[0118] According to the law of conservation of probability mass, the state of the sample at time t probability density The time derivative along its transformed trajectory satisfies the formula for the instantaneous rate of change:

[0119] (5)

[0120] in Represents the velocity field exist divergence at that point, It is a vector field Jacobian matrix The trace is equal to the divergence. Divergence measures whether the velocity field diverges or converges at a point; positive divergence leads to a higher probability density function. The decrease in negative divergence leads to a decrease in probability density. rise.

[0121] We can integrate equations (4) and (5) together. Integrating both sides of equation (5) from t=0 to t=1:

[0122] (6)

[0123] The integral on the left side of formula (6) is the integral of the derivative with respect to time, which is equal to the difference between the values ​​of the integrand at the endpoints of the interval:

[0124] (7)

[0125] (8)

[0126] Given initial conditions Solve The termination condition is And z follows a standard Gaussian noise distribution. Then there is Substituting into formula (8) and rearranging the terms, we get:

[0127] (9)

[0128] Integral term in formula (9) This represents the accumulation of divergence along the flow trajectory, calculating the total "volume scaling" of the input data in the probability density space during continuous flow, used to correct the probability change from the speech distribution to the noise distribution. The Hutchinson random trajectory estimation method is used to calculate... Substituting into formula (9), we obtain the log-likelihood. This formula illustrates that, in stream matching, a complex sample The logarithmic probability density can be decomposed into a Gaussian noise distribution. The logarithmic probability density is added to the integral correction term for the velocity field divergence. Based on this, we obtain... The log-likelihood of the probability of appearing in the real audio distribution defined by this stream matching model. Real audio samples should have a higher likelihood under this model, and a threshold can be set in the fake detection task. and Make a judgment by comparison, if Above the threshold The system considers test samples to conform to the training distribution and therefore be genuine audio; otherwise, it considers samples to not conform to the true distribution and be fake audio.

[0129] In one embodiment, the step of inputting the mel spectrum features into a flow matching model to perform an inverse operation on the mel spectrum features and mapping the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables includes:

[0130] Define the time range for the reverse operation and divide it into N time steps;

[0131] Specifically, based on the forward flow path from the prior noise distribution (t=1) to the real audio distribution (t=0) during the training of the flow matching model, the time range of the reverse operation is set to [0,1], where t=0 corresponds to the mel spectrum feature of the audio to be detected (the state at the end of the real audio distribution), and t=1 corresponds to the state at the end of the target prior noise distribution. This ensures that the reverse path is completely symmetrical with the forward path of the model training, guaranteeing the accuracy of the distribution mapping.

[0132] The time range [0,1] is divided using an equally spaced discretization method. The number of time steps N is set to a range of 50-200 (preferably N=100, balancing computational accuracy and efficiency: too small a N will lead to increased numerical integration error, while too large a N will increase computation time). The formula for calculating the time step Δt is: Δt=(1-0) / N=1 / N. After the division, the time series is obtained as t0=0, t1=Δt, t2=2Δt, ..., t N =1, each time step t i This corresponds to an intermediate state calculation node in the reverse process.

[0133] For each time step, the current MEL spectrum feature state value and the corresponding time input are fed into the flow matching model to obtain the predicted velocity field at the current time.

[0134] Specifically, the input data is constructed as follows: the mel spectrum feature state value at the initial moment of the inverse operation (t=0) is the mel spectrum feature x of the audio to be detected. t0 =x (the dimension is consistent with the MEL spectral features during model training, such as 64 dimensions); for subsequent time steps t i (i≥1), the current state value is the intermediate state x obtained from the integration in the previous step. ti Set the current state value x ti With the corresponding time t i The vectors are concatenated to form the model input vector (64+1=65 dimensions) to ensure that the model can predict the dynamic velocity field by incorporating time information.

[0135] Velocity field prediction logic: The core of the flow matching model is a trained velocity field prediction network (such as a 3-layer fully connected network). This network has learned the inverse flow law of "real audio distribution → prior noise distribution" through training. After the input vector is fed into the network, the predicted velocity field f(x) at the current time is output. ti , t i (The velocity field vector is 64-dimensional, consistent with the state value dimension), indicating the current state x. ti At time step t i The "direction and speed" that should move in the direction of the prior noise distribution.

[0136] The inverse ordinary differential equation is solved by numerical integration to calculate the state value at the next time step;

[0137] Specifically, for each time step Predicting the velocity field Solving the inverse ODE using the Euler method:

[0138] (10).

[0139] Repeat the velocity field prediction-integration calculation operation until all N time steps are calculated and the final state value is obtained;

[0140] Specifically, after N steps of calculation, the state at time t=1 is obtained. That is, the final state value.

[0141] The final state value is verified; if the final state value conforms to the standard Gaussian distribution characteristics, then the final state value is determined as the potential noise variable corresponding to the mel spectrum feature of the audio data to be detected.

[0142] Specifically, if the final state value meets all the verification indicators, it is confirmed that it conforms to the prior noise distribution characteristics and is identified as the potential noise variable z corresponding to the mel spectrum feature of the audio to be detected; if the final state value does not meet any of the verification indicators (such as mean = 0.5, standard deviation = 1.5 or p value < 0.05), the audio to be detected is directly determined to be fake audio (because the real audio should strictly conform to the prior noise distribution after reverse mapping, and the fake audio will not conform to the Gaussian property due to the distribution deviation).

[0143] S140. Calculate the log-likelihood value of the audio data to be detected based on the potential noise variables;

[0144] Specifically, based on the principle of conservation of probability mass and combined with the velocity field divergence characteristics of the flow matching model, the log-likelihood value is derived.

[0145] In one embodiment, calculating the log-likelihood value of the audio data to be detected based on the latent noise variable includes:

[0146] The corresponding standard Gaussian log probability density is calculated based on the potential noise variable to obtain the probability density value;

[0147] Specifically, the probability density value is calculated according to formula (4).

[0148] The probability density value and the potential noise variable are coupled and integrated to obtain the velocity field divergence integral term;

[0149] Specifically, the velocity field divergence integral term is calculated according to formulas (5) and (6).

[0150] The log-likelihood value of the audio data to be detected is obtained by calculating based on the probability density value and the velocity field divergence integral term.

[0151] Specifically, it is known that z follows a standard Gaussian noise distribution. We obtain:

[0152] (11)

[0153] For Dimension. Calculate the integral term of formula (9), and then substitute it into formula (9) to obtain the log-likelihood. .

[0154] S150. The log-likelihood value is compared with a preset threshold to determine whether the audio data to be detected is real audio.

[0155] Specifically, using validation set data (containing known real audio and fake audio), the log-likelihood values ​​of all validation set samples are calculated. The log-likelihood distribution of real audio samples (usually normally distributed with a mean of μ1) and the log-likelihood distribution of fake audio samples (mean μ2, and μ2 < μ1) are statistically analyzed. In addition, ROC curve analysis is used to select a value that maximizes the true positive rate (the proportion of correctly identifying real audio) and minimizes the false positive rate (the proportion of fake audio being mistaken for real audio) as the preset threshold T. For example, in the financial field, where it is necessary to strictly control the false positive rate, the threshold can be set slightly lower than the lower limit of the log-likelihood distribution of real audio to ensure that the risk of missing real audio is extremely low.

[0156] Judgment rule: If the log-likelihood value of the audio to be detected is greater than T: the audio is determined to conform to the real audio distribution and is real audio; if the log-likelihood value of the audio to be detected is less than or equal to T: the audio deviates from the real audio distribution and is fake audio.

[0157] In one embodiment, comparing the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio includes:

[0158] The preset threshold is determined based on the statistical analysis of the validation set used during the training of the flow matching model.

[0159] Specifically, all real samples and fake samples in the validation set are input into the trained flow matching model. Following the "reverse operation - log-likelihood calculation" process, the log-likelihood value of each sample is obtained, forming a "real sample likelihood value set" and a "fake sample likelihood value set". Statistical analysis is then performed on the two sets: the mean and standard deviation of the real sample likelihood value set are calculated, and the mean and standard deviation of the fake sample likelihood value set are calculated. Finally, a histogram of the likelihood value distribution of the two sets is plotted to observe the overlapping areas of the distribution.

[0160] Choose a threshold that maximizes the true positive rate (TPR, the proportion of correctly identified genuine audio) and minimizes the false positive rate (FPR, the proportion of fake audio being mistaken for genuine audio), balancing the accuracy of counterfeit detection with the needs of key areas (e.g., strict control of FPR in the financial field, and a balance between TPR and FPR in the medical field). Use the ROC curve (Receptor Operating Characteristic) analysis method, with the threshold as the variable, to traverse the possible range of likelihood values ​​for the validation set (e.g., -130 to -70), calculate the TPR and FPR corresponding to each threshold, and plot the ROC curve. Find the point on the ROC curve that is closest to the top left corner (i.e., the point with the largest Youden index, Youden index = TPR - FPR), and the likelihood value corresponding to this point is the preset threshold T.

[0161] Retrieve the log-likelihood value and a preset threshold, and perform a numerical comparison:

[0162] Specifically, the log-likelihood value of the audio to be detected is retrieved from the inference process of the stream matching model; a pre-determined threshold T is retrieved from the model parameter storage module to ensure that the threshold is completely matched with the current version of the stream matching model (to avoid threshold invalidation due to model updates). A direct numerical comparison method is adopted, and since the log-likelihood value is a continuous value, no additional data conversion is required.

[0163] If the log-likelihood value is greater than the preset threshold, it is determined that the log-likelihood value of the audio data to be detected conforms to the true audio distribution characteristics defined by the stream matching model, that is, the audio data to be detected is true audio.

[0164] If the log-likelihood value is less than or equal to a preset threshold, the log-likelihood value of the audio data to be detected is determined to deviate from the true audio distribution characteristics, that is, the audio data to be detected is fake audio.

[0165] Specifically, since the model has only learned the distribution of real audio data, a real audio sample will fit this distribution well, resulting in a larger calculated log-likelihood value. Conversely, a fake audio sample has a data distribution that deviates from the real distribution, leading to a lower calculated log-likelihood. A threshold is set: if the log-likelihood is greater than this threshold, it is considered real audio; otherwise, it is considered fake audio. This threshold is determined statistically by analyzing the distribution of real and fake speech on the test set within the validation set, ensuring that this method achieves optimal detection accuracy.

[0166] In one embodiment, after comparing the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio, the method further includes:

[0167] The system provides feedback and visualization of the authentication results, and generates authentication audit logs.

[0168] Specifically, a visual interface is developed based on a web-based or client-side platform (such as a financial risk control system client or a medical audio management platform), employing an intuitive and easy-to-understand presentation format to adapt to the needs of users in different scenarios (such as financial risk control personnel needing to quickly identify risks, and medical management personnel needing to conduct compliance reviews). The specific content and format of the presentation include:

[0169] Results Overview Panel: The core information is displayed in a card layout. The "Real Audio" card has a green background and a checkmark icon, while the "Fake Audio" card has a red background and a warning icon. The audio ID, judgment result, log-likelihood value and preset threshold are highlighted in the card, making it easy for users to quickly obtain key conclusions.

[0170] Likelihood comparison chart: A bar chart or line chart is used to show the comparison between the log likelihood value of the audio to be detected and the preset threshold. The horizontal axis is the "data type" (log likelihood value, preset threshold) and the vertical axis is the "numerical size". The distribution range of the likelihood value is also marked (e.g., "real audio likelihood value range: -92~-75" "fake audio likelihood value range: -130~-92"), which intuitively reflects the distribution of the audio to be detected.

[0171] Audio Feature Auxiliary View: The MEL spectrum of the audio to be detected is visualized (the horizontal axis represents time frames, the vertical axis represents MEL frequency points, and the color intensity represents logarithmic energy values), and compared with the MEL spectrum template of real audio (typical real audio MEL spectrum features extracted from the training set). The difference areas are marked (such as the high-frequency energy abnormal fluctuation areas that may exist in fake audio), providing feature-level auxiliary evidence for the judgment result.

[0172] Based on the standardized result dataset, audit logs that comply with industry standards (such as the "Audio Authentication Audit Standard" in the financial sector and the "Medical Audio Data Management Measures" in the medical field) are generated. In addition to containing all information from the result data processing stage, the logs include supplementary audit-related fields. The audit logs are in CSV or TXT format (for easy archiving and retrieval), with each log entry corresponding to one audio authentication record.

[0173] For example, in the financial sector, audio authentication technology can be widely used in scenarios involving audio interaction, such as telephone customer service, voice command transactions, and remote identity verification, to ensure transaction security, verify customer identities, and prevent fraud caused by forged audio.

[0174] With the development of fintech, many banks and financial institutions offer remote voice account opening services, where customers communicate with customer service representatives by phone and follow voice prompts to complete the account opening process. However, criminals may use audio spoofing technology to simulate customer voices and open fraudulent accounts, posing a risk to financial institutions. Specific applications are as follows:

[0175] During the remote account opening process, financial institutions' systems automatically record the audio of the conversation between the customer and customer service personnel as audio data to be tested. For example, when a customer calls the bank's remote account opening hotline and answers relevant questions according to the voice prompts, such as their name, ID number, and contact information, the entire call is recorded.

[0176] The acquired audio data to be detected undergoes pre-emphasis, framing, and windowing processing to obtain a preprocessed audio signal. For example, a pre-emphasis filter is used to enhance the high-frequency components of the audio signal, making the signal spectrum flatter and facilitating subsequent analysis; the audio signal is divided into frames of fixed duration, and each frame is windowed to reduce spectral leakage. A short-time Fourier transform is performed on the preprocessed audio signal to obtain a spectrogram. The Fourier transform converts the time-domain signal to a frequency-domain signal, allowing analysis of the energy distribution of the audio at different frequencies. The spectrogram is then filtered through a MEL filter bank to extract energy features at the MEL frequency scale, forming MEL spectral features. The MEL filter bank simulates the human ear's frequency perception characteristics, making the extracted features more consistent with the human auditory system.

[0177] The extracted MEL spectral features are input into a pre-trained flow matching model. The model performs an inverse operation on the MEL spectral features, defining a time range for the inverse operation and dividing it into N time steps. For each time step, the current MEL spectral feature state value and the corresponding time are input into the flow matching model to obtain the predicted velocity field at the current time. For example, in the first time step, the initial MEL spectral feature state and the corresponding time are input into the model, and the model calculates the predicted velocity field at that time based on its internal parameters and algorithm. The inverse ordinary differential equation is solved using numerical integration to calculate the state value at the next time step. The velocity field prediction-integration calculation operation is repeated until all N time steps are completed, and the final state value is obtained. The final state value is verified. If the final state value conforms to the standard Gaussian distribution characteristics, it is identified as the potential noise variable corresponding to the MEL spectral features of the audio data to be detected.

[0178] The log probability density of the corresponding standard Gaussian distribution is calculated based on the latent noise variable to obtain the probability density value. For example, the probability density function formula of the standard Gaussian distribution can be used to substitute the latent noise variable into the calculation to obtain the corresponding probability density value. The probability density value and the latent noise variable are coupled and integrated to obtain the velocity field divergence integral term. Based on the probability density value and the velocity field divergence integral term, the log-likelihood value of the audio data to be detected is calculated.

[0179] The preset threshold is determined based on the validation set used during the training of the stream matching model. For example, by analyzing a large number of validation sets of real and fake audio data, the range of log-likelihood values ​​that can distinguish between real and fake audio is statistically determined, thereby determining an appropriate preset threshold. The log-likelihood value is retrieved and compared with the preset threshold. If the log-likelihood value is greater than the preset threshold, the log-likelihood value of the audio data to be detected is determined to conform to the real audio distribution characteristics defined by the stream matching model, that is, the audio data to be detected is real audio, and the customer is allowed to continue to complete the account opening process; if the log-likelihood value is less than or equal to the preset threshold, the log-likelihood value of the audio data to be detected is determined to deviate from the real audio distribution characteristics, that is, the audio data to be detected is fake audio, the system immediately terminates the account opening process, and triggers a risk warning mechanism.

[0180] Finally, the authentication results are fed back and visualized. For example, the authentication results are displayed in a prominent color (e.g., red for fake audio, green for genuine audio) on the customer service interface, along with detailed prompts informing customer service personnel of the handling procedures. An authentication audit log is generated, recording detailed information such as the authentication time, audio file information, log-likelihood value, preset threshold, and authentication results for subsequent auditing and traceability.

[0181] In other words, by applying audio authentication technology, financial institutions can effectively identify forged audio during remote voice account opening processes, preventing criminals from using fake voices to open fraudulent accounts and reducing financial risks. At the same time, it improves the security and reliability of the account opening process, enhancing customer trust in financial institutions. Furthermore, the feedback of authentication results and the generation of audit logs help financial institutions improve their risk management systems and enhance their ability to respond to fraudulent activities.

[0182] For another example, in the healthcare field, audio authentication technology can be applied to scenarios such as remote medical consultations, voice-based medical order recording, and voice interaction with medical devices. In these scenarios, the authenticity of the audio directly affects the accuracy of medical diagnosis, the correctness of medical order execution, and the security of patient-medical device interaction.

[0183] With the development of telemedicine technology, patients can consult with doctors remotely via telephone or video call. Doctors make diagnoses and provide treatment suggestions based on patients' voice descriptions of symptoms, medical history, and other information. However, some criminals may forge patient audio, providing false symptom information to obtain inappropriate medical resources or maliciously interfere, which poses a serious threat to the quality of medical care and patient safety. Specific applications are as follows:

[0184] During telemedicine consultations, the medical platform automatically records the audio of the patient-doctor conversation as audio data to be analyzed. For example, when a patient logs into the telemedicine consultation system using a mobile phone or computer and has a video or voice call with a doctor, the system records the entire call in real time in the background.

[0185] The acquired audio data to be detected undergoes pre-emphasis, framing, and windowing processing to obtain a preprocessed audio signal. Pre-emphasis enhances the high-frequency components of the audio signal, making the signal spectrum flatter; framing divides the continuous audio signal into frames of fixed duration, facilitating subsequent analysis; windowing reduces spectral leakage. For example, a first-order high-pass filter is used for pre-emphasis, dividing the audio signal into 20-30ms frames, and a Hamming window is used for windowing. A short-time Fourier transform is performed on the preprocessed audio signal to obtain a spectrogram. The Fourier transform converts the time-domain signal to a frequency-domain signal, allowing analysis of the energy distribution of the audio at different frequencies. The spectrogram is then filtered through a Mel filter bank to extract energy features at the Mel frequency scale, forming Mel spectral features. The Mel filter bank simulates the human ear's frequency perception characteristics, making the extracted features more consistent with the human auditory system.

[0186] The extracted MEL spectrum features of the patient's consultation audio are input into a trained flow matching model. The time range for the inverse operation is defined and divided into N time steps. For each time step, the current MEL spectrum feature state value and the corresponding time are input into the flow matching model to obtain the predicted velocity field at the current time. The inverse ordinary differential equation is solved using numerical integration to calculate the state value for the next time step. This velocity field prediction-integration calculation operation is repeated until all N time steps are calculated, yielding the final state value. The final state value is then validated; if it conforms to a standard Gaussian distribution, it is identified as the latent noise variable corresponding to the MEL spectrum features of the audio data to be detected.

[0187] The log probability density of the corresponding standard Gaussian distribution is calculated based on the latent noise variable to obtain the probability density value. For example, the probability density function formula of the standard Gaussian distribution can be used for calculation. The probability density value and the latent noise variable are coupled and integrated to obtain the velocity field divergence integral term. Based on the probability density value and the velocity field divergence integral term, the log-likelihood value of the audio data to be detected is calculated.

[0188] The preset threshold is determined based on the validation set used during the training of the flow matching model. For example, by analyzing a large number of real and fake medical consultation audio data validation sets, the range of log-likelihood values ​​that can distinguish between real and fake audio is statistically determined, thereby determining an appropriate preset threshold. The log-likelihood value is retrieved and compared with the preset threshold. If the log-likelihood value is greater than the preset threshold, it is determined that the log-likelihood value of the audio data to be detected conforms to the real audio distribution characteristics defined by the flow matching model, that is, the audio data to be detected is real patient consultation audio, and the doctor can continue to make diagnosis and treatment suggestions based on the audio; if the log-likelihood value is less than or equal to the preset threshold, it is determined that the log-likelihood value of the audio data to be detected deviates from the real audio distribution characteristics, that is, the audio data to be detected is fake audio, and the system immediately issues an alarm to remind the doctor to handle it with caution.

[0189] Finally, the authentication results are fed back and visualized. For example, the authentication results are displayed in a prominent color (e.g., red for fake audio, green for genuine audio) on the doctor's consultation interface, along with detailed prompts informing the doctor of the appropriate action. An authentication audit log is generated, recording detailed information such as the authentication time, audio file information, log-likelihood value, preset threshold, and authentication results for subsequent auditing and traceability.

[0190] In other words, by applying audio authentication technology, medical institutions can effectively identify forged audio during telemedicine consultations, preventing criminals from using fake voices to obtain improper medical resources or maliciously interfere with the process, thus ensuring the accuracy of medical diagnoses and the rationality of medical resource allocation. Simultaneously, it improves the security and reliability of telemedicine consultations, enhancing patients' trust in telemedicine. Furthermore, the feedback of authentication results and the generation of audit logs help medical institutions improve their telemedicine quality management systems and enhance their ability to respond to fraudulent activities.

[0191] The aforementioned audio forgery detection method selects MEL spectrum features as model input. MEL spectrum features possess non-linear mapping characteristics to audio frequencies, preserving the core acoustic features of the audio while suppressing redundant interference from irrelevant noise and compression distortion. Compared to the original audio signal or other features, it has stronger resistance to interference factors. The flow matching model accurately captures the probability distribution of real audio through training. Its inverse mapping and log-likelihood calculation process essentially judges whether the audio to be detected belongs to the distribution based on the overall distribution law of real audio, rather than relying on local, easily interfered details. Even if the audio to be detected is compressed or mixed with noise, as long as its core acoustic features still partially retain the distribution trend of real audio, the model can still accurately identify its "real distribution affiliation" through the calculation of the log-likelihood value. If interference causes the audio to deviate significantly from the real distribution, it will be judged as forged with a lower log-likelihood value, avoiding the errors of existing methods due to local feature distortion. The problem lies in the fact that the flow matching model only learns the probability distribution of real audio. The authentication process does not need to distinguish between the "real" and "fake" binary categories. Instead, it transforms the problem into an out-of-distribution detection problem: "Does the audio to be detected conform to the real audio distribution?" This is achieved by extracting the mel spectrum features of the audio to be detected, inputting them into the flow matching model to perform an inverse operation, mapping them back to the prior noise distribution space to obtain a latent noise variable, and then calculating the log-likelihood value based on this variable to represent the probability that the audio to be detected belongs to the real distribution. This technical approach allows the model to be trained without relying on any fake audio samples, covering all fake audio that deviates from the real audio distribution using only the real audio distribution. Regardless of whether the fake audio is generated by a generative adversarial network, variational autoencoder, or diffusion model, as long as its distribution differs from the real audio, it will show a significantly lower log-likelihood value, thus achieving effective detection of various types of fake audio and overcoming the generalization bottleneck of existing methods that "cannot detect unseen fake samples."

[0192] Figure 3 This is a schematic block diagram of an audio authentication device 300 provided in an embodiment of the present invention. Figure 3As shown, corresponding to the above audio spoofing detection method, the present invention also provides an audio spoofing detection device 300. This audio spoofing detection device 300 includes a unit for performing the above audio spoofing detection method, and the device can be configured in a server. Specifically, please refer to... Figure 3 The audio authentication device 300 includes:

[0193] Acquisition unit 301 is used to acquire the audio data to be detected;

[0194] Extraction unit 302 is used to extract the mel spectrum features of the audio data to be detected;

[0195] The input execution unit 303 is used to input the mel spectrum features into the flow matching model to perform inverse operations on the mel spectrum features and to reverse map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables;

[0196] The calculation unit 304 is used to calculate the log-likelihood value of the audio data to be detected based on the potential noise variables;

[0197] The comparison and determination unit 305 is used to compare the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio.

[0198] In one embodiment, the extraction unit 302 includes:

[0199] The preprocessing module is used to pre-emphasize, frame, and window the audio data to be detected to obtain a preprocessed audio signal.

[0200] The transformation module is used to perform a short-time Fourier transform on the preprocessed audio signal to obtain a spectrum.

[0201] The filtering and extraction module is used to filter the spectrum through a MEL filter bank to extract energy features at the MEL frequency scale, forming MEL spectrum features.

[0202] In one embodiment, training the flow matching model includes:

[0203] A certain amount of real audio data is collected and preprocessed to obtain audio frames suitable for training the stream matching model. Mel spectrum features are extracted from the audio frames to form a Mel spectrum feature dataset. The extracted Mel spectrum feature dataset is divided into training, validation, and test sets according to a certain ratio. The parameters of the stream matching model are initialized, and the model is trained using the training set. After each training round, the detection accuracy of the model is evaluated using the validation set to select the target model. The target model is then evaluated using the test set. If the test is passed, the final stream matching model is obtained.

[0204] In one embodiment, the input execution unit 303 includes:

[0205] The partitioning module is used to set the time range for the reverse operation and divide it into N time steps;

[0206] The input module is used to input the current MEL spectrum feature state value and the corresponding time into the flow matching model for each time step to obtain the predicted velocity field at the current time.

[0207] The solver module is used to solve the inverse ordinary differential equation using numerical integration methods to calculate the state value at the next time step.

[0208] The execution module is used to repeatedly perform the velocity field prediction-integration calculation operation until all N time steps of calculation are completed and the final state value is obtained;

[0209] The verification module is used to verify the final state value; if the final state value conforms to the standard Gaussian distribution characteristics, the final state value is determined as the potential noise variable corresponding to the mel spectrum feature of the audio data to be detected.

[0210] In one embodiment, the computing unit 304 includes:

[0211] The first calculation module is used to calculate the corresponding standard Gaussian log probability density based on the potential noise variable, so as to obtain the probability density value;

[0212] A coupling module is used to perform coupled integration on the probability density value and the latent noise variable to obtain the velocity field divergence integral term;

[0213] The second calculation module is used to calculate the log-likelihood value of the audio data to be detected based on the probability density value and the velocity field divergence integral term.

[0214] In one embodiment, the comparison determination unit 305 includes:

[0215] The determination module is used to statistically determine a preset threshold based on the validation set used during the training of the flow matching model.

[0216] The retrieval module is used to retrieve the log-likelihood value and a preset threshold, and perform a numerical comparison:

[0217] The first determination module is used to determine that if the log-likelihood value is greater than a preset threshold, the log-likelihood value of the audio data to be detected conforms to the real audio distribution characteristics defined by the stream matching model, that is, the audio data to be detected is real audio.

[0218] The second determination module is used to determine that if the log-likelihood value is less than or equal to a preset threshold, the log-likelihood value of the audio data to be detected deviates from the true audio distribution characteristics, that is, the audio data to be detected is fake audio.

[0219] In one embodiment, the device further includes:

[0220] The visualization generation unit is used to provide feedback and visualize the authentication results, as well as generate authentication audit logs.

[0221] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned audio counterfeit detection device 300 and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0222] The aforementioned audio authentication device 300 can be implemented as a computer program, which can, for example, Figure 4 It runs on the computer device shown.

[0223] Please see Figure 4 , Figure 4 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0224] See Figure 4 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0225] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform an audio authentication method.

[0226] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0227] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an audio authentication method.

[0228] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0229] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following steps:

[0230] Acquire the audio data to be detected; extract the mel spectrum features of the audio data to be detected; input the mel spectrum features into the stream matching model to perform an inverse operation on the mel spectrum features and map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables; calculate the log-likelihood value of the audio data to be detected based on the latent noise variables; compare the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio.

[0231] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0232] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0233] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform the following steps:

[0234] Acquire the audio data to be detected; extract the mel spectrum features of the audio data to be detected; input the mel spectrum features into the stream matching model to perform an inverse operation on the mel spectrum features and map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables; calculate the log-likelihood value of the audio data to be detected based on the latent noise variables; compare the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio.

[0235] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0236] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0237] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0238] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0239] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0240] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An audio authentication method, characterized in that, include: Acquire the audio data to be detected; Extract the mel spectrum features of the audio data to be detected; The mel spectrum features are input into the flow matching model to perform an inverse operation on the mel spectrum features and to map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables. The log-likelihood value of the audio data to be detected is calculated based on the potential noise variables. The log-likelihood value is compared with a preset threshold to determine whether the audio data to be detected is real audio. The step of inputting the mel spectrum features into the flow matching model to perform an inverse operation on the mel spectrum features and mapping the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables includes: Define the time range for the reverse operation and divide it into N time steps; For each time step, the current MEL spectrum feature state value and the corresponding time input are fed into the flow matching model to obtain the predicted velocity field at the current time. The inverse ordinary differential equation is solved by numerical integration to calculate the state value at the next time step; Repeat the velocity field prediction-integration calculation operation until all N time steps are calculated and the final state value is obtained; The final state value is verified; if the final state value conforms to the standard Gaussian distribution characteristics, then the final state value is determined as the potential noise variable corresponding to the mel spectrum feature of the audio data to be detected. The step of calculating the log-likelihood value of the audio data to be detected based on the latent noise variables includes: The corresponding standard Gaussian log probability density is calculated based on the potential noise variable to obtain the probability density value; The probability density value and the potential noise variable are coupled and integrated to obtain the velocity field divergence integral term; The log-likelihood value of the audio data to be detected is obtained by calculating based on the probability density value and the velocity field divergence integral term.

2. The audio authentication method according to claim 1, characterized in that, The extraction of the mel spectrum features of the audio data to be detected includes: The audio data to be detected is pre-emphasized, framed, and windowed to obtain a pre-processed audio signal; The preprocessed audio signal is subjected to a short-time Fourier transform to obtain a spectrogram; The spectrum is filtered through a MEL filter bank to extract energy features at the MEL frequency scale, forming MEL spectrum features.

3. The audio authentication method according to claim 1, characterized in that, The training process of the stream matching model includes: Collect a certain amount of real audio data and preprocess the collected real audio data to obtain audio frames suitable for training the stream matching model. Extract mel spectrum features from audio frames to form a mel spectrum feature dataset; The extracted mel spectrum feature dataset is divided into training set, validation set and test set according to a certain ratio; Initialize the flow matching model parameters and train the flow matching model using the training set; At the end of each training round, the detection accuracy of the model is evaluated using a validation set in order to select the target model; The target model is evaluated using a test set; if the test passes, the final flow matching model is obtained.

4. The audio authentication method according to claim 1, characterized in that, The step of comparing the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio includes: The preset threshold is determined based on the statistical analysis of the validation set used during the training of the flow matching model. Retrieve the log-likelihood value and a preset threshold, and perform a numerical comparison: If the log-likelihood value is greater than the preset threshold, it is determined that the log-likelihood value of the audio data to be detected conforms to the true audio distribution characteristics defined by the stream matching model, that is, the audio data to be detected is true audio. If the log-likelihood value is less than or equal to a preset threshold, the log-likelihood value of the audio data to be detected is determined to deviate from the true audio distribution characteristics, that is, the audio data to be detected is fake audio.

5. The audio authentication method according to claim 1, characterized in that, After comparing the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio, the method further includes: The system provides feedback and visualization of the authentication results, and generates authentication audit logs.

6. An audio authentication device, characterized in that, include: The acquisition unit is used to acquire the audio data to be detected; The extraction unit is used to extract the mel spectrum features of the audio data to be detected; The input execution unit is used to input the mel spectrum features into the flow matching model to perform inverse operations on the mel spectrum features and to reverse map the mel spectrum features back to the prior noise distribution space to obtain the corresponding latent noise variables. The calculation unit is used to calculate the log-likelihood value of the audio data to be detected based on the potential noise variables; The comparison and determination unit is used to compare the log-likelihood value with a preset threshold to determine whether the audio data to be detected is real audio. The input execution unit includes: The partitioning module is used to set the time range for the reverse operation and divide it into N time steps; The input module is used to input the current MEL spectrum feature state value and the corresponding time into the flow matching model for each time step to obtain the predicted velocity field at the current time. The solver module is used to solve the inverse ordinary differential equation using numerical integration methods to calculate the state value at the next time step. The execution module is used to repeatedly perform the velocity field prediction-integration calculation operation until all N time steps of calculation are completed and the final state value is obtained; The verification module is used to verify the final state value; if the final state value conforms to the standard Gaussian distribution characteristics, the final state value is determined as the potential noise variable corresponding to the mel spectrum feature of the audio data to be detected. The computing unit includes: The first calculation module is used to calculate the corresponding standard Gaussian log probability density based on the potential noise variable, so as to obtain the probability density value; A coupling module is used to perform coupled integration on the probability density value and the latent noise variable to obtain the velocity field divergence integral term; The second calculation module is used to calculate the log-likelihood value of the audio data to be detected based on the probability density value and the velocity field divergence integral term.

7. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images