Audio identification method and device, equipment, storage medium and program product

Abstract

The invention provides an audio identification method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring an audio to be identified; determining audio features of the to-be-identified audio according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features comprise at least one of a GFCC (Gamma Frequency Cepstrum Coefficient) feature, an MFCC (Mel Frequency Cepstrum Coefficient) feature, an LFCC (Linear Frequency Cepstrum Coefficient) feature and a watermark feature; according to the audio characteristics of the to-be-identified audio, determining whether the to-be-identified audio is a synthetic voice; wherein the synthetic voice is processed audio data. According to the method provided by the invention, the accuracy of audio identification is improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing, and in particular to an audio identification method, device, equipment, storage medium and program product. Background Art

[0002] With the continuous iteration of voice models, the audio forged through audio synthesis technology, namely synthetic speech, is getting closer and closer to real audio, namely natural speech in terms of naturalness, realism, and diversity. This is accompanied by a series of security issues. Therefore, it is necessary to identify synthetic speech.

[0003] The current method for identifying synthetic speech is relatively simple, and it is difficult to determine whether the audio to be identified is natural speech or synthetic speech. Therefore, there is a problem of low accuracy in identifying synthetic speech. Summary of the invention

[0004] The present application provides an audio identification method, apparatus, device, storage medium and program product to solve the technical problem of low accuracy in identifying synthesized speech.

[0005] In a first aspect, the present application provides an audio identification method, comprising:

[0006] Get the audio to be identified;

[0007] Determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of a gamma-tone frequency cepstral coefficient GFCC feature, a Mel frequency cepstral coefficient MFCC feature, a linear frequency cepstral coefficient LFCC feature, and a watermark feature;

[0008] According to the audio features of the audio to be identified, it is determined whether the audio to be identified is synthesized speech; wherein the synthesized speech is processed audio data.

[0009] Optionally, the above method, according to a preset audio identification model, determines the audio features of the audio to be identified, including:

[0010] Inputting the audio to be identified into a preset audio identification model; wherein the preset audio identification model includes a preset feature extraction network;

[0011] According to a preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified.

[0012] Optionally, in the above method, the preset feature extraction network includes a first extraction network, a second extraction network, a third extraction network, and a fourth extraction network; according to the preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified, including:

[0013] According to the gammatone filter in the first extraction network, feature extraction processing is performed on the audio to be identified to obtain the GFCC feature of the audio to be identified;

[0014] According to the Mel filter in the second extraction network, feature extraction processing is performed on the audio to be identified to obtain the MFCC features of the audio to be identified;

[0015] According to the linear filter in the third extraction network, feature extraction processing is performed on the audio to be identified to obtain the LFCC feature of the audio to be identified;

[0016] According to the high-pass filter in the fourth extraction network, feature extraction processing is performed on the audio to be identified to obtain the watermark feature of the audio to be identified.

[0017] Optionally, in the above method, the watermark feature is a high-frequency pulse signal or a sine-cosine wave signal; and the feature extraction process is performed on the audio to be identified according to the high-pass filter in the fourth extraction network to obtain the watermark feature of the audio to be identified, including:

[0018] Determine the frequency spectrum information of the audio to be identified according to the high-pass filter in the fourth extraction network; wherein the frequency spectrum information represents the relationship between the frequency and amplitude of the audio signal;

[0019] According to the spectrum information, watermark features are extracted from the audio to be identified.

[0020] Optionally, in the above method, the watermark feature is in the format of a coded string, and the coded string represents at least one of user information, time information, and geographic information; and according to the high-pass filter in the fourth extraction network, feature extraction processing is performed on the audio to be identified to obtain the watermark feature of the audio to be identified, including:

[0021] Extracting a coded string from the audio to be identified according to a high-pass filter in a fourth extraction network;

[0022] The encoded string is decoded to obtain the watermark feature.

[0023] Optionally, the above method, according to the audio features of the audio to be identified, determines whether the audio to be identified is a synthesized speech, including:

[0024] The GFCC features, MFCC features, LFCC features, and watermark features are fused to obtain a fused feature vector of the audio to be identified; wherein the fused feature vector represents the GFCC features, MFCC features, LFCC features, and watermark features;

[0025] Determine the probability value of the audio to be identified based on the fused feature vector; wherein the probability value represents the possibility that the audio to be identified is a synthetic speech;

[0026] According to the probability value, determine whether the audio to be identified is synthetic speech.

[0027] Optionally, the above method, according to the fused feature vector, determines the probability value of the audio to be identified, including:

[0028] According to the preset fully connected layer and the preset activation function, the fused feature vector is mapped to obtain the probability value of the audio to be identified.

[0029] Optionally, the above method, according to a preset fully connected layer and a preset activation function, performs mapping processing on the fused feature vector to obtain a probability value of the audio to be identified, including:

[0030] The fused feature vector is mapped using a preset first fully connected layer to obtain a first vector; wherein the first vector is a vector greater than 2 dimensions;

[0031] Processing the first vector using a preset activation function to obtain a second vector; wherein the dimension of the second vector is consistent with the dimension of the first vector;

[0032] The second vector is mapped using a preset second fully connected layer to obtain a third vector; wherein the third vector is a 2-dimensional vector;

[0033] The third vector is processed using a preset classification function to obtain a probability value of the audio to be identified.

[0034] Optionally, the above method, according to the probability value, determines whether the audio to be identified is a synthetic speech, including:

[0035] If the probability value is within a preset threshold range, it is determined that the audio to be identified is synthetic speech.

[0036] In a second aspect, the present application provides an audio identification device, comprising:

[0037] An acquisition module, used to acquire the audio to be identified;

[0038] The first processing module is used to determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of a gamma-tone frequency cepstral coefficient GFCC feature, a Mel frequency cepstral coefficient MFCC feature, a linear frequency cepstral coefficient LFCC feature, and a watermark feature;

[0039] The second processing module is used to determine whether the audio to be identified is synthetic speech according to the audio features of the audio to be identified; wherein the synthetic speech is processed audio data.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0041] Memory stores computer-executable instructions;

[0042] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0044] In a fifth aspect, the present application provides a computer program product, comprising: a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.

[0045] The audio identification method, device, equipment, storage medium and program product provided by the present application extract audio features from the audio to be identified based on a pre-trained audio identification model, wherein the audio features include gamma-tone frequency cepstral coefficient GFCC features, mel-frequency cepstral coefficient MFCC features, linear frequency cepstral coefficient LFCC features, watermark features, etc. The fusion processing of multiple features in the audio features improves the robustness of the audio identification model and the accuracy of audio identification. By considering the hidden watermark information, the accuracy of audio identification is also guaranteed. The audio identification method of the present application improves the accuracy of identifying synthetic speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] Figure 1 A flowchart of an audio identification method provided by this application;

[0048] Figure 2 A flowchart of another audio identification method provided by this application;

[0049] Figure 3 A flowchart of another audio identification method provided by the present application;

[0050] Figure 4 A flowchart of an audio identification model training method provided in this application;

[0051] Figure 5 A schematic diagram of a spectrum diagram provided for this application;

[0052] Figure 6 A schematic diagram of another spectrum diagram provided for this application;

[0053] Figure 7 A flowchart of another audio identification method provided by the present application;

[0054] Figure 8 A flowchart of an audio identification method provided by the present application;

[0055] Fig. 9 A schematic diagram of the structure of the audio identification device provided by this application;

[0056] Fig.10 A schematic diagram of the structure of the electronic device provided in this application.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0060] In the prior art, the method of extracting features in speech identification methods is relatively simple and single, and it is difficult to accurately distinguish the authenticity of speech, that is, it is difficult to determine whether the audio to be identified is natural speech or synthetic speech.

[0061] Therefore, there is a problem that the accuracy of identifying synthesized speech is low.

[0062] The audio identification method provided by the present application extracts audio features from the audio to be identified based on a pre-trained audio identification model, wherein the audio features include gamma-tone frequency cepstral coefficient GFCC features, mel-frequency cepstral coefficient MFCC features, linear frequency cepstral coefficient LFCC features, and watermark features, and fuses multiple features in the audio features to improve the robustness of the audio identification model and the accuracy of audio identification. By considering the hidden watermark information, the accuracy of audio identification is also guaranteed. The audio identification method of the present application improves the accuracy of identifying synthetic speech.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Figure 1 This is a flow chart of an audio identification method provided by the present application. The execution subject of the method can be a server, a host or other device, such as Figure 1 As shown, the method may include:

[0065] S101, obtaining audio to be authenticated.

[0066] The audio to be identified may refer to the audio to be identified as natural speech or synthesized speech. Natural speech may refer to a speech signal generated by natural human speech, and synthesized speech may refer to a speech signal generated by processing by a computer program or electronic device.

[0067] There may be multiple formats of the audio to be identified. For example, the format of the audio to be identified may include but is not limited to WAV (Waveform Audio), MP3 (Moving Picture Experts Group Audio Layer III), FLAC (Free Lossless Audio Codec), AAC (Advanced Audio Coding), OGG (ogg Vorbis), AIFF (Audio Interchange File Format) and other formats.

[0068] In this application, audio may also be referred to as speech or speech signal.

[0069] S102. Determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of the gamma-tone frequency cepstral coefficient GFCC feature, the Mel frequency cepstral coefficient MFCC feature, the linear frequency cepstral coefficient LFCC feature, and the watermark feature.

[0070] The preset audio identification model may refer to a pre-trained end-to-end model, wherein the end-to-end model may be understood as learning and predicting directly from the original input data to the final output result without the need for intermediate manual processing or staged processing steps. In this embodiment, the model structure of the preset audio identification model is not specifically limited.

[0071] The gammatone frequency cepstral coefficient GFCC (Gammatone Frequency Ceptral Coefficient) feature can refer to a frequency feature based on the gammatone filter bank. GFCC simulates the basilar membrane response of the human ear and is particularly suitable for processing broadband and narrowband noise. It is more sensitive to changes in the pitch and intonation of speech, performs better under low signal-to-noise ratio conditions, and has a certain degree of noise resistance.

[0072] The Mel-Frequency Cepstral coefficient (MFCC) feature can refer to the nonlinear characteristics of the sensitivity to different frequencies that are consistent with human ear perception based on the Mel frequency scale and simulate human auditory perception.

[0073] The Linear Frequency Cepstral Coefficient (LFCC) feature is based on the linear frequency scale and is more sensitive to high-frequency components. If the synthesized speech has abnormalities in the high frequency, LFCC provides high-frequency supplementary information in feature extraction.

[0074] The watermark feature may refer to watermark information embedded in the audio signal, and the watermark information may represent at least one of high-frequency pulse signals, sine and cosine wave signals, user information, and time and geographic information. In this embodiment, the watermark information is hidden information.

[0075] In a possible implementation, the GFCC features, MFCC features, LFCC features, and watermark features are expressed in the form of feature vectors or feature matrices, which can be used for further audio analysis, classification, or recognition tasks.

[0076] It can be understood that by inputting the audio to be identified into a preset audio identification model and processing the audio identification model, the audio features of the audio to be identified can be determined.

[0077] In an optional implementation, determining the audio features of the audio to be identified according to a preset audio identification model may include:

[0078] The audio to be identified is input into a preset audio identification model; wherein the preset audio identification model includes a preset feature extraction network; according to the preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified.

[0079] Among them, the preset feature extraction network may refer to a pre-set neural network for feature extraction, wherein the feature extraction network may refer to a deep learning network, whose function is to extract target features from the input audio data, and the target features may be expressed in the form of feature vectors or feature matrices.

[0080] It can be understood that the structure and parameters of the feature extraction network are pre-set and trained so that the audio to be identified is subjected to feature extraction processing according to the feature extraction network to obtain the audio features of the audio to be identified.

[0081] Exemplarily, the feature extraction network may include four branches, each branch is processed separately to obtain corresponding features, that is, four features may be obtained, namely, GFCC feature, MFCC feature, LFCC feature, and watermark feature.

[0082] The beneficial effect of this arrangement is that, through a preset feature extraction network, the audio features of the audio to be identified are extracted, and the audio features include at least one of the GFCC features, MFCC features, LFCC features, and watermark features, so that the extracted audio features are more directional and rich, thereby improving the accuracy of audio identification.

[0083] S103. Determine whether the audio to be identified is synthesized speech based on the audio features of the audio to be identified; wherein the synthesized speech is processed audio data.

[0084] Among them, synthesized speech is processed audio data. It can be understood that the processed speech data is generated using computers or other technical means (such as text-to-speech conversion, speech synthesizer, etc.), not natural human voice. Synthesized speech usually has some characteristics that make it different from natural human voice, such as spectral characteristics, pitch, rhythm, etc.

[0085] In a possible implementation, the preset audio identification model may include a preset classifier, and the classifier may output two different results, namely, a result of synthesized speech or a result of non-synthesized speech. According to the audio features of the audio to be identified, it is determined whether the audio to be identified is synthesized speech. The audio features of the audio to be identified may be input into the preset classifier, and the classifier outputs a probability value or label to indicate whether the audio to be identified is natural speech or synthesized speech. For example, if the probability value exceeds the preset probability threshold, it is synthesized speech; if the probability value does not exceed the preset probability threshold, it is not synthesized speech.

[0086] The present application provides an audio identification method, which performs identification through a preset audio identification model, wherein the preset audio identification model combines GFCC features, MFCC features, LFCC features, and watermark features, and considers as many audio features as possible, thereby effectively improving the accuracy of audio identification.

[0087] Figure 2 This is a flow chart of another audio identification method provided by the present application. The execution subject of the method can be a server, a host or other device, such as Figure 2 As shown, the method may include:

[0088] S201, obtaining audio to be authenticated.

[0089] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0090] S202: Perform feature extraction processing on the audio to be identified according to the gammatone filter in the first extraction network to obtain the GFCC feature of the audio to be identified.

[0091] The first extraction network may refer to a feature extraction network in a preset audio identification model, and the first extraction network may include a gammatone filter to extract the GFCC features of the audio to be identified.

[0092] A gammatone filter may refer to a filter that mimics the human cochlea. The gammatone filter decomposes the audio signal into multiple frequency bands to simulate the response of the human ear to different frequencies. Thus, the audio to be identified is processed by the gammatone filter, and its cepstral coefficients, i.e., GFCC features, can be further calculated.

[0093] Exemplarily, according to a gammatone filter, feature extraction processing is performed on the audio to be identified to obtain the GFCC features of the audio to be identified, which can include: subjecting the audio to be identified to pre-emphasis, framing, windowing, fast Fourier transform, gammatone filter, logarithmic operation, discrete cosine transform and other processing in sequence to obtain the GFCC features of the audio to be identified.

[0094] Among them, pre-emphasis can refer to the process of high-frequency enhancement of speech signals, the purpose of which is to compensate for the attenuation of high-frequency components of speech signals during transmission and enhance the high-frequency components to improve the effect of subsequent processing. Exemplarily, pre-emphasis can also use a high-pass filter to achieve the function of high-frequency enhancement of speech signals.

[0095] Framing may refer to dividing a continuous speech signal into several short time frames. For example, each frame may be speech data with a duration of 20 to 40 milliseconds. The purpose is to assume that the speech signal is smooth when processing each frame, thereby simplifying the analysis and processing process.

[0096] Windowing may refer to processing using a window function. Since the speech signal of each frame may be discontinuous at the boundary, windowing is to reduce the impact of this situation on spectrum analysis. Exemplarily, the window function of windowing may be a Hamming window or a Hanning window.

[0097] Fast Fourier Transform (FFT) may refer to performing a fast Fourier transform on a windowed signal of each frame, wherein the windowed signal of each frame is a time domain signal, and the time domain signal is converted into a frequency domain signal, i.e., a spectrum, through a fast Fourier transform.

[0098] Gammatone filter can refer to a filter bank that imitates the auditory characteristics of the human ear, and is used to simulate the frequency selectivity of the human ear. The gammatone filter bank can be composed of multiple gammatone filters, each filter corresponding to a center frequency, covering the entire spectrum range of the speech signal. Therefore, the gammatone filter bank can decompose the spectrum of the speech signal and obtain the energy distribution of different frequency bands.

[0099] Logarithmic operation may refer to a logarithmic operation method. Since the spectral energy of a speech signal may have a large dynamic range, that is, the span of adjacent energy values ​​is large, by taking the logarithm, the range of the spectral energy of the speech signal can be compressed to a smaller range, making the eigenvalue more stable and easier to process.

[0100] Discrete cosine transform (DCT) can refer to the discrete cosine transform of the band energy after logarithmic operation. Discrete cosine transform can convert the speech signal from the frequency domain to the cepstrum domain, thereby compressing the feature dimension and making the feature more concentrated. The result of discrete cosine transform is the cepstrum coefficient, and thus, the cepstrum coefficient is the gamma-tone frequency cepstrum coefficient.

[0101] S203: Perform feature extraction processing on the audio to be identified according to the Mel filter in the second extraction network to obtain MFCC features of the audio to be identified.

[0102] The second extraction network may refer to a feature extraction network in a preset audio identification model, and the second extraction network may include a Mel filter to extract MFCC features of the audio to be identified.

[0103] Mel filter can refer to a filter based on the Mel scale, which is a nonlinear frequency scale based on human ear perception. Since the human ear's perception of frequency is not linear, but more sensitive to low frequencies and rougher to high frequencies, the Mel scale maps the frequency to a scale that is more in line with human auditory perception through a nonlinear transformation. Therefore, after the audio to be identified is processed by the Mel filter, its cepstral coefficients, namely MFCC features, can be further calculated.

[0104] Exemplarily, according to the Mel filter, feature extraction processing is performed on the audio to be identified to obtain the MFCC features of the audio to be identified, which can include: subjecting the audio to be identified to pre-emphasis, framing, windowing, fast Fourier transform, Mel filter, logarithmic operation, discrete cosine transform and other processing in sequence to obtain the MFCC features of the audio to be identified.

[0105] The Mel filter may be a Mel filter group, which includes multiple Mel filters that are evenly distributed on the Mel scale. Each filter covers a certain frequency range, and the bandwidth of the filter increases with increasing frequency. This design imitates the perception characteristics of the human ear to different frequencies.

[0106] The processing methods of pre-emphasis, framing, windowing, fast Fourier transform, logarithmic operation, and discrete cosine transform are similar to the above steps of obtaining the GFCC features of the audio to be identified, and will not be repeated here.

[0107] Therefore, the Mel filter bank can decompose the spectrum of the speech signal, that is, the energy distribution of the speech signal in different Mel frequency bands, perform logarithmic operation on the energy value of each frequency band to obtain the logarithmic energy value, and perform discrete cosine transform on the logarithmic energy value to obtain MFCC features.

[0108] S204: Perform feature extraction processing on the audio to be identified according to the linear filter in the third extraction network to obtain LFCC features of the audio to be identified.

[0109] The third extraction network may refer to a feature extraction network in a preset audio identification model, and the third extraction network may include a linear filter to extract LFCC features of the audio to be identified.

[0110] Linear filter is a filter based on linear scale. The steps of processing spectrum of linear filter and Mel filter are similar. The difference between linear filter and Mel filter is that Mel filter is distributed on Mel scale on frequency axis, while linear filter is distributed on linear scale on frequency axis. Therefore, after the audio to be identified is processed by linear filter, its cepstral coefficient, i.e. LFCC feature, can be further calculated.

[0111] Exemplarily, according to a linear filter, feature extraction processing is performed on the audio to be identified to obtain the LFCC features of the audio to be identified, which can include: subjecting the audio to be identified to pre-emphasis, framing, windowing, fast Fourier transform, linear filter, logarithmic operation, discrete cosine transform and other processing in sequence to obtain the LFCC features of the audio to be identified.

[0112] The processing methods of pre-emphasis, framing, windowing, fast Fourier transform, logarithmic operation, and discrete cosine transform are similar to the above steps of obtaining the GFCC features of the audio to be identified, and will not be repeated here.

[0113] The linear filter may be a linear filter group, which includes a plurality of linear filters distributed on a linear scale, and each filter covers a certain frequency range, which is usually in a rectangular or triangular shape.

[0114] Therefore, the linear filter bank can decompose the spectrum of the speech signal, that is, the energy distribution of the speech signal in different Mel frequency bands, perform logarithmic operation on the energy value of each frequency band to obtain the logarithmic energy value, and perform discrete cosine transform on the logarithmic energy value to obtain the LFCC feature.

[0115] S205. Perform feature extraction processing on the audio to be identified according to the high-pass filter in the fourth extraction network to obtain watermark features of the audio to be identified.

[0116] The fourth extraction network may refer to a feature extraction network in a preset audio identification model, and the fourth extraction network may include a high-pass filter to extract the watermark features of the audio to be identified.

[0117] A high-pass filter may refer to a filter that allows high-frequency signals to pass through and attenuates low-frequency signals, and is used to remove low-frequency noise. In this embodiment, the execution order of S202, S203, S204, and S205 is not specifically limited.

[0118] In an optional implementation, the watermark feature may be a high-frequency pulse signal or a sine-cosine wave signal; step S205 may include:

[0119] According to the high-pass filter in the fourth extraction network, the spectrum information of the audio to be identified is determined; wherein the spectrum information represents the relationship between the frequency and amplitude of the audio signal; according to the spectrum information, the watermark feature is extracted from the audio to be identified.

[0120] Among them, the high-frequency pulse signal may refer to a signal that is very short in time but has a high frequency, and the sine-cosine wave signal may refer to a sine wave signal or a cosine wave signal, that is, the signal is in the form of sine or cosine.

[0121] In a possible implementation, the high-frequency pulse signal is a high-frequency sine wave, and the expression form of the high-frequency pulse signal satisfies:

[0122] F1(t)=A1sin(2π*f*t);

[0123] Wherein, F1(t) represents a high-frequency pulse signal, A1 represents the amplitude of the signal, and f represents the frequency. For example, A1=0.01, and f=15000 Hz.

[0124] In an optional implementation, the watermark feature is a frequency modulation-based signal. Exemplarily, the frequency modulation signal is expressed in the form that satisfies:

[0125] F2(t)=A2sin(2π*(f0+kt)*t);

[0126] Among them, F2(t) represents the frequency modulation signal, A2 represents the amplitude of the signal, f0 represents the initial frequency, and k is the modulation frequency, which is used to control the change of the signal frequency over time.

[0127] Exemplarily, based on a high-pass filter, feature extraction processing is performed on the audio to be identified to obtain the watermark features of the audio to be identified, which can include: subjecting the audio to be identified to pre-emphasis, framing, windowing, fast Fourier transform, high-pass filter, logarithmic operation, discrete cosine transform and other processing in sequence to obtain the watermark features of the audio to be identified.

[0128] Among them, if the spectrum information output by the high-pass filter is closer to the high-frequency pulse signal or the sine and cosine wave signal, the watermark feature obtained through logarithmic operation and discrete cosine transform processing will be more directional, thereby improving the accuracy of audio identification.

[0129] The beneficial effect of such a setting is that it takes into account the situation where the watermark feature is a high-frequency pulse signal or a sine-cosine wave signal, so that in the process of audio identification, it is more inclined to be identified as synthetic speech, thereby improving the accuracy of audio identification.

[0130] In an optional implementation, the watermark feature may be in the format of a coded string, where the coded string represents at least one of user information, time information, and geographic information; step S205 may include:

[0131] According to the high-pass filter in the fourth extraction network, the encoded string is extracted from the audio to be identified; the encoded string is decoded to obtain the watermark feature.

[0132] The encoded string may refer to a string obtained after encoding. Encoding is the process of converting information into another form according to a certain rule. It can usually be converting the information to be encoded into a string of characters. The information to be encoded may refer to at least one of user information, time information, and geographic information. Exemplarily, the information to be encoded is user ID, timestamp, longitude, latitude, etc. Exemplarily, the encoding technology may use regular expressions, data parsing libraries, and other technologies.

[0133] If the encoded string is extracted from the audio to be identified through a high-pass filter, the encoded string is decoded using a decoding technology to obtain a watermark feature, which represents at least one of user information, time information, and geographic information. Exemplarily, the decoding technology can use regular expressions, data parsing libraries, and other technologies.

[0134] The beneficial effect of this setting is that it takes into account the case where the watermark feature is a coded string, which represents at least one of user information, time information, and geographic information, so that during the audio identification process, it is more likely to be identified as synthetic speech, thereby improving the accuracy of audio identification.

[0135] It should be noted that the watermark features may also have other forms, and are also applicable to the step of performing feature extraction processing on the audio to be identified according to the high-pass filter in the fourth extraction network to obtain the watermark features of the audio to be identified. The form of the watermark features is not limited here.

[0136] It can be understood that the first extraction network, the second extraction network, the third extraction network, and the fourth extraction network are four branches that do not interfere with each other, so as to realize the functions of respectively extracting GFCC features, MFCC features, LFCC features, and watermark features.

[0137] S206. Determine whether the audio to be identified is synthesized speech based on the audio features of the audio to be identified; wherein the audio features include at least one of GFCC features, MFCC features, LFCC features, and watermark features, and the synthesized speech is processed audio data.

[0138] Exemplarily, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0139] Another audio identification method provided by the present application performs identification through a preset audio identification model, wherein the preset audio identification model combines GFCC features, MFCC features, LFCC features, and watermark features, taking into account as many audio features as possible, thereby effectively improving the accuracy of audio identification.

[0140] Figure 3 This is a flowchart of another audio identification method provided by the present application. The execution subject of this method can be a server, a host or other device, such as Figure 3 As shown, the method may include:

[0141] S301, obtaining audio to be authenticated.

[0142] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0143] S302. Determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of the gamma-tone frequency cepstral coefficient GFCC feature, the Mel frequency cepstral coefficient MFCC feature, the linear frequency cepstral coefficient LFCC feature, and the watermark feature.

[0144] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0145] S303, fusing the GFCC features, MFCC features, LFCC features, and watermark features to obtain a fused feature vector of the audio to be identified; wherein the fused feature vector represents the GFCC features, MFCC features, LFCC features, and watermark features.

[0146] The fusion process may refer to combining the above-mentioned different types of features (GFCC features, MFCC features, LFCC features, watermark features) to form a comprehensive feature vector, namely, a fused feature vector. By fusion, the advantages of different features can be utilized to improve the robustness and accuracy of audio identification.

[0147] In an optional implementation, the GFCC feature, the MFCC feature, the LFCC feature, and the watermark feature are fused to obtain a fused feature vector of the audio to be identified, which may include:

[0148] The GFCC features are processed by the first Res2Net module to obtain the GFCC feature vector; the MFCC features are processed by the second Res2Net module to obtain the MFCC feature vector; the LFCC features are processed by the third Res2Net module to obtain the LFCC feature vector; the watermark features are processed by the Transformer module to obtain the watermark feature vector; the GFCC feature vector, MFCC feature vector, LFCC feature vector and watermark feature vector are concatenated to obtain the fused feature vector of the audio to be identified.

[0149] Among them, Res2Net can refer to an improved neural network architecture that represents multi-scale features in a finer-grained manner by constructing hierarchical residual-like connections within a residual block. The n-layer Res2Net network means that the Res2Net module consists of n layers of basic processing units (such as convolutional layers, pooling layers, fully connected layers, etc.).

[0150] The first Res2Net module may refer to a module including a 30-layer Res2Net network, which is used to process the GFCC features to obtain a GFCC feature vector. Exemplarily, the GFCC feature vector may be a 1024-dimensional vector.

[0151] The second Res2Net module may refer to a module including a 30-layer Res2Net network, which is used to process the MFCC features to obtain an MFCC feature vector. Exemplarily, the MFCC feature vector may be a 1024-dimensional vector.

[0152] The third Res2Net module may refer to a module including a 30-layer Res2Net network, which is used to process the LFCC features to obtain an LFCC feature vector. Exemplarily, the LFCC feature vector may be a 1024-dimensional vector.

[0153] Transformer can refer to a neural network architecture based on the attention mechanism, where the attention mechanism can be understood as the neural network model weightedly averaging these elements by calculating the correlation between each input element and other elements (i.e., the attention weight) to generate a new representation.

[0154] In this application, the Transformer module may refer to a module including 12 layers of Transformer blocks (the basic building blocks of the Transformer model), which is used to process watermark features to obtain a watermark feature vector. Among them, a layer of Transformer block includes an encoder and a decoder. Specifically, the encoder is used to process the input sequence, and the decoder is used to generate the output sequence. Exemplarily, the watermark feature vector can be a 1024-dimensional vector.

[0155] Therefore, the GFCC features, MFCC features, LFCC features, and watermark features are fused to obtain a fused feature vector of the audio to be identified. It can be understood that the GFCC feature vector, MFCC feature vector, LFCC feature vector, and watermark feature vector are spliced ​​to obtain a fused feature vector of the audio to be identified.

[0156] In a possible implementation, a concat function may be used to complete the step of concatenating the GFCC feature vector, the MFCC feature vector, the LFCC feature vector, and the watermark feature vector.

[0157] Exemplarily, if the GFCC feature vector, the MFCC feature vector, the LFCC feature vector, and the watermark feature vector are all 1024-dimensional vectors, the fused feature vector of the audio to be identified obtained by concatenation is a 4096-dimensional vector.

[0158] S304. Determine a probability value of the audio to be identified based on the fused feature vector; wherein the probability value represents the possibility that the audio to be identified is synthetic speech.

[0159] In an optional implementation, step S304 may include:

[0160] According to the preset fully connected layer and the preset activation function, the fused feature vector is mapped to obtain the probability value of the audio to be identified.

[0161] Among them, the preset fully connected layer is a pre-set fully connected layer. In the field of neural networks, a fully connected layer may refer to each neuron being connected to all neurons in the previous layer, and is usually used to output a feature vector converted into a specific output form. Exemplarily, the fused feature vector is a 4096-dimensional vector, which is processed by the fully connected layer and converted into a specific output form, which is a 2-dimensional vector.

[0162] The preset activation function is a pre-set activation function. In the field of neural networks, the activation function can refer to a nonlinear function applied to the output of a neural network layer. Its function is to introduce nonlinearity so that the neural network can learn complex patterns.

[0163] The beneficial effect of such a setting is that the multi-dimensional fused feature vector is reduced in dimension to obtain the probability value of the audio to be identified, which can effectively and intuitively distinguish whether the audio to be identified is natural speech or synthetic speech.

[0164] In an optional implementation, step S304 may further include:

[0165] The fused feature vector is mapped using a preset first fully connected layer to obtain a first vector; wherein the first vector is a vector greater than 2 dimensions; the first vector is processed using a preset activation function to obtain a second vector; wherein the dimension of the second vector is consistent with the dimension of the first vector; the second vector is mapped using a preset second fully connected layer to obtain a third vector; wherein the third vector is a 2-dimensional vector; the third vector is processed using a preset classification function to obtain a probability value of the audio to be identified.

[0166] The first fully connected layer may refer to a connection layer that reduces the dimension of the fused feature vector. If the dimension of the feature fusion vector is x, and the dimension of the first vector obtained by the dimension reduction is y, then x>y>2.

[0167] In a possible implementation, the activation function may be a sigmoid, tanh, or ReLU function, which is not limited here. The dimension of the second vector obtained by the activation function is consistent with that of the first vector. For example, if the first vector is a y-dimensional vector, the second vector is also a y-dimensional vector.

[0168] The second fully connected layer may refer to a connection layer that reduces the dimension of the second vector. The vector obtained by processing the second fully connected layer is a third vector. The dimension of the third vector is 2. Exemplarily, the third vector may be expressed as (2, 1, 1).

[0169] In the field of neural networks, the classification function plays the role of converting the model output into a probability distribution, so that users can make classification decisions based on these probability values. In this application, the probability distribution can be understood as the probability that the audio to be identified belongs to natural speech or synthetic speech.

[0170] In a possible implementation, the classification function may be a Softmax function or a Sigmoid function, which is not limited here. Through the processing of the classification function, the output of the audio identification model can be more intuitively interpreted as the probability that the input audio signal belongs to natural speech and synthetic speech.

[0171] The beneficial effect of the above arrangement is that the multi-dimensional fusion feature vector is reduced to two dimensions to obtain the probability value of the audio to be identified, which can effectively and intuitively distinguish whether the audio to be identified is natural speech or synthetic speech.

[0172] S305: Determine whether the audio to be identified is synthetic speech according to the probability value.

[0173] In an optional implementation, step S305 may include:

[0174] If the probability value is within a preset threshold range, it is determined that the audio to be identified is synthetic speech.

[0175] The preset threshold range may be a pre-set threshold range. In a possible implementation, the preset threshold range may be 0.5 to 1.

[0176] If the probability value is in the range of 0.5 to 1, the audio to be identified is synthetic speech.

[0177] If the probability value is not within the preset threshold range, that is, the probability value is not within the interval of 0.5 to 1, the audio to be identified is natural speech.

[0178] Another audio identification method provided by the present application fuses GFCC features, MFCC features, LFCC features, and watermark features, performs a series of processing based on the fused feature vector, obtains the probability value of the audio to be identified, and then judges whether the audio to be identified is synthetic speech based on the probability value. It integrates as many audio features as possible, effectively improving the accuracy of audio identification.

[0179] Figure 4 This is a flowchart of an audio identification model training method provided by the present application. The execution subject of the method can be a server, a host or other device, such as Figure 4 As shown, the method may include:

[0180] S401, obtaining a natural speech dataset and a synthetic speech dataset; the natural speech dataset represents a natural and truly recorded speech dataset, and the synthetic speech dataset represents a synthetic speech dataset embedded with preset watermark information.

[0181] In an optional implementation, obtaining a synthetic speech dataset may include:

[0182] Acquire an initial synthetic speech data set; the initial synthetic speech data set is characterized as a synthetic speech data set embedded with watermark information; embed the watermark information into the initial synthetic speech data set to obtain a synthetic speech data set; the watermark information includes at least one of a high-frequency pulse signal, a sine and cosine wave signal, user information, time information, and geographic information.

[0183] Exemplarily, the initial synthesized speech may be speech synthesized by using TTS (Text-to-Speech) speech synthesis technology, and the initial synthesized speech is not embedded with watermark information.

[0184] It should be noted that there can be multiple types of watermark information. This application gives examples of several common watermark information, but does not limit the form of the watermark information. When training an audio identification model, if there is other forms of watermark information, a synthetic speech data set with an embedded watermark is obtained, which is also suitable for the audio identification model training method of this application, and the audio identification model can be applied to any of the above-mentioned audio identification methods.

[0185] S402. Train a preset initial model according to a natural speech data set, a synthetic speech data set, a first label, and a second label to obtain a new initial model; the first label represents a label corresponding to the natural speech data set, and the second label represents a label corresponding to the synthetic speech data set.

[0186] Exemplarily, the value of the first tag is true, and the value of the second tag is false, or the value of the first tag is 0, and the value of the second tag is 1.

[0187] S403: If the new initial model meets the training completion condition, a trained audio identification model is obtained.

[0188] The training completion condition may refer to some standards or indicators set during the training process. When these standards or indicators reach a certain threshold, the model training is considered to be completed.

[0189] In a possible implementation, the training completion condition may be to set the maximum number of training rounds to 100, and stop training when the training reaches 100 rounds, or the training completion condition is that the training is completed when the value of the loss function is less than 0.01. No restriction is imposed on the training completion condition herein.

[0190] The present application provides a method for training an audio identification model, in which the training data of the audio identification model are a natural speech data set and a synthetic speech data set embedded with hidden watermark information, which enhances the feature complexity of the synthetic speech, enables the audio identification model to better recognize and distinguish natural speech from synthetic speech, and effectively improves the accuracy of audio identification.

[0191] In order to more intuitively describe the steps of extracting watermark features from synthetic speech embedded with watermark information, Figure 5 A schematic diagram of a spectrum diagram provided for this application, such as Figure 5 As shown, Figure 5 is the spectrum of the synthesized speech without embedded watermark information ( Figure 5 Spectrum of synthesized speech with watermark information embedded with high-frequency sine waves ( Figure 5 (below) for comparison.

[0192] The horizontal axis of the spectrum graph represents frequency, and the vertical axis represents amplitude.

[0193] When the watermark information is a high-frequency sine wave, the synthesized speech without the watermark information embedded and the synthesized speech with the watermark information embedded are not easily perceived by the user.

[0194] When the fast Fourier transform (FFT) is used to convert the synthesized speech without watermark information and the synthesized speech with watermark information into the spectrum of the synthesized speech without watermark information and the spectrum of the synthesized speech with watermark information, it can be seen that the spectrum of the synthesized speech with watermark information presents a sudden and steep peak at 15000HZ. Therefore, it can be considered that Figure 5 The following is the synthesized speech with embedded watermark information.

[0195] Therefore, using Figure 5 The spectrum graph below is processed by high-pass filter, logarithmic operation, discrete cosine transform, etc., to obtain the watermark feature of the audio, which is more inclined to indicate that the audio is a synthetic speech embedded with watermark information.

[0196] Figure 6 A schematic diagram of another spectrum diagram provided for this application, such as Figure 6 As shown, Figure 6 is the spectrum of the synthesized speech without embedded watermark information ( Figure 6 Spectrum of synthesized speech with watermark information embedded in FM signal ( Figure 6 (below) for comparison.

[0197] The horizontal axis of the spectrum graph represents frequency, and the vertical axis represents amplitude.

[0198] When the watermark information is a high-frequency sine wave, the synthesized speech without the watermark information embedded and the synthesized speech with the watermark information embedded are not easily perceived by the user.

[0199] When the Fast Fourier Transform (FFT) is used to convert the synthesized speech without watermark information and the synthesized speech with watermark information into the spectrum of the synthesized speech without watermark information and the spectrum of the synthesized speech with watermark information, it can be seen that the frequency of the spectrum of the synthesized speech with watermark information presents a relatively stable peak at 15000HZ to 16000HZ. Therefore, it can be considered that Figure 6 The following is the synthesized speech with embedded watermark information.

[0200] Therefore, using Figure 6 The spectrum graph below is processed by high-pass filter, logarithmic operation, discrete cosine transform, etc., to obtain the watermark feature of the audio, which is more inclined to indicate that the audio is a synthetic speech embedded with watermark information.

[0201] Figure 7 This is a flow chart of another audio identification method provided by the present application. The execution subject of the method can be a server, a host or other device, such as Figure 7 As shown, after being processed by the audio identification method, the audio to be identified is input and the identification result of whether it is natural speech or synthetic speech can be output.

[0202] Specifically, the method may include:

[0203] S701, obtaining audio to be authenticated.

[0204] S702, extract the GFCC features of the audio to be identified, and process them through 30 layers of Res2Net to obtain a 1024-dimensional vector corresponding to the GFCC features.

[0205] S703, extracting the MFCC features of the audio to be identified, and processing them through 30 layers of Res2Net to obtain a 1024-dimensional vector corresponding to the MFCC features.

[0206] S704, extract the LFCC features of the audio to be identified, and process them through 30 layers of Res2Net to obtain a 1024-dimensional vector corresponding to the LFCC features.

[0207] S705, extracting the watermark feature of the audio to be identified, and processing it through 12 layers of transfomer block to obtain a 1024-dimensional vector corresponding to the watermark feature.

[0208] S706 , by using the concat function, concatenate the 1024-dimensional vector corresponding to the GFCC feature, the 1024-dimensional vector corresponding to the MFCC feature, the 1024-dimensional vector corresponding to the LFCC feature, and the 1024-dimensional vector corresponding to the watermark feature into a 4096-dimensional vector.

[0209] S707, map the 4096-dimensional vector to a 128-dimensional vector through the fully connected layer 1 (Fully connected 1).

[0210] S708 . Use the ReLU activation function to process the mapped 128-dimensional vector into a new 128-dimensional vector.

[0211] S709: Map the new 128-dimensional vector into a 2-dimensional vector through the fully connected layer 2 (Fully connected 2).

[0212] S710: Use the Softmax function to process the 2D vector to obtain a probability value.

[0213] S711. If the probability value is close to 1, the output result is false, that is, synthesized speech.

[0214] S712: If the probability value is close to 0, the output result is true, that is, natural speech.

[0215] For example, in order to better demonstrate the data trend of the audio to be identified after being processed by the audio identification model, and then determine whether the audio to be identified is natural speech or synthetic speech, Figure 8 A flowchart of an audio identification method provided by this application is Figure 7 Another form of flow chart.

[0216] like Figure 8 As shown in the figure, the audio to be identified passes through 4 layers of feature extraction branches, and 4 1024-dimensional vectors are obtained respectively. After being spliced ​​through the concat function, a 4096-dimensional vector is obtained. After being processed by the fully connected layer 1, the ReLU activation function, the fully connected layer 2, and the Softmax function, the probability value is obtained. If the probability value is close to 1, the output result is false, that is, synthetic speech; if the probability value is close to 0, the output result is true, that is, natural speech.

[0217] Another audio identification method provided by the present application uses an audio identification model for identification, wherein the audio identification model combines GFCC features, MFCC features, LFCC features, and watermark features, considers as many audio features as possible, and uses Res2Net or transfomer block to process the features, thereby effectively improving the robustness of the model and the accuracy of audio identification.

[0218] Fig. 9 A schematic diagram of the structure of the audio identification device provided in this application, such as Fig. 9 As shown, the audio identification device 90 provided in this embodiment includes: an acquisition module 901, a first processing module 902, and a second processing module 903;

[0219] The acquisition module 901 is used to acquire the audio to be identified.

[0220] The first processing module 902 is used to determine the audio features of the audio to be identified based on a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of the gamma-tone frequency cepstral coefficient GFCC feature, the Mel frequency cepstral coefficient MFCC feature, the linear frequency cepstral coefficient LFCC feature, and the watermark feature.

[0221] The second processing module 903 is used to determine whether the audio to be identified is synthetic speech according to the audio features of the audio to be identified; wherein the synthetic speech is processed audio data.

[0222] In a possible implementation, the first processing module 902 is further configured to input the audio to be identified into a preset audio identification model; wherein the preset audio identification model includes a preset feature extraction network;

[0223] According to a preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified.

[0224] In a possible implementation, the first processing module 902 is further configured to perform feature extraction processing on the audio to be identified according to the gammatone filter in the first extraction network to obtain the GFCC feature of the audio to be identified;

[0225] According to the Mel filter in the second extraction network, feature extraction processing is performed on the audio to be identified to obtain the MFCC features of the audio to be identified;

[0226] According to the linear filter in the third extraction network, feature extraction processing is performed on the audio to be identified to obtain the LFCC feature of the audio to be identified;

[0227] According to the high-pass filter in the fourth extraction network, feature extraction processing is performed on the audio to be identified to obtain the watermark feature of the audio to be identified.

[0228] In a possible implementation, the first processing module 902 is further configured to determine frequency spectrum information of the audio to be identified according to the high-pass filter in the fourth extraction network; wherein the frequency spectrum information represents the relationship between the frequency and amplitude of the audio signal;

[0229] According to the spectrum information, watermark features are extracted from the audio to be identified.

[0230] In a possible implementation, the first processing module 902 is further configured to extract a coded string from the audio to be identified according to a high-pass filter in the fourth extraction network;

[0231] The encoded string is decoded to obtain the watermark feature.

[0232] In a possible implementation, the second processing module 903 is further used to fuse the GFCC features, the MFCC features, the LFCC features, and the watermark features to obtain a fused feature vector of the audio to be identified; wherein the fused feature vector represents the GFCC features, the MFCC features, the LFCC features, and the watermark features;

[0233] Determine the probability value of the audio to be identified based on the fused feature vector; wherein the probability value represents the possibility that the audio to be identified is a synthetic speech;

[0234] According to the probability value, determine whether the audio to be identified is synthetic speech.

[0235] In a possible implementation, the second processing module 903 is further configured to perform mapping processing on the fused feature vector according to a preset fully connected layer and a preset activation function to obtain a probability value of the audio to be identified.

[0236] In a possible implementation, the second processing module 903 is further used to map the fused feature vector using a preset first fully connected layer to obtain a first vector; wherein the first vector is a vector greater than 2 dimensions;

[0237] Processing the first vector using a preset activation function to obtain a second vector; wherein the dimension of the second vector is consistent with the dimension of the first vector;

[0238] The second vector is mapped using a preset second fully connected layer to obtain a third vector; wherein the third vector is a 2-dimensional vector;

[0239] The third vector is processed using a preset classification function to obtain a probability value of the audio to be identified.

[0240] In a possible implementation, the second processing module 903 is further configured to determine that the audio to be identified is synthetic speech if the probability value is within a preset threshold range.

[0241] The audio identification device provided in the present application can execute the method provided in the above-mentioned audio identification method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0242] Fig.10 A schematic diagram of the structure of the electronic device provided in this application, such as Fig.10 As shown, the electronic device 100 provided in the embodiment of the present application includes: at least one processor 1001 and a memory 1002. Optionally, the device 100 also includes a communication component 1003. The processor 1001, the memory 1002 and the communication component 1003 are connected via a bus 1004.

[0243] In a specific implementation process, at least one processor 1001 executes the computer-executable instructions stored in the memory 1002, so that at least one processor 1001 executes the above method.

[0244] The specific implementation process of the processor 1001 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0245] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0246] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0247] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0248] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0249] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0250] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0251] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0252] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0253] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0254] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0255] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0256] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0257] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0258] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0259] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An audio identification method, characterized in that: include: Get the audio to be identified; Determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of a gamma-tone frequency cepstral coefficient GFCC feature, a Mel frequency cepstral coefficient MFCC feature, a linear frequency cepstral coefficient LFCC feature, and a watermark feature; According to the audio features of the audio to be identified, it is determined whether the audio to be identified is synthesized speech; wherein the synthesized speech is processed audio data.

2. The method according to claim 1, characterized in that Determining the audio features of the audio to be identified according to a preset audio identification model includes: Inputting the audio to be identified into a preset audio identification model; wherein the preset audio identification model includes a preset feature extraction network; According to a preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified.

3. The method according to claim 2, characterized in that The preset feature extraction network includes a first extraction network, a second extraction network, a third extraction network, and a fourth extraction network; According to a preset feature extraction network, feature extraction processing is performed on the audio to be identified to obtain audio features of the audio to be identified, including: Performing feature extraction processing on the audio to be identified according to the gammatone filter in the first extraction network to obtain the GFCC feature of the audio to be identified; Performing feature extraction processing on the audio to be identified according to the Mel filter in the second extraction network to obtain MFCC features of the audio to be identified; Performing feature extraction processing on the audio to be identified according to the linear filter in the third extraction network to obtain the LFCC feature of the audio to be identified; According to the high-pass filter in the fourth extraction network, feature extraction processing is performed on the audio to be identified to obtain the watermark feature of the audio to be identified.

4. The method according to claim 3, characterized in that: The watermark feature is a high-frequency pulse signal or a sine-cosine wave signal; according to the high-pass filter in the fourth extraction network, the feature extraction process is performed on the audio to be identified to obtain the watermark feature of the audio to be identified, including: Determine the frequency spectrum information of the audio to be identified according to the high-pass filter in the fourth extraction network; wherein the frequency spectrum information represents the relationship between the frequency and amplitude of the audio signal; The watermark feature is extracted from the audio to be identified according to the frequency spectrum information.

5. The method according to claim 3, characterized in that: The watermark feature is in the format of a coded string, and the coded string represents at least one of user information, time information, and geographic information; According to the high-pass filter in the fourth extraction network, feature extraction processing is performed on the audio to be identified to obtain the watermark feature of the audio to be identified, including: Extracting a coded string from the audio to be identified according to the high-pass filter in the fourth extraction network; The encoded character string is decoded to obtain the watermark feature.

6. The method according to claim 1, characterized in that Determining whether the audio to be identified is a synthesized speech according to the audio feature of the audio to be identified includes: The GFCC features, MFCC features, LFCC features, and watermark features are fused to obtain a fused feature vector of the audio to be identified; wherein the fused feature vector represents the GFCC features, MFCC features, LFCC features, and watermark features; Determine the probability value of the audio to be identified according to the fused feature vector; wherein the probability value represents the possibility that the audio to be identified is a synthetic speech; According to the probability value, it is determined whether the audio to be identified is synthetic speech.

7. The method according to claim 6, characterized in that Determining the probability value of the audio to be identified according to the fused feature vector includes: According to a preset fully connected layer and a preset activation function, the fused feature vector is mapped to obtain a probability value of the audio to be identified.

8. The method according to claim 7, characterized in that According to a preset fully connected layer and a preset activation function, mapping processing is performed on the fused feature vector to obtain a probability value of the audio to be identified, including: The fused feature vector is mapped using a preset first fully connected layer to obtain a first vector; wherein the first vector is a vector greater than 2 dimensions; Processing the first vector using a preset activation function to obtain a second vector; wherein the dimension of the second vector is consistent with the dimension of the first vector; The second vector is mapped using a preset second fully connected layer to obtain a third vector; wherein the third vector is a 2-dimensional vector; The third vector is processed using a preset classification function to obtain a probability value of the audio to be identified.

9. The method according to claim 6, characterized in that Determining whether the audio to be identified is a synthetic speech according to the probability value includes: If the probability value is within a preset threshold range, it is determined that the audio to be identified is synthetic speech.

10. An audio identification device, characterized in that: include: An acquisition module, used to acquire the audio to be identified; A first processing module, used to determine the audio features of the audio to be identified according to a preset audio identification model; wherein the preset audio identification model is a pre-trained end-to-end model, and the audio features include at least one of a gamma-tone frequency cepstral coefficient GFCC feature, a Mel frequency cepstral coefficient MFCC feature, a linear frequency cepstral coefficient LFCC feature, and a watermark feature; The second processing module is used to determine whether the audio to be identified is synthetic speech according to the audio features of the audio to be identified; wherein the synthetic speech is processed audio data.

11. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.

Images