Audio authentication method, related apparatus and storage medium

Abstract

Embodiments of the present application relate to the field of artificial intelligence, and provide an audio authentication method, related device and storage medium. The audio authentication method comprises: obtaining a plurality of counterfeit audios and a plurality of real audios; performing time sequence feature extraction on the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios; dividing the plurality of counterfeit audio time sequence features and the plurality of real audio time sequence features into different sets respectively to obtain a plurality of counterfeit audio time sequence feature sets and a plurality of real audio time sequence feature sets; when a to-be-identified audio is obtained, calculating a first similarity parameter of the to-be-identified audio and the plurality of counterfeit audio time sequence feature sets and a second similarity parameter of the to-be-identified audio and the plurality of real audio time sequence feature sets respectively; and if the first similarity parameter is greater than the second similarity parameter, determining that an audio category of the to-be-identified audio is a counterfeit category.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence, and more particularly to an audio authentication method, related device and storage medium. BACKGROUND

[0002] With the rapid development of 5G technology, voice deep forging related technologies, such as text-to-speech (TTS) and voice conversion (VC), are becoming more mature, and have been widely used in medical rehabilitation (such as "rebuilding" voices for voiceless patients) and entertainment (such as funny videos). While meeting people's daily needs, it has also brought many security risks. In view of this security risk, many scholars have proposed related voice authentication methods. However, due to the difference between the data distribution of the business site and the data distribution of the model training stage, there is a high false rejection rate and a poor detection rate. Therefore, a new audio authentication method is needed to improve the accuracy of audio authentication. SUMMARY

[0003] Embodiments of the present application provide an audio authentication method, related device and storage medium, which can improve the accuracy of audio authentication.

[0004] In a first aspect, embodiments of the present application provide an audio authentication method, which comprises:

[0005] obtaining a plurality of forged audios and a plurality of real audios;

[0006] performing time sequence feature extraction on the plurality of forged audios and the plurality of real audios to obtain a plurality of forged audio time sequence features corresponding to the plurality of forged audios and a plurality of real audio time sequence features corresponding to the plurality of real audios;

[0007] dividing the plurality of forged audio time sequence features and the plurality of real audio time sequence features into different sets respectively to obtain a plurality of forged audio time sequence feature sets and a plurality of real audio time sequence feature sets;

[0008] When a to-be-identified audio is obtained, a first similarity parameter between the to-be-identified audio and the plurality of forged audio time sequence feature sets and a second similarity parameter between the to-be-identified audio and the plurality of real audio time sequence feature sets are calculated respectively;

[0009] If the first similarity parameter is greater than the second similarity parameter, it is determined that the audio category of the to-be-identified audio is a forged category.

[0010] In one embodiment, the time sequence feature extraction on the plurality of counterfeit audios and the plurality of real audios obtains a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios, including:

[0011] The counterfeit audio is split into a plurality of counterfeit sub-audios sequentially ordered in time;

[0012] The frequency domain feature extraction is respectively performed on each counterfeit sub-audio to obtain a first counterfeit frequency domain feature corresponding to each counterfeit sub-audio;

[0013] The counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio.

[0014] In one embodiment, the counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio, including:

[0015] Each counterfeit sub-audio is determined as a target sub-audio;

[0016] The feature difference between the first counterfeit frequency domain feature of the first counterfeit sub-audio after the target sub-audio and the first counterfeit frequency domain feature of the target sub-audio is determined as a neighboring frequency domain change feature of the target sub-audio, to obtain a neighboring frequency domain change feature of each target sub-audio;

[0017] The plurality of first counterfeit frequency domain features and the plurality of neighboring frequency domain change features corresponding to the counterfeit audio are spliced to obtain a plurality of second counterfeit frequency domain features of the counterfeit audio;

[0018] The counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of second counterfeit frequency domain features of the counterfeit audio.

[0019] In one embodiment, the counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of second counterfeit frequency domain features of the counterfeit audio, including:

[0020] The feature difference between the first counterfeit frequency domain feature of the second counterfeit sub-audio after the target sub-audio and the first counterfeit frequency domain feature of the target sub-audio is determined as an inter-frequency domain change feature of the target sub-audio, to obtain an inter-frequency domain change feature of each target sub-audio;

[0021] Splicing corresponding to the plurality of second counterfeit frequency domain features and the plurality of inter-frequency domain variation features of the counterfeit audio to obtain a plurality of third counterfeit frequency domain features of the counterfeit audio;

[0022] Based on the plurality of third counterfeit frequency domain features of the counterfeit audio, the counterfeit audio timing feature corresponding to the counterfeit audio is determined.

[0023] In one embodiment, the first similarity parameter of the to-be-identified audio and a plurality of counterfeit audio timing feature sets, the second similarity parameter of the to-be-identified audio and a plurality of real audio timing feature sets, respectively, comprises:

[0024] The average value of a plurality of counterfeit audio timing features in the counterfeit audio timing feature set is determined as the set audio feature of the counterfeit audio timing feature set;

[0025] The audio feature similarity between the audio feature of the to-be-identified audio and a plurality of set audio features of the plurality of counterfeit audio timing feature sets is calculated to obtain a plurality of audio feature similarities of the plurality of counterfeit audio timing feature sets.

[0026] The maximum value of a plurality of audio feature similarities of a plurality of counterfeit audio timing feature sets is determined as the first similarity parameter.

[0027] In one embodiment, the timing feature extraction of a plurality of counterfeit audios and a plurality of real audios to obtain a plurality of counterfeit audio timing features corresponding to a plurality of counterfeit audios and a plurality of real audio timing features corresponding to a plurality of real audios comprises:

[0028] Frame and window operation is performed on the counterfeit audio to obtain each frame of audio signal;

[0029] Fast Fourier transform is performed on each frame of audio signal to obtain the amplitude spectrum of linear frequency domain;

[0030] The amplitude spectrum of linear frequency domain is filtered using a linear frequency filter bank to obtain the energy of each filter in the filter bank;

[0031] Based on the energy of each filter, logarithmic operation and discrete cosine transform are performed to obtain the counterfeit audio timing feature of the counterfeit audio.

[0032] In a second aspect, the embodiments of the present application provide an audio authentication device having the function of implementing the audio authentication method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, which can be software and / or hardware.

[0033] In one embodiment, the audio authentication device comprises:

[0034] An acquisition module configured to acquire a plurality of counterfeit audios and a plurality of real audios;

[0035] A feature extraction module configured to perform time sequence feature extraction on the plurality of counterfeit audios and the plurality of real audios, to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios;

[0036] A division module configured to respectively divide the plurality of counterfeit audio time sequence features and the plurality of real audio time sequence features into different sets, to obtain a plurality of counterfeit audio time sequence feature sets and a plurality of real audio time sequence feature sets;

[0037] A calculation module configured to, when a to-be-identified audio is acquired, respectively calculate a first similarity parameter of the to-be-identified audio and the plurality of counterfeit audio time sequence feature sets and a second similarity parameter of the to-be-identified audio and the plurality of real audio time sequence feature sets;

[0038] A determination module configured to, if the first similarity parameter is greater than the second similarity parameter, determine that an audio category of the to-be-identified audio is a counterfeit category.

[0039] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which includes instructions, when the instructions are run on a computer, causing the computer to execute the audio authentication method in the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computing device, which includes a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the audio authentication method in the first aspect when executing the computer program.

[0041] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor coupled with a transceiver of a terminal device, for executing the technical solution provided in the first aspect of the present application.

[0042] In a sixth aspect, an embodiment of the present application provides a chip system, which includes a processor for supporting the terminal device to implement the functions involved in the first aspect, for example, generating or processing the information involved in the image processing method provided in the first aspect.

[0043] In a possible design, the chip system further includes a memory for saving program instructions and data necessary for the terminal. The chip system can be composed of a chip, or can include the chip and other discrete devices.

[0044] In a seventh aspect, an embodiment of the present application provides a computer program product comprising instructions which, when the computer program product runs on a computer, cause the computer to execute the audio authentication method of the first aspect.

[0045] Compared with the prior art, in the embodiment of the present application, a plurality of counterfeit audios and a plurality of real audios are obtained; time sequence features of the plurality of counterfeit audios and the plurality of real audios are extracted to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios; the plurality of counterfeit audio time sequence features and the plurality of real audio time sequence features are respectively divided into different sets to obtain a plurality of counterfeit audio time sequence feature sets and a plurality of real audio time sequence feature sets; when a to-be-identified audio is obtained, a first similarity parameter of the to-be-identified audio and the plurality of counterfeit audio time sequence feature sets and a second similarity parameter of the to-be-identified audio and the plurality of real audio time sequence feature sets are calculated; if the first similarity parameter is greater than the second similarity parameter, it is determined that the audio category of the to-be-identified audio is a counterfeit category. The first similarity parameter of the to-be-identified audio and the counterfeit audio time sequence feature set containing the plurality of counterfeit audio time sequence features is calculated, and then the second similarity parameter of the to-be-identified audio and the real audio time sequence feature set containing the plurality of real audio time sequence features is calculated. According to the first similarity parameter and the second similarity parameter, it is determined whether the audio category of the to-be-identified audio is a counterfeit category. The rich counterfeit feature information can be utilized, and the accuracy of audio authentication can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] The purposes, features and advantages of the embodiments of the present application will become easy to understand by reading the detailed description of the embodiments of the present application with reference to the accompanying drawings. Among them:

[0047] Figure 1 An audio authentication system schematic diagram for the audio authentication method in the embodiments of the present application;

[0048] Figure 2 A flowchart schematic diagram of the audio authentication method of the embodiments of the present application;

[0049] Figure 3 A structure schematic diagram of the audio authentication device of the embodiments of the present application;

[0050] Figure 4 A structure schematic diagram of the computing device of the embodiments of the present application;

[0051] Figure 5 A structure schematic diagram of the mobile phone in the embodiments of the present application;

[0052] Figure 6 A structure schematic diagram of the server in the embodiments of the present application.

[0053] In the drawings, like or corresponding elements are denoted by like or corresponding reference numerals. DETAILED DESCRIPTION

[0054] The terms "first", "second", and the like in the description and in the claims of the present application and the above drawings do not necessarily mean a specific order or sequence, but are used to distinguish similar objects. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or modules does not have to be limited to only those steps or modules clearly listed, but can include other steps or modules not clearly listed or inherent to such a process, method, product, or device. The division of modules in the embodiments of the present application is only a logical division, and in actual application, another division mode can be used, for example, a plurality of modules can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be through some interface, indirect coupling between modules, and the communication connection can be electrical or other similar forms, which are not limited in the embodiments of the present application. In addition, the modules or sub-modules described as separate components can or can not be physically separated, can or can not be physical modules, or can be distributed into a plurality of circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0055] The embodiments of the present application also provide an audio authentication method, related device and storage medium, which can be applied to an audio authentication system. The audio authentication system can include an audio authentication device, which can be deployed integrally or separately. The audio authentication device is used to obtain a plurality of counterfeit audios and a plurality of real audios; time sequence features of the plurality of counterfeit audios and the plurality of real audios are extracted to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios; the plurality of counterfeit audio time sequence features and the plurality of real audio time sequence features are respectively divided into different sets to obtain a plurality of counterfeit audio time sequence feature sets and a plurality of real audio time sequence feature sets; when a to-be-identified audio is obtained, a first similarity parameter of the to-be-identified audio and the plurality of counterfeit audio time sequence feature sets and a second similarity parameter of the to-be-identified audio and the plurality of real audio time sequence feature sets are calculated; and if the first similarity parameter is greater than the second similarity parameter, it is determined that an audio category of the to-be-identified audio is a counterfeit category.

[0056] The scheme provided in the embodiments of the present application relates to technologies such as artificial intelligence (AI), machine learning (ML), and the like, and is specifically explained through the following embodiments:

[0057] The AI is a theory, method, technology, and application system for simulating, extending, and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge, and using the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning, and decision-making.

[0058] The AI technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction systems, mechatronics, and the like. The software technologies of artificial intelligence mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, and the like.

[0059] The existing audio authentication method has a high false rejection rate and a poor detection rate due to the difference between the data distribution of the business site and the data distribution in the model training stage.

[0060] Compared with the prior art, in the embodiments of the present application, a plurality of counterfeit audios and a plurality of real audios are obtained; time sequence features of the plurality of counterfeit audios and the plurality of real audios are extracted to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios; the plurality of counterfeit audio time sequence features and the plurality of real audio time sequence features are respectively divided into different sets to obtain a plurality of counterfeit audio time sequence feature sets and a plurality of real audio time sequence feature sets; when a to-be-identified audio is obtained, a first similarity parameter of the to-be-identified audio and the plurality of counterfeit audio time sequence feature sets and a second similarity parameter of the to-be-identified audio and the plurality of real audio time sequence feature sets are calculated; if the first similarity parameter is greater than the second similarity parameter, it is determined that the audio category of the to-be-identified audio is a counterfeit category. The present application calculates the first similarity parameter of the to-be-identified audio and the counterfeit audio time sequence feature set containing a plurality of counterfeit audio time sequence features, and then calculates the second similarity parameter of the to-be-identified audio and the real audio time sequence feature set containing a plurality of real audio time sequence features. According to the first similarity parameter and the second similarity parameter, it is determined whether the audio category of the to-be-identified audio is a counterfeit category. The abundant counterfeit feature information can improve the accuracy of audio authentication.

[0061] In some embodiments, referring to Figure 1 , the audio authentication method provided by the embodiments of the present application can be implemented based on Figure 1 An audio authentication system is shown. The audio authentication system can include an electronic device 100 and a memory 200. The electronic device 100 can be a server or a terminal device.

[0062] It should be noted that the server involved in the embodiments of the present application can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.

[0063] The terminal device involved in the embodiments of the present application can be a device that provides voice and / or data connectivity to users, a handheld device with wireless connection function, or other processing devices connected to a wireless modem. For example, mobile phones (also known as "cellular" phones) and computers with mobile terminals, such as portable, pocket, handheld, built-in computer or vehicle-mounted mobile devices, which exchange voice and / or data with a wireless access network. For example, personal communication service (English full name: Personal Communication Service, English abbreviation: PCS) phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (Wireless Local Loop, English abbreviation: WLL) stations, personal digital assistants (English full name: Personal Digital Assistant, English abbreviation: PDA) and other devices.

[0064] Referring to Figure 2 , Figure 2 A flowchart of an audio authentication method provided by the embodiments of the present application is shown. The method can be executed by an audio authentication device. The method includes steps 101-105:

[0065] Step 101, obtaining a plurality of counterfeit audios and a plurality of real audios.

[0066] In the embodiments of the present application, the category of the fake audio is a fake category, and the category of the real audio is a real category. The real audio can be a naturally recorded audio, for example, the real audio is a recording of a person naturally speaking, that is, a natural speech. The fake audio can be artificially synthesized, and the audio type of the fake audio can be fake type 1, fake type 2, and the like, which can be set according to specific conditions. Different fake types can represent fake speeches in different fields, for example, the fields of mathematics, literature, finance, and the like, which can be set according to specific conditions.

[0067] In a specific embodiment, the real audio is obtained; the real audio is segmented based on the real audio to obtain speech segments of the real audio; and the speech segments of the real audio are randomly spliced based on the speech segments of the real audio to obtain spliced speech, which is determined as the fake audio.

[0068] In order to improve the randomness of segmentation, the real audio can be divided into multiple segments according to different lengths. Therefore, in some embodiments, the real audio is segmented based on the real audio to obtain speech segments of the real audio, including: the real audio is segmented according to a first length to obtain speech segments of the real audio with the first length; and the real audio is segmented according to a second length to obtain speech segments of the real audio with the second length. The speech segments of the real audio include the speech segments of the real audio with the first length and the speech segments of the real audio with the second length. The first length and the second length can be pre-set, for example, the first length can be 0.2 seconds, and the second length can be 0.4 seconds.

[0069] In order to improve the randomness of splicing, in some embodiments, the speech segments are randomly spliced based on the speech segments to obtain spliced speech, including the following steps: obtaining a first segment set and a second segment set, the first segment set including the speech segments of the real audio with the first length, and the second segment set including the speech segments of the real audio with the second length; randomly selecting a speech segment from the first segment set as a first speech segment; randomly selecting a speech segment from the second segment set as a second speech segment; and splicing the first speech segment and the second speech segment to obtain the spliced speech.

[0070] Specifically, in some embodiments, randomly selecting a speech segment from the first segment set as a first speech segment can be randomly selecting N speech segments from the first segment set as the first speech segments. Randomly selecting a speech segment from the second segment set as a second speech segment can be randomly selecting N speech segments from the second segment set as the second speech segments. N is a positive integer.

[0071] At step 102, time sequence features are extracted from the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios.

[0072] In an embodiment, in a specific embodiment, the plurality of counterfeit audios and the plurality of real audios are respectively input into a first preset feature extraction model for time sequence feature extraction to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios. The first preset feature extraction model can be a recurrent neural network (RNN) or a Transformer model. The input of the first preset feature extraction model is a serialized acoustic feature or an original audio waveform. The recurrent connection of the hidden layer captures the dependency in the time sequence, and the LSTM or GRU variant solves the long-time dependency problem. For example, the LSTM / GRU network directly models the serialized acoustic feature for speaker recognition or speech emotion analysis.

[0073] In another specific embodiment, time sequence features are extracted from the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios, including:

[0074] (1) The counterfeit audio is split into a plurality of counterfeit sub-audios sorted in time.

[0075] Specifically, the counterfeit audio is split into a plurality of counterfeit sub-audios with equal time lengths sorted in time. The number of the plurality of counterfeit sub-audios can be set according to specific conditions. For example, if the counterfeit audio has a length of 8s, the counterfeit audio is split into 8 counterfeit sub-audios with a length of 1s.

[0076] (2) Frequency domain features are extracted from each counterfeit sub-audio to obtain first counterfeit frequency domain features corresponding to each counterfeit sub-audio.

[0077] The first counterfeit frequency domain feature is a linear frequency cepstral coefficient (LFCC) feature. In other embodiments, the first counterfeit frequency domain feature can be a mel frequency cepstral coefficient (MFCC) feature. The mel frequency cepstral coefficient (MFCC) is a commonly used feature extraction method in speech processing, and the parameter settings thereof directly affect the performance of the feature and the effect of subsequent audio counterfeit detection tasks. The LFCC feature is a feature extraction method used in speech signal processing, and the core difference between it and the traditional MFCC (mel frequency cepstral coefficient) is the division method of the frequency scale. The MFCC is based on the mel frequency scale (analog human auditory characteristics), while the LFCC directly uses the linear frequency scale and is suitable for scenarios that require linear frequency resolution (such as audio analysis, acoustic feature modeling, etc.).

[0078] In one specific embodiment, frequency domain feature extraction is performed on each counterfeit sub-audio to obtain the first counterfeit frequency domain feature corresponding to each counterfeit sub-audio, including: performing frame division and windowing operations on the counterfeit sub-audio to obtain each frame of audio signal. Fast Fourier transform is performed on each frame of audio signal to obtain the amplitude spectrum in the linear frequency domain. The amplitude spectrum in the linear frequency domain is filtered using a linear frequency filter bank to obtain the energy of each filter in the filter bank. Logarithmic operation and discrete cosine transform are performed based on the energy of each filter to obtain the first counterfeit frequency domain feature corresponding to the counterfeit sub-audio.

[0079] (3) determining the counterfeit audio timing feature corresponding to the counterfeit audio based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio.

[0080] In one specific embodiment, the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio are determined as the counterfeit audio timing feature corresponding to the counterfeit audio.

[0081] For example, the counterfeit audio has a length of 8s, the counterfeit audio is divided into 8 counterfeit sub-audios each having a length of 1s, and each first counterfeit frequency domain feature contains 5 coefficients. The plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio are represented using a matrix P1, the matrix P1 contains 8 columns and 5 rows, the matrix P1 is an 8*5 matrix, and the matrix P1 is as follows,

[0082]

[0083] In another specific embodiment, the plurality of counterfeit audio timing features corresponding to the counterfeit audio are determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio, including:

[0084] (1) each counterfeit sub-audio is determined as a target sub-audio.

[0085] (2) determining a feature difference value between the first forged frequency domain feature of the first forged sub-audio after the target sub-audio and the first forged frequency domain feature of the target sub-audio as a neighboring frequency domain change feature of the target sub-audio, to obtain the neighboring frequency domain change feature of each target sub-audio.

[0086] wherein, when there is no forged sub-audio after the target sub-audio, the first forged frequency domain feature of the target sub-audio is determined as the neighboring frequency domain change feature of the target sub-audio.

[0087] For example, the multiple neighboring frequency domain change features corresponding to the multiple forged sub-audios of the forged audio are represented by a matrix P2, the matrix P2 contains 8 columns and 5 rows, the matrix P2 is an 8*5 matrix, and the matrix P2 is as follows,

[0088]

[0089] (3) splicing the multiple first forged frequency domain features and the multiple neighboring frequency domain change features corresponding to the forged audio to obtain multiple second forged frequency domain features of the forged audio.

[0090] Specifically, the multiple second forged frequency domain features of the forged audio are recorded as a matrix P5, the matrix P5 contains 8 columns and 10 rows, the matrix P2 is an 8*10 matrix, and P5 is as follows,

[0091]

[0092] (4) determining the forged audio timing feature corresponding to the forged audio based on the multiple second forged frequency domain features of the forged audio.

[0093] In one specific embodiment, the multiple second forged frequency domain features of the forged audio are determined as the forged audio timing feature corresponding to the forged audio.

[0094] In another specific embodiment, the forged audio timing feature corresponding to the forged audio is determined based on the multiple second forged frequency domain features of the forged audio, including:

[0095] (1) determining a feature difference value between the first forged frequency domain feature of the second forged sub-audio after the target sub-audio and the first forged frequency domain feature of the target sub-audio as an inter-frequency domain change feature of the target sub-audio, to obtain the inter-frequency domain change feature of each target sub-audio.

[0096] wherein, when there is no second forged sub-audio after the target sub-audio, the first forged frequency domain feature of the target sub-audio is determined as the neighboring frequency domain change feature of the target sub-audio.

[0097] For example, the plurality of adjacent frequency domain change features corresponding to the plurality of counterfeit sub-audio of the counterfeit audio are represented by a matrix P3, the matrix P3 contains 8 columns and 5 rows, the matrix P3 is an 8*5 matrix, and the matrix P3 is as follows,

[0098]

[0099] (2) Splicing the plurality of second counterfeit frequency domain features corresponding to the counterfeit audio and the plurality of inter-frequency domain change features to obtain a plurality of third counterfeit frequency domain features of the counterfeit audio.

[0100] Specifically, the plurality of third counterfeit frequency domain features of the counterfeit audio are recorded as a matrix P6, the matrix P6 contains 8 columns and 15 rows, the matrix P6 is an 8*15 matrix, and P6 is as follows,

[0101]

[0102] (3) Determining the counterfeit audio timing feature corresponding to the counterfeit audio based on the plurality of third counterfeit frequency domain features of the counterfeit audio.

[0103] In one specific embodiment, the plurality of third counterfeit frequency domain features of the counterfeit audio are determined as the counterfeit audio timing feature corresponding to the counterfeit audio.

[0104] In another specific embodiment, the counterfeit audio timing feature corresponding to the counterfeit audio is determined based on the plurality of third counterfeit frequency domain features of the counterfeit audio, including:

[0105] The plurality of third counterfeit frequency domain features of the counterfeit audio are input into a second preset feature extraction model for feature extraction to obtain the counterfeit audio timing feature corresponding to the counterfeit audio. The second preset feature extraction model can be a recurrent neural network (RNN) or a Transformer model.

[0106] Specifically, a second preset feature extraction model and a classification model are obtained, the second preset feature extraction model is used to receive audio timing features and input corrected audio timing features, and the classification model is used to classify according to the corrected audio timing features. A plurality of audio samples and corresponding sample labels are obtained, the plurality of audio samples include a plurality of counterfeit audios and a plurality of real audios, and the sample labels are counterfeit categories and counterfeit types, real categories and real types. For example, the sample label of audio sample 1 is: counterfeit category, counterfeit type 1. It indicates that the audio sample 1 belongs to the counterfeit category, and the counterfeit type is specifically counterfeit type 1.

[0107] The second preset feature extraction model and the classification model are iteratively trained based on a plurality of audio samples and corresponding sample labels, to obtain a trained second preset feature extraction model and a trained classification model, and the plurality of third counterfeit frequency domain features of the counterfeit audio are input into the trained second preset feature extraction model for feature extraction, to obtain counterfeit audio timing features corresponding to the counterfeit audio.

[0108] Based on the same principle, in the embodiments of the present application, the timing feature extraction is performed on the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio timing features corresponding to the plurality of counterfeit audios and a plurality of real audio timing features corresponding to the plurality of real audios, and the method further comprises:

[0109] (1) The real audio is split into a plurality of real sub-audios sequentially ordered in time.

[0110] (2) The frequency domain feature extraction is performed on each real sub-audio respectively to obtain a first real frequency domain feature corresponding to each real sub-audio.

[0111] The first real frequency domain feature is a linear frequency cepstral coefficient (LFCC) feature.

[0112] In a specific embodiment, the frequency domain feature extraction is performed on each real sub-audio respectively to obtain a first real frequency domain feature corresponding to each real sub-audio, which comprises: performing a framing and windowing operation on the real sub-audio to obtain each frame of audio signal. The fast Fourier transform is performed on each frame of audio signal to obtain the amplitude spectrum in the linear frequency domain. The linear frequency filter bank is used to filter the amplitude spectrum in the linear frequency domain to obtain the energy of each filter in the filter bank. The logarithmic operation and the discrete cosine transform are performed based on the energy of each filter to obtain the first real frequency domain feature corresponding to the real sub-audio.

[0113] (3) The real audio timing feature corresponding to the real audio is determined based on the plurality of first real frequency domain features corresponding to the plurality of real sub-audios of the real audio.

[0114] In a specific embodiment, the plurality of first real frequency domain features corresponding to the plurality of real sub-audios of the real audio are determined as the real audio timing feature corresponding to the real audio.

[0115] In another specific embodiment, the plurality of real audio timing features corresponding to the real audio are determined based on the plurality of first real frequency domain features corresponding to the plurality of real sub-audios of the real audio, which comprises:

[0116] (1) Each real sub-audio is determined as a target sub-audio.

[0117] (2) determining a feature difference between the first real frequency domain feature of the first real sub-audio after the target sub-audio and the first real frequency domain feature of the target sub-audio as a neighboring frequency domain change feature of the target sub-audio, to obtain the neighboring frequency domain change feature of each target sub-audio.

[0118] wherein, when there is no real sub-audio after the target sub-audio, the first real frequency domain feature of the target sub-audio is determined as the neighboring frequency domain change feature of the target sub-audio.

[0119] (3) splicing the plurality of first real frequency domain features and the plurality of neighboring frequency domain change features corresponding to the real audio to obtain a plurality of second real frequency domain features of the real audio.

[0120] (4) determining a real audio timing feature corresponding to the real audio based on the plurality of second real frequency domain features of the real audio.

[0121] In one specific embodiment, the plurality of second real frequency domain features of the real audio are determined as the real audio timing feature corresponding to the real audio.

[0122] In another specific embodiment, the real audio timing feature corresponding to the real audio is determined based on the plurality of second real frequency domain features of the real audio, comprising:

[0123] (1) determining a feature difference between the first real frequency domain feature of the second real sub-audio after the target sub-audio and the first real frequency domain feature of the target sub-audio as an inter-frequency domain change feature of the target sub-audio, to obtain the inter-frequency domain change feature of each target sub-audio.

[0124] wherein, when there is no second real sub-audio after the target sub-audio, the first real frequency domain feature of the target sub-audio is determined as the neighboring frequency domain change feature of the target sub-audio.

[0125] (2) splicing the plurality of second real frequency domain features and the plurality of inter-frequency domain change features corresponding to the real audio to obtain a plurality of third real frequency domain features of the real audio;

[0126] (3) determining a real audio timing feature corresponding to the real audio based on the plurality of third real frequency domain features of the real audio.

[0127] In one specific embodiment, the plurality of third real frequency domain features of the real audio are determined as the real audio timing feature corresponding to the real audio.

[0128] In another specific embodiment, the real audio timing feature corresponding to the real audio is determined based on the plurality of third real frequency domain features of the real audio, comprising:

[0129] The third real frequency domain features of the real audio are input into a second preset feature extraction model for feature extraction, to obtain real audio time sequence features corresponding to the real audio. The second preset feature extraction model can be a recurrent neural network (RNN) or a Transformer model.

[0130] Specifically, a second preset feature extraction model and a classification model are obtained, the second preset feature extraction model is used to receive audio time sequence features and input modified audio time sequence features, and the classification model is used for classification according to the modified audio time sequence features. A plurality of audio samples and corresponding sample labels are obtained, the plurality of audio samples include a plurality of real audios and a plurality of fake audios, and the sample labels are real categories and real types, real categories and real types. For example, the sample label of the audio sample 1 is: real category, real type 1. It indicates that the audio sample 1 belongs to the real category, and the real type is real type 1.

[0131] The second preset feature extraction model and the classification model are iteratively trained based on the plurality of audio samples and the corresponding sample labels, to obtain a trained second preset feature extraction model and a classification model. The third real frequency domain features of the real audio are input into the trained second preset feature extraction model for feature extraction, to obtain real audio time sequence features corresponding to the real audio.

[0132] In step 103, the plurality of real audio time sequence features and the plurality of real audio time sequence features are respectively divided into different sets, to obtain a plurality of real audio time sequence feature sets and a plurality of real audio time sequence feature sets.

[0133] In one specific embodiment, the plurality of fake audio time sequence features and the plurality of real audio time sequence features are respectively divided into different sets, to obtain a plurality of fake audio time sequence feature sets and a plurality of real audio time sequence feature sets, including: obtaining a fake type of a fake audio corresponding to a fake audio time sequence feature, and putting fake audio time sequence features of the same fake type into the same set as a fake audio time sequence feature set, to obtain a plurality of fake audio time sequence feature sets. For example, in the fake audio time sequence feature set corresponding to the fake type 1, the fake types of the fake audios corresponding to each fake audio time sequence feature are all fake type 1, and in the fake audio time sequence feature set corresponding to the fake type 2, the fake types of the fake audios corresponding to each fake audio time sequence feature are all fake type 2.

[0134] Further, the real type of a real audio corresponding to a real audio time sequence feature is obtained, and real audio time sequence features of the same real type are put into the same set as a real audio time sequence feature set, to obtain a plurality of real audio time sequence feature sets.

[0135] In another specific embodiment, the plurality of fake audio timing features and the plurality of real audio timing features are respectively divided into different sets to obtain a plurality of fake audio timing feature sets and a plurality of real audio timing feature sets, including: clustering the plurality of fake audio timing features to obtain a plurality of feature clustering clusters; and determining one feature clustering cluster as one fake audio timing feature set to obtain the plurality of fake audio timing feature sets.

[0136] Further, the plurality of real audio timing features are clustered to obtain a plurality of feature clustering clusters; and one feature clustering cluster is determined as one real audio timing feature set to obtain the plurality of real audio timing feature sets.

[0137] Step 104, when the to-be-identified audio is acquired, the first similarity parameter of the to-be-identified audio and the plurality of fake audio timing feature sets, and the second similarity parameter of the to-be-identified audio and the plurality of real audio timing feature sets are respectively calculated.

[0138] In the embodiments of the present application, the first similarity parameter of the to-be-identified audio and the plurality of fake audio timing feature sets, and the second similarity parameter of the to-be-identified audio and the plurality of real audio timing feature sets are respectively calculated, including:

[0139] (1) The average value of the plurality of fake audio timing features in the fake audio timing feature set is determined as the set audio feature of the fake audio timing feature set.

[0140] (2) The audio feature similarity between the audio feature of the to-be-identified audio and the plurality of set audio features of the plurality of fake audio timing feature sets is calculated to obtain the plurality of audio feature similarities of the plurality of fake audio timing feature sets.

[0141] In the embodiments of the present application, the set audio feature is denoted as m ij , where i∈{0,1} represents the true or false category, i=0 represents the real category, i=1 represents the fake category, and j represents the serial number of the set corresponding to the set audio feature.

[0142] In the embodiments of the present application, the audio feature similarity between the audio feature of the to-be-identified audio and the set audio feature of the fake audio timing feature set is calculated to obtain the audio feature similarity of the fake audio timing feature set. The audio feature similarity can be a cosine distance, an Euclidean distance, etc.

[0143] In the embodiments of the present application, the audio feature similarity is an Euclidean distance. For the to-be-identified audio x, the feature f(x) of the to-be-identified audio x is extracted to calculate the Euclidean distance between the feature f(x) of the to-be-identified audio x and each set audio feature m ij , and the Euclidean distance is used to measure the to-be-identified audio x and the set audio feature mij The similarity between the feature f(x) of the audio to be identified x and each set audio feature m ij The Euclidean distance between f(x) and m

[0144]

[0145] where d(f(x), m ij ) represents the Euclidean distance between the feature f(x) of the audio to be identified x and the set audio feature m ij , that is, the audio feature similarity.

[0146] (3) Determine the maximum value of the plurality of audio feature similarities of the plurality of counterfeit audio time sequence feature sets as the first similarity parameter.

[0147] For the plurality of set audio features of the plurality of counterfeit audio time sequence feature sets of the counterfeit category, the first similarity parameter d1 is calculated as follows:

[0148] d1 = max d(f(x), m ij ), i = 1, j ∈ {1,..., K}.

[0149] where d1 represents the first similarity parameter, d(f(x), m ij ) represents the audio feature similarity between the feature f(x) of the audio to be identified x and the set audio feature m ij .

[0150] In the embodiments of the present application, the first similarity parameter of the audio to be identified and the plurality of counterfeit audio time sequence feature sets, and the second similarity parameter of the audio to be identified and the plurality of real audio time sequence feature sets are calculated respectively, including:

[0151] (1) Determine the average value of the plurality of real audio time sequence features in the real audio time sequence feature set as the set audio feature of the real audio time sequence feature set;

[0152] (2) Calculate the audio feature similarity between the audio feature of the audio to be identified and the plurality of set audio features of the plurality of real audio time sequence feature sets, to obtain the plurality of audio feature similarities of the plurality of real audio time sequence feature sets.

[0153] (3) Weighted sum the plurality of audio feature similarities of the plurality of real audio time sequence feature sets based on the set weight of each real audio time sequence feature set, to obtain the second similarity parameter.

[0154] In a specific embodiment, a feature quantity parameter of the real audio timing feature in the real audio timing feature set is obtained, a signal-to-noise ratio parameter of the real audio corresponding to the real audio timing feature set is obtained, and a set weight γ of the real audio timing feature set is determined based on the feature quantity parameter and the signal-to-noise ratio parameter of the real audio timing feature set j . The set weight corresponding to the jth real audio timing feature set is γ j . The signal-to-noise ratio (SNR) is the ratio of the power of the audio signal to the power of the noise, and the unit is decibel (dB). The higher the SNR, the less the noise (such as current sound and background noise) affects the sound quality.

[0155] In determining the set weight of the real audio timing feature set, the feature quantity parameter and the signal-to-noise ratio parameter can be normalized to map them to the [0, 1] interval, and then the comprehensive weight is calculated through a preset weight distribution formula. Specifically, let the feature quantity parameter be N (reflecting the richness of the feature dimension), the signal-to-noise ratio parameter be SNR (reflecting the purity of the signal), the normalized feature quantity be N j , and the normalized signal-to-noise ratio be S j , then the set weight γ j can be represented as γ j = a * N j + b * S j , where a and b are both in [0, 1] and can be set according to specific circumstances.

[0156] Suppose that the number of real audio timing features in one of the real audio timing feature sets is 10, the feature quantity parameter is 10, and the signal-to-noise ratio SNR of the audio is 25 dB. After normalization, the feature quantity parameter is 0.5, and the normalized signal-to-noise ratio is 0.5. If a = 0.6 and b = 0.4 are set, the jth set weight γ j = 0.5.

[0157] For multiple set audio features of multiple real audio timing feature sets of the real category, the second similarity parameter d2 is calculated as follows:

[0158]

[0159] where γ j represents the set weight corresponding to the jth real audio timing feature set.

[0160] In the embodiments of the present application, the hardware parameters of the electronic device are obtained, the maximum process quantity and the maximum processing quantity of each process are determined based on the GPU hardware parameters of the electronic device. A plurality of processes are established based on the maximum process quantity and the maximum processing quantity of each process, and each process is controlled to process a corresponding number of real audios.

[0161] Step 105: If the first similarity parameter is greater than the second similarity parameter, then the audio category of the audio to be identified is determined to be a fake category.

[0162] Furthermore, if the first similarity parameter is less than the second similarity parameter, then the audio category of the audio to be identified is determined to be the true category.

[0163] Compared to existing technologies, in this embodiment, multiple fake audio files and multiple real audio files are acquired; temporal features are extracted from the multiple fake audio files and multiple real audio files to obtain multiple fake audio temporal features corresponding to the multiple fake audio files and multiple real audio temporal features corresponding to the multiple real audio files; the multiple fake audio temporal features and multiple real audio temporal features are respectively divided into different sets to obtain multiple fake audio temporal feature sets and multiple real audio temporal feature sets; when an audio file to be identified is acquired, a first similarity parameter between the audio file to be identified and the multiple fake audio temporal feature sets and a second similarity parameter between the audio file to be identified and the multiple real audio temporal feature sets are calculated respectively; if the first similarity parameter is greater than the second similarity parameter, the audio category of the audio file to be identified is determined to be a fake category. This application calculates a first similarity parameter between the audio to be identified and a set of fake audio temporal features containing multiple fake audio temporal features, and then calculates a second similarity parameter between the audio to be identified and a set of real audio temporal features containing multiple real audio temporal features. Based on the first and second similarity parameters, it determines whether the audio category of the audio to be identified is a fake category. It can utilize rich fake feature information and improve the accuracy of audio fake detection.

[0164] See Figure 3 ,like Figure 3 The diagram shows a structural schematic of an audio authentication device. The audio authentication device in this embodiment can achieve the functionality described above. Figure 2 The steps of the audio authentication method executed in the corresponding embodiments are described below. The functions implemented by the audio authentication device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The audio authentication device 60 may include an acquisition module 601, a feature extraction module 602, a segmentation module 603, a calculation module 604, and a determination module 605. The functional implementation of these modules can be found in [reference needed]. Figure 2 The operations performed in the corresponding embodiments will not be described in detail here.

[0165] The audio authentication device includes:

[0166] The acquisition module 601 is configured to acquire multiple fake audio files and multiple real audio files;

[0167] The feature extraction module 602 is configured to extract temporal features from multiple fake audios and multiple real audios to obtain multiple fake audio temporal features corresponding to the multiple fake audios and multiple real audio temporal features corresponding to the multiple real audios.

[0168] The partitioning module 603 is configured to partition the multiple forged audio timing features and the multiple real audio timing features into different sets, thereby obtaining multiple sets of forged audio timing features and multiple sets of real audio timing features.

[0169] The calculation module 604 is configured to, when the audio to be identified is acquired, calculate the first similarity parameter between the audio to be identified and multiple sets of temporal features of fake audio, and the second similarity parameter between the audio to be identified and multiple sets of temporal features of real audio.

[0170] The determination module 605 is configured to determine the audio category of the audio to be identified as a fake category if the first similarity parameter is greater than the second similarity parameter.

[0171] The audio authentication device 60 in this application embodiment has been described above from the perspective of modular functional entities. The audio authentication device in this application embodiment will be described below from the perspective of hardware processing.

[0172] Figure 3 The devices shown can all have the following characteristics: Figure 4 The structure shown, when Figure 3 The audio authentication device 60 shown has the following features: Figure 4 When the structure shown is used, Figure 4 The processor and transceiver in the device can perform the same or similar functions as the modules provided in the aforementioned device embodiments. Figure 4 The memory in the processor stores the computer program that needs to be called when executing the above audio authentication method.

[0173] This application also provides a terminal device, such as... Figure 5 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablet computers, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0174] Figure 5 This diagram illustrates a partial structure of a mobile phone related to the terminal device provided in the embodiments of this application. (Reference) Figure 5The mobile phone includes Radio Frequency (RF) circuit 1010, memory 1020, input unit 1030, display unit 1040, sensor 1050, audio circuit 1060, wireless fidelity (WiFi) module 1070, processor 1080, and power supply 1090, etc. Those skilled in the art can understand that the structure of the mobile phone shown in the figure does not constitute a limitation on the mobile phone, and can include more or less components than the figure, or combine certain components, or different component arrangements. Figure 5 The structure of the mobile phone shown in the figure does not constitute a limitation on the mobile phone, and can include more or less components than the figure, or combine certain components, or different component arrangements.

[0175] The specific introduction of each component of the mobile phone is as follows: Figure 5 The specific introduction of each component of the mobile phone is as follows:

[0176] The RF circuit 1010 can be used for receiving and sending signals in the process of information or call, especially receiving the downlink information of the base station and processing it by the processor 1080, and sending the uplink data to the base station. Usually, the RF circuit 1010 includes but is not limited to antenna, at least one amplifier, transceiver, coupler, Low Noise Amplifier (LNA), duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0177] The memory 1020 can be used to store software programs and modules, and the processor 1080 can execute various function applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0178] The input unit 1030 can be used to receive inputted digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 can include a touch panel 1031 and other input devices 1032. The touch panel 1031, also called a touch screen, can collect the touch operation of a user thereon or nearby (such as the operation of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection device according to the pre-set program. Optionally, the touch panel 1031 can include two parts of a touch detection device and a touch controller. The touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into touch coordinates, and then sends it to the processor 1080, and can also receive the command from the processor 1080 and execute it. In addition, the touch panel 1031 can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel 1031, the input unit 1030 can also include other input devices 1032. Specifically, the other input devices 1032 can include one or more of a physical keyboard, a function key (such as a volume control key, an on-off key, etc.), a trackball, a mouse, a joystick, and the like.

[0179] The display unit 1040 can be used to display information input by a user or information provided to the user as well as various menus of the phone. The display unit 1040 can include a display panel 1041, which can be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, a touch panel 1031 can cover the display panel 1041, and when the touch panel 1031 detects a touch operation thereon or nearby, it transmits to the processor 1080 to determine the type of touch event, and then the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in the above embodiment, the touch panel 1031 and the display panel 1041 are implemented as two independent components to realize the input and output functions of the phone, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the phone. Figure 5

[0180] The phone can also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1041 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and when at rest, it can detect the magnitude and direction of gravity, which can be used for applications that identify the posture of the phone (such as landscape / portrait screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. As for other sensors that the phone can also be configured, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, they will not be described here.

[0181] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the phone. The audio circuit 1060 can convert received audio data into an electrical signal, which is transmitted to the speaker 1061 to be converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 to be converted into audio data, which is then output to the processor 1080 for processing, and then transmitted to another phone via the RF circuit 1010, or output to the memory 1020 for further processing.

[0182] ​Wi-Fi belongs to short distance wireless transmission technology, and the mobile phone can help users send and receive e-mails, browse web pages and access streaming media through the Wi-Fi module 1070, which provides users with wireless broadband Internet access. Although Figure 5 The Wi-Fi module 1070 is shown, but it can be understood that it does not belong to the necessary structure of the mobile phone, and can be omitted as needed without changing the essence of the application.

[0183] The processor 1080 is the control center of the mobile phone, which connects all parts of the mobile phone through various interfaces and lines, executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1020 and calling data stored in the memory 1020, thereby monitoring the mobile phone as a whole. Optionally, the processor 1080 can include one or more processing units; optionally, the processor 1080 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1080.

[0184] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 1080 through a power management system, so as to realize the functions of managing charging, discharging and power consumption management through the power management system.

[0185] Although not shown, the mobile phone can also include a camera, a Bluetooth module, etc., which will not be described here.

[0186] In the embodiments of the present application, the processor 1080 included in the mobile phone also has the function of controlling the execution of the above audio authentication method flow executed by the audio authentication device.

[0187] The embodiments of the present application also provide a server, please refer to Figure 6 , Figure 6Figure 1 shows a schematic diagram of a server structure according to an embodiment of the present application. The server 1100 can have a great difference due to different configurations or performances, and can include one or more central processing units (CPU) 1122 (e.g., one or more processors) and a memory 1132, one or more storage media 1130 (e.g., one or more mass storage devices) storing application programs 1142 or data 1144. The memory 1132 and the storage media 1130 can be temporary storage or persistent storage. The programs stored in the storage media 1130 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations in the server. Further, the central processing unit 1122 can be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0188] The server 1100 can also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0189] The steps performed by the server in the above embodiments can be based on the server structure shown in Figure 1. For example, the steps performed by the server in the above embodiments can be based on the server structure shown in Figure 1. Figure 6 The steps performed by the server in the above embodiments can be based on the server structure shown in Figure 1. For example, the steps performed by the server in the above embodiments can be based on the server structure shown in Figure 1. Figure 6 The steps performed by the audio authentication device 60 in the above embodiments can be based on the server structure shown in Figure 1. For example, the central processing unit 1122 executes the following operations by invoking instructions in the memory 1132: Figure 6 The steps performed by the audio authentication device 60 in the above embodiments can be based on the server structure shown in Figure 1. For example, the central processing unit 1122 executes the following operations by invoking instructions in the memory 1132:

[0190] Obtaining a plurality of counterfeit audios and a plurality of real audios;

[0191] Extracting timing features from the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio timing features corresponding to the plurality of counterfeit audios and a plurality of real audio timing features corresponding to the plurality of real audios;

[0192] Dividing the plurality of counterfeit audio timing features and the plurality of real audio timing features into different sets respectively to obtain a plurality of counterfeit audio timing feature sets and a plurality of real audio timing feature sets;

[0193] When the to-be-identified audio is acquired, a first similarity parameter of the to-be-identified audio and a plurality of counterfeit audio time sequence feature sets is calculated, and a second similarity parameter of the to-be-identified audio and a plurality of real audio time sequence feature sets is calculated;

[0194] If the first similarity parameter is greater than the second similarity parameter, it is determined that the audio category of the to-be-identified audio is a counterfeit category.

[0195] In an embodiment, the time sequence feature extraction of the plurality of counterfeit audios and the plurality of real audios obtains a plurality of counterfeit audio time sequence features corresponding to the plurality of counterfeit audios and a plurality of real audio time sequence features corresponding to the plurality of real audios, which includes:

[0196] The counterfeit audio is split into a plurality of counterfeit sub-audios sequentially ordered in time;

[0197] The frequency domain feature extraction is performed on each counterfeit sub-audio to obtain a first counterfeit frequency domain feature corresponding to each counterfeit sub-audio;

[0198] The counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio.

[0199] In an embodiment, the determination of the counterfeit audio time sequence feature corresponding to the counterfeit audio based on the plurality of second counterfeit frequency domain features of the counterfeit audio includes:

[0200] Each counterfeit sub-audio is determined as a target sub-audio;

[0201] The feature difference between the first counterfeit frequency domain feature of the first counterfeit sub-audio after the target sub-audio and the first counterfeit frequency domain feature of the target sub-audio is determined as a neighboring frequency domain change feature of the target sub-audio, and a neighboring frequency domain change feature of each target sub-audio is obtained;

[0202] The plurality of first counterfeit frequency domain features and the plurality of neighboring frequency domain change features corresponding to the counterfeit audio are spliced to obtain a plurality of second counterfeit frequency domain features of the counterfeit audio;

[0203] The counterfeit audio time sequence feature corresponding to the counterfeit audio is determined based on the plurality of second counterfeit frequency domain features of the counterfeit audio.

[0204] In an embodiment, the determination of the counterfeit audio time sequence feature corresponding to the counterfeit audio based on the plurality of second counterfeit frequency domain features of the counterfeit audio includes:

[0205] determining a feature difference value between the first counterfeit frequency domain feature of a second counterfeit sub-audio after the target sub-audio and the first counterfeit frequency domain feature of the target sub-audio as an inter-phase frequency domain variation feature of the target sub-audio, to obtain an inter-phase frequency domain variation feature of each target sub-audio;

[0206] splicing a plurality of second counterfeit frequency domain features and a plurality of inter-phase frequency domain variation features corresponding to the counterfeit audio to obtain a plurality of third counterfeit frequency domain features of the counterfeit audio;

[0207] determining the counterfeit audio timing feature corresponding to the counterfeit audio based on a plurality of third counterfeit frequency domain features of the counterfeit audio.

[0208] In one embodiment, the first similarity parameter of the to-be-identified audio and a plurality of counterfeit audio timing feature sets and the second similarity parameter of the to-be-identified audio and a plurality of real audio timing feature sets are calculated respectively, comprising:

[0209] determining an average value of a plurality of counterfeit audio timing features in the counterfeit audio timing feature set as a set audio feature of the counterfeit audio timing feature set;

[0210] calculating an audio feature similarity between the audio feature of the to-be-identified audio and a plurality of set audio features of a plurality of counterfeit audio timing feature sets to obtain a plurality of audio feature similarities of a plurality of counterfeit audio timing feature sets;

[0211] determining a maximum value of a plurality of audio feature similarities of a plurality of counterfeit audio timing feature sets as the first similarity parameter.

[0212] In one embodiment, the timing feature extraction of a plurality of counterfeit audios and a plurality of real audios is performed to obtain a plurality of counterfeit audio timing features corresponding to a plurality of counterfeit audios and a plurality of real audio timing features corresponding to a plurality of real audios, comprising:

[0213] frame and windowing operations are performed on the counterfeit audio to obtain each frame of audio signal;

[0214] fast Fourier transform is performed on each frame of audio signal to obtain an amplitude spectrum in a linear frequency domain;

[0215] linear frequency filter banks are used to filter the amplitude spectrum in the linear frequency domain to obtain an energy of each filter in the filter bank;

[0216] logarithmic operation and discrete cosine transform are performed based on the energy of each filter to obtain the counterfeit audio timing feature of the counterfeit audio.

[0217] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0218] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and module described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0219] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0220] The modules described as separate components can or can not be physically separate, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place, or can be distributed to a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0221] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of software functional module. If the integrated module is realized in the form of software functional module and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0222] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various optional implementation manners.

[0223] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part.

[0224] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on the computer, the flow or function according to the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that the computer can store or be integrated into a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0225] The above describes the technical solutions provided by the embodiments of the present application in detail. The principles and implementation manners of the embodiments of the present application are described by applying specific examples. The above examples are only used to help understand the method and core idea of the embodiments of the present application; at the same time, for those skilled in the art, according to the idea of the embodiments of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the embodiments of the present application.

Claims

1. An audio authentication method, characterized by, The audio authentication method comprises: Obtaining a plurality of counterfeit audios and a plurality of real audios; Temporal feature extraction is performed on the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio temporal features corresponding to the plurality of counterfeit audios and a plurality of real audio temporal features corresponding to the plurality of real audios; The plurality of counterfeit audio temporal features and the plurality of real audio temporal features are respectively divided into different sets to obtain a plurality of counterfeit audio temporal feature sets and a plurality of real audio temporal feature sets; When a to-be-identified audio is obtained, a first similarity parameter between the to-be-identified audio and the plurality of counterfeit audio temporal feature sets, and a second similarity parameter between the to-be-identified audio and the plurality of real audio temporal feature sets are calculated, an average value of the plurality of counterfeit audio temporal features in the counterfeit audio temporal feature set is determined as a set audio feature of the counterfeit audio temporal feature set, an audio feature similarity between an audio feature of the to-be-identified audio and a plurality of set audio features of the plurality of counterfeit audio temporal feature sets is calculated to obtain a plurality of audio feature similarities of the plurality of counterfeit audio temporal feature sets, and a maximum value of the plurality of audio feature similarities of the plurality of counterfeit audio temporal feature sets is determined as the first similarity parameter; If the first similarity parameter is greater than the second similarity parameter, it is determined that an audio category of the to-be-identified audio is a counterfeit category.

2. The audio authentication method of claim 1, wherein, The temporal feature extraction on the plurality of counterfeit audios and the plurality of real audios to obtain a plurality of counterfeit audio temporal features corresponding to the plurality of counterfeit audios and a plurality of real audio temporal features corresponding to the plurality of real audios comprises: The counterfeit audio is split into a plurality of counterfeit sub-audios sequentially sorted in time; Frequency domain feature extraction is performed on each counterfeit sub-audio to obtain a first counterfeit frequency domain feature corresponding to each counterfeit sub-audio; The counterfeit audio temporal feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio.

3. The audio authentication method of claim 2, wherein, The counterfeit audio temporal feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio. The counterfeit audio temporal feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio. The counterfeit audio temporal feature corresponding to the counterfeit audio is determined based on the plurality of first counterfeit frequency domain features corresponding to the plurality of counterfeit sub-audios of the counterfeit audio. ​ ​ 4. The audio authentication method of claim 3, wherein, ​ determining a feature difference between the first fake frequency domain feature of a second fake sub-audio after the target sub-audio and the first fake frequency domain feature of the target sub-audio as an inter-phase frequency domain change feature of the target sub-audio, to obtain an inter-phase frequency domain change feature of each target sub-audio; splicing the second fake frequency domain features and the inter-phase frequency domain change features corresponding to the fake audios to obtain third fake frequency domain features of the fake audios; determining fake audio timing features corresponding to the fake audios based on the third fake frequency domain features of the fake audios.

5. The audio authentication method of claim 1, wherein, The timing feature extraction on the plurality of fake audios and the plurality of real audios to obtain a plurality of fake audio timing features corresponding to the plurality of fake audios and a plurality of real audio timing features corresponding to the plurality of real audios, includes: performing frame division and windowing operation on the fake audios to obtain each frame of audio signal; performing fast Fourier transform on each frame of audio signal to obtain amplitude spectrum in linear frequency domain; filtering the amplitude spectrum in linear frequency domain using a linear frequency filter bank to obtain energy of each filter in the filter bank; performing logarithmic operation and discrete cosine transform based on the energy of each filter to obtain the fake audio timing features of the fake audios.

6. An audio authentication device, characterized by The audio authentication device includes: an acquisition module configured to acquire a plurality of fake audios and a plurality of real audios; a feature extraction module configured to perform timing feature extraction on the plurality of fake audios and the plurality of real audios to obtain a plurality of fake audio timing features corresponding to the plurality of fake audios and a plurality of real audio timing features corresponding to the plurality of real audios; a division module configured to respectively divide the plurality of fake audio timing features and the plurality of real audio timing features into different sets to obtain a plurality of fake audio timing feature sets and a plurality of real audio timing feature sets; a calculation module configured to, when a to-be-identified audio is acquired, respectively calculate a first similarity parameter of the to-be-identified audio and the plurality of fake audio timing feature sets, a second similarity parameter of the to-be-identified audio and the plurality of real audio timing feature sets, determine an average value of the plurality of fake audio timing features in the fake audio timing feature set as a set audio feature of the fake audio timing feature set, calculate an audio feature similarity between an audio feature of the to-be-identified audio and a plurality of set audio features of the plurality of fake audio timing feature sets to obtain a plurality of audio feature similarities of the plurality of fake audio timing feature sets, and determine a maximum value of the plurality of audio feature similarities of the plurality of fake audio timing feature sets as the first similarity parameter; a determination module configured to, if the first similarity parameter is greater than the second similarity parameter, determine that an audio category of the to-be-identified audio is a fake category.

7. A computing device, comprising: It includes a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1-5 when executing the computer program. It includes a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, which comprises instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-5.

9. A computer program product comprising instructions, which, when executed on a computer or processor, cause the computer or the processor to perform the method of any one of claims 1-5.

Images