Forged telecommunication audio detection method, computer device and storage medium

Abstract

The invention discloses a forged telecommunication audio detection method, a computer device and a storage medium, and the method comprises the steps: obtaining first voice data of a first voice communication party in a first voice collection period, extracting first statistical feature data and first biological feature data from the first voice data, and inputting a first input tensor obtained by processing the first statistical feature data and the first biological feature data into the artificial intelligence model for processing, and obtaining a first detection result of the artificial intelligence model. The method can capture the first statistical feature data, the first biological feature data and other features in the first voice data, thereby accurately and efficiently judging whether the first voice data of the first voice communication party belongs to the forged telecommunication audio or not, selecting to execute operations such as release, ensuring normal voice passage, and improving the user experience. The possibility that bad voice information generated by a computer is propagated through a telecommunication voice communication network is reduced, and communication safety is guaranteed. The method is widely applied to the technical field of communication.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a method for detecting counterfeit telecommunication audio, a computer device and a storage medium. Background Art

[0002] Telecom fraud is a method of deception carried out by using telecommunications networks. Traditional telecommunications fraud is that a person makes a phone call and talks to the person who answers the call, and then manually speaks the relevant content to achieve improper purposes.

[0003] One form of telecommunications fraud is to imitate the voice of a relative, colleague, boss, or other specific person that the person answering the call may know, so that the person answering the call mistakenly believes that they are talking to a specific person, in order to increase the credibility of the lie. The person making the call and imitating the way of speaking is easily detected by the person answering the call. Currently, deep fake audio technology has emerged, such as using computer technology to generate voice, and the timbre of the voice can be adjusted to be the same as any specific person. Telecommunications fraud using deep fake audio technology is to play the computer-generated voice to the person answering the call after dialing the person answering the call. In other words, the person answering the call is actually having a voice call with a computer, not a voice call with a real human.

[0004] Telecommunications fraud using deep fake audio technology has reached an unprecedented level of concealment and deception. The deceiver can easily imitate the voice of a specific person known to the caller, implement precise and efficient deception actions, and cause serious harm. However, due to the limitations of factors such as the knowledge level and hearing level of the caller, it is difficult for the caller to determine whether the voice of the call is generated using deep fake audio technology. Summary of the invention

[0005] In response to the current technical problem that it is difficult to determine whether the voice of the other party in a voice call is generated using deep fake audio technology, the purpose of the present invention is to provide a fake telecommunications audio detection method, computer device and storage medium.

[0006] In one aspect, an embodiment of the present invention includes a method for detecting counterfeit telecommunication audio, the method comprising the following steps:

[0007] determining a first speech collection period;

[0008] During the first voice collection period, obtaining first voice data of a first voice communication party;

[0009] extracting at least one first statistical feature data and at least one first biometric feature data from the first voice data;

[0010] Processing the first statistical feature data and the first biometric feature data to obtain a first input tensor;

[0011] The first input tensor is input into an artificial intelligence model for processing to obtain a first detection result of the artificial intelligence model; the first detection result indicates whether the first voice data belongs to forged telecommunication audio or not.

[0012] Further, determining the first voice collection period includes:

[0013] Obtaining a call initiation request; the call initiation request is used to request to establish a voice call process between a first voice communication party and a second voice communication party;

[0014] In response to the call initiation request, a time period before the voice call process is established is determined as the first voice collection period.

[0015] Further, extracting at least one first statistical feature data and at least one first biometric feature data from the first voice data includes:

[0016] Acquire frequency domain features of the first speech data;

[0017] According to the frequency domain characteristics, obtaining the F ratio of each frequency band;

[0018] Obtaining a mean value of pitch variation of the first speech data;

[0019] Acquire the average breathing interval of the first voice data;

[0020] The F ratio and the pitch variation mean are used as the first statistical feature data, and the average breathing interval is used as the first biometric feature data.

[0021] Furthermore, the processing of the first statistical feature data and the first biometric feature data to obtain a first input tensor includes:

[0022] The first statistical feature data and the first biometric feature data are folded to obtain the first input tensor.

[0023] Furthermore, the forged telecommunication audio detection method further comprises:

[0024] When the first detection result indicates that the first voice data is not forged telecommunication audio, releasing the first voice data;

[0025] When the first detection result indicates that the first voice data is forged telecommunication audio, a warning message is generated, the warning message is sent to the second voice communication party, selection information of the second voice communication party is obtained, and according to the selection information, the first voice data is released or the voice call process is hung up;

[0026] or

[0027] When the first detection result indicates that the first voice data is forged telecommunication audio, user attribute information of the first voice communication party is obtained, and the first voice data is released or the voice call process is hung up according to the user attribute information.

[0028] Furthermore, the forged telecommunication audio detection method further comprises:

[0029] When the first voice data is released according to the selection information, backing up the first voice data and the warning information;

[0030] After the voice call process ends, the first voice data and the warning information are sent to the second voice communication party.

[0031] Further, determining the first voice collection period includes:

[0032] Obtaining a call initiation request; the call initiation request is used to request to establish a voice call process between a first voice communication party and a second voice communication party;

[0033] In response to the call initiation request, establishing the voice call process;

[0034] A time period during which the voice call is in progress is determined as the first voice collection period.

[0035] Furthermore, the forged telecommunication audio detection method further comprises:

[0036] When the first detection result indicates that the first voice data is forged telecommunication audio, interrupting the voice call process;

[0037] After interrupting the voice call process, resuming the voice call process;

[0038] Determining a second voice collection period during a time period after the voice call process is resumed;

[0039] During the second voice collection period, obtaining second voice data of the first voice communication party;

[0040] extracting at least one second statistical feature data and at least one second biometric feature data from the second voice data;

[0041] Comparing the second statistical feature data and the second biometric feature data with the first statistical feature data and the first biometric feature data to obtain a comparison similarity;

[0042] According to the comparison similarity, the second voice data is released or the voice call process is terminated.

[0043] On the other hand, an embodiment of the present invention further includes a computer device, including a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load at least one program to execute the counterfeit telecommunication audio detection method in the embodiment.

[0044] On the other hand, an embodiment of the present invention further includes a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the counterfeit telecommunication audio detection method in the embodiment.

[0045] The beneficial effect of the present invention is that through the forged telecommunications audio detection method in the embodiment, by using an artificial intelligence model to process the first input tensor, it is possible to capture features such as the first statistical feature data and the first biometric data in the first voice data, thereby accurately and efficiently judging whether the first voice data of the first voice communication party belongs to forged telecommunications audio, thereby choosing to execute operations such as release or hang up (blocking), which can ensure the normal voice passage, reduce the possibility of computer-generated bad voice information being transmitted through the telecommunications voice communication network, and ensure communication security. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A structural diagram of a communication system to which the forged telecommunication audio detection method can be applied in an embodiment;

[0047] Figure 2 This is a schematic diagram of the structure of the IMS in the embodiment;

[0048] Figure 3 This is a schematic diagram of the interface of the AAV_AS in the embodiment;

[0049] Figure 4 A schematic diagram of a calling side signaling process in an embodiment;

[0050] Figure 5 A schematic diagram of a signaling process on the called side in an embodiment;

[0051] Figure 6 A schematic diagram of the steps of the method for detecting counterfeit telecommunication audio in an embodiment;

[0052] Figure 7 is a schematic diagram of the first speech collection period in the embodiment;

[0053] Figure 8is a schematic diagram of an encryption process of first voice data in an embodiment;

[0054] Fig. 9 is a schematic diagram of a decryption and reassembly process of first voice data in an embodiment;

[0055] Fig.10 A schematic diagram of extracting feature data from first speech data in an embodiment;

[0056] Fig.11 It is a schematic diagram of the structure of a CNN convolutional neural network in an embodiment;

[0057] Fig.12 Schematic diagram of the process of steps S6-S7 in the embodiment;

[0058] Fig.13 A schematic diagram of a disaster recovery process in an embodiment;

[0059] Fig.14 Schematic diagram of the principles of steps S8-S10 in the embodiment. DETAILED DESCRIPTION

[0060] Terminology explanation:

[0061] VoLTE (Voice over Long-Term Evolution) service: HD voice service refers to the voice service provided through the LTE network, providing a higher quality call experience.

[0062] IMS (IP Multimedia Subsystem): IP Multimedia Subsystem, an IP-based communication framework used to provide multimedia services such as voice, video, and messaging.

[0063] UE (User Equipment): User equipment refers to the terminal device used by users, such as a smartphone or computer, to access the network.

[0064] S_CSCF (Serving Call Session Control Function): A component in the IMS network that is responsible for establishing and controlling call sessions.

[0065] P_CSCF (Proxy Call Session Control Function): A component in the IMS network that acts as a proxy between user equipment and the IMS core network, processing registration and session control messages.

[0066] MAR (Message Authorization Request): Message authorization request, which is a request used in telecommunications networks to verify the source and content of a message to ensure the legitimacy and security of the message.

[0067] HSS (Home Subscriber Server): A home subscriber server is a key database in an IMS network, used to store user identity information, service subscription information, and authentication information.

[0068] MRF (Multimedia Resource Function): A component in the IMS network that is responsible for the management and control of multimedia resources, such as the playback and recording of audio and video streams.

[0069] IP_SM_GW (IP Short Message Gateway): IP short message gateway, used to process the sending and receiving of IP short messages and transfer short messages from one network to another.

[0070] AAV_AS (Audio Authentication and Verification Application Server): Audio authentication and verification application server is a core component mentioned in the present invention, responsible for the collection, analysis and verification of audio data to detect and defend against audio forgery.

[0071] In this embodiment, the method for detecting counterfeit telecommunication audio can be applied to Figure 1 In the communication system shown. Figure 1 , the first voice communication party and the second voice communication party are two parties in a voice call process (already established or about to be established), and the first voice communication party and the second voice communication party are both connected to the IMS. In this embodiment, the first voice communication party and the second voice communication party can be devices with communication functions such as mobile phones, landlines or servers, respectively. Among them, the first voice communication party is a party that (suspectedly) intends to have a voice call process with the second voice communication party through the IMS, thereby sending a voice imitating a specific person to the second voice communication party. Since the telecommunications spoofer is usually the calling side, in this embodiment, the first voice communication party is used as the calling side and the second voice communication party is used as the called side for explanation. When applying the forged telecommunications audio detection method, the first voice communication party can also be the called side, and the second voice communication party can also be the calling side.

[0072] In this embodiment, the structure of IMS (IP Multimedia Subsystem) is as follows Figure 2 See Figure 2, AAV_AS (Audio Authentication and Verification Application Server) is the main part of IMS, Figure 2 The core component AAV_AS and its interaction mechanism with related nodes in the IMS network are demonstrated, aiming to build a protection system against illegal audio forgery, which can ensure the security and authenticity of calls between the first voice communication party and the second voice communication party.

[0073] In this embodiment, Figure 2 In the IMS structure shown in the figure, the interface design of AAV_AS is as follows Figure 3 See Figure 3 , the specific interfaces are:

[0074] X1: Used by AAV_AS and P_CSCF to transmit multimedia data using RTP and RTCP protocols.

[0075] C1: Used by AAV_AS and MAU to initiate playback requests using the SIP protocol.

[0076] ISC: used by AAV_AS and S_CSCF to transmit authorization messages using the SIP protocol.

[0077] In this embodiment, the audio authentication and verification application server AAV_AS (Audio Authentication and Verification Application Server) is mainly composed of five key modules: service control module, encryption module, database module, decryption / preprocessing module and neural network analysis module. The functions of each module are as follows:

[0078] Service control module: As the core dispatching center of the system, this module is responsible for receiving and responding to users' SIP requests, managing the forwarding and secure deletion of user data, generating and sending the SIP message results of the caller's audio authentication analysis to the S-CSCF on the called side, and is also responsible for the reorganization of user audio packets and sending ringback tone reminders to the called party.

[0079] Encryption module: To ensure the security and privacy of user audio data, the encryption module encrypts the user data before storing it in the database; correspondingly, the decryption module decrypts the data immediately after it is read from the database, thereby ensuring that user data will not be leaked during the entire processing process.

[0080] Database module: The database module is mainly used to cache encrypted user audio data packets to ensure that data security is maintained even during the data storage stage.

[0081] Decryption / preprocessing module: The decryption / preprocessing module is responsible for decrypting the user's audio data packets and reassembling the decrypted data packets into complete audio files. At the same time, it uses traditional audio forgery detection technology to extract the key feature values ​​of the decrypted and reassembled audio data to prepare for subsequent in-depth analysis.

[0082] Neural network analysis module: The neural network analysis module uses advanced neural network technology to perform in-depth analysis on the pre-processed audio data, including convolution processing and feature mapping, and finally passes the analysis results to AAV_AS (Automatic Audio Verification Application Server) to complete the entire authentication process.

[0083] In this embodiment, Figure 1 The steps in the forged telecommunication audio detection method can be performed by the IMS in the IMS. Figure 2 The AAV_AS shown is used to perform various steps in the counterfeit telecommunication audio detection method.

[0084] In this embodiment, each step in the forged telecommunication audio detection method can be executed when the voice call process between the first voice communication party and the second voice communication party is about to be established or after the voice call process is established. Therefore, before executing each step in the forged telecommunication audio detection method, the call process between the first voice communication party (calling party) and the second voice communication party (called party) can be executed first.

[0085] In this embodiment, the calling side signaling process is as follows: Figure 4 See Figure 4 The signaling direction of the calling process in IMS is as follows:

[0086] 1. The calling user sends a call initiation request: The calling user initiates a call application, and the Invite message goes to the calling-side S-CSCF according to the normal process.

[0087] 2. Determination of the special service number of the calling party: The calling party queries the calling party side HSS whether the calling party is a mechanical voice special service number and saves it.

[0088] 3. The calling party sends a call request to the called party: The calling party sends an Invite request to the I-CSCF on the called party according to the normal process.

[0089] In this embodiment, the signaling process on the called side is as follows: Figure 5 See Figure 4 , the signaling direction of the called process in IMS is as follows:

[0090] 1. Determination of the special service number of the calling party: The S-CSCF on the called party receives the call request initiated by the calling party according to the normal process, and analyzes the Invite message to see whether the calling party is a special service number of mechanical voice. If so, it is released according to the normal process, otherwise it goes to step 2.

[0091] 2. Callee service query: The S-CSCF on the called side queries the called HSS whether the called party has activated the deep audio forgery detection service. If it is activated, proceed to 3; otherwise, release according to the normal process.

[0092] 3. Notify AAV_AS to start the service: The S-CSCF on the called side initiates an Invite message to the AAV_AS on the called side, notifying AAV_AS to prepare to protect the user's call security.

[0093] By executing Figure 4 and Figure 5 In the signaling process on the calling side and the signaling process on the called side shown, the IMS starts to establish a voice call process between the first voice communication party and the second voice communication party, and starts to execute the forged telecommunication audio detection method.

[0094] Reference Figure 6 , the counterfeit telecommunication audio detection method comprises the following steps:

[0095] S1. Determine the first voice collection period;

[0096] S2. During the first voice collection period, obtaining the first voice data of the first voice communication party;

[0097] S3 extracts at least one first statistical feature data and at least one first biometric feature data from the first voice data;

[0098] S4. Processing the first statistical feature data and the first biometric feature data to obtain a first input tensor;

[0099] S5. Input the first input tensor into the artificial intelligence model for processing to obtain the first detection result of the artificial intelligence model.

[0100] Steps S1-S5 can be specifically divided into the following stages:

[0101] 1. Call initiation phase: When the caller initiates a voice call request, the AAV_AS network element first initiates a MAR (Message Authorization Request) message to the HSS (Home Subscriber Server) through the S_CSCF to query whether the caller is a registered legal artificial intelligence customer service or mechanical voice user.

[0102] 2. Authorization query: After receiving the call request, the S_CSCF of the called party immediately initiates a MAR (Message Authorization Request) to the HSS (Home Subscriber Server) to check whether the called user allows the deep audio forgery detection service to be enabled. If the called user authorizes the service and the caller is not a legitimate mechanical audio user, the S_CSCF sends an INVITE message to the AAV_AS through the ISC interface, instructing the called party AAV_AS to start preheating preparations.

[0103] 3. Audio data collection and preprocessing: Within t seconds before the call is officially established, P_CSCF (Proxy CallSession Control Function) will synchronously forward the audio stream data packets to AAV_AS. AAV_AS receives and encrypts these data packets and temporarily stores them in the database. At the same time, the service control module of AAV_AS starts the timer to ensure that after collecting the complete t seconds of data, it sends an instruction to P_CSCF to stop forwarding the audio data packets.

[0104] 4. Audio analysis and decision-making: After the collection period, AAV_AS extracts the encrypted audio data packets from the database, decrypts the data through the decryption / preprocessing module and integrates it into a continuous audio stream. Next, the audio stream undergoes preprocessing steps such as Fourier transform to extract key features, and then passes it to the deep neural network for deep fake audio analysis and identification.

[0105] 5. Result processing and notification: After the analysis is completed, the service control module receives the results from the neural network. If the audio is determined to be a real human conversation, it is directly released without additional processing. On the contrary, if the audio is detected to be forged, it requests the MRF (Multimedia Resource Function) to play a warning message through the C1 interface to inform the called party that the other party is using forged audio technology.

[0106] 6. Backup message forwarding mechanism: During or at the end of a call, if the user has hung up and the analysis results show that the audio is forged, AAV_AS will send a message to IP_SM_GW (IP Short Message Gateway) through S_CSCF to notify the user of the warning information about the forged audio in the form of a text message to ensure that the user will not suffer financial losses without being aware of it.

[0107] In step S1, Figure 7 As shown in part (a) of FIG. 1 , t seconds before the voice call process between the first voice communication party and the second voice communication party officially starts is determined as the first voice collection period. Alternatively, Figure 7As shown in part (b) of FIG. 1 , during the period after the voice call process between the first voice communication party and the second voice communication party starts, a continuous period of t seconds is arbitrarily selected as the first voice collection period.

[0108] If you follow Figure 7 The first voice collection period is determined by the method shown in part (a). Since the voice call between the first voice communication party and the second voice communication party has not yet officially started at this time, the first voice communication party generally does not make any voice calls. The AAV_AS can send a prompt tone to the first voice communication party to prompt the first voice communication party to make a voice call through the prompt tone.

[0109] When executing step S2, the service control module in the AAV_AS may record the voice uttered by the first voice communication party during the first voice collection period, thereby obtaining the first voice data.

[0110] Since the first voice data may include private information, after executing step S2, the AAV_AS may encrypt the first voice data to ensure information security. Specifically, the encryption module in the AAV_AS may encrypt the first voice data. Figure 8 The encryption process shown in Figure 1. Figure 8 , the encryption process includes the following steps:

[0111] 1. Obtain random entropy: Before encrypting the user audio data packet (first voice data), the encryption module can extract a series of unpredictable data from the system as a source of random entropy. These data may include but are not limited to dynamically changing parameters such as the system's current timestamp (Ti), CPU occupancy (Ca), and ambient temperature (Tm), so as to increase the unpredictability of key generation.

[0112] 2. Generate a temporary key: The encryption module uses the random entropy data obtained above as input and generates a temporary and unique key through a certain algorithm or function. This key will only be used for the encryption process of this data packet, ensuring that even if the key is intercepted, it will not affect the security of other data packets due to its one-time use characteristics.

[0113] 3. Encrypted data packet: Once the temporary key is generated, the encryption module uses the key to encrypt the first voice data. The encryption process ensures that the original data is not accessed by unauthorized persons during transmission and storage.

[0114] 4. Save data: After encryption is completed, the encryption module stores the encrypted data packet together with the generated temporary key combination in the database module.

[0115] When step S3 needs to be executed, the encrypted data corresponding to the first voice data can be read from the database module, and the decryption / preprocessing module in the AAV_AS decrypts and reassembles the encrypted data to restore the first voice data. Fig. 9 , the decryption and reassembly performed by the decryption / preprocessing module includes the following process:

[0116] 1. Separation of data packets and temporary keys: The decryption module extracts encrypted data packets from the database. Each data packet contains the encrypted first voice data and the temporary key for decryption. The decryption / preprocessing module first separates the two parts of the data packet and extracts the temporary key and the encrypted data.

[0117] 2. Decrypt data packets: The decryption / preprocessing module uses the separated temporary key to decrypt the data packets one by one according to the corresponding decryption algorithm.

[0118] 3. Data reorganization: After decryption is completed, the decryption / preprocessing module reassembles the decrypted data fragments according to the predefined communication protocol, and finally restores them into a continuous time-domain audio stream (first voice data).

[0119] After the first voice data is obtained through decryption and reassembly, the first voice data may be further processed by the decryption / preprocessing module. Specifically, the decryption / preprocessing module may execute steps S3 and S4.

[0120] In step S3, the decryption / preprocessing module is as follows Fig.10 As shown, the audio authentication algorithm is used to extract feature data from the time domain audio stream (first voice data) obtained by decryption and reorganization.

[0121] Specifically, the decryption / preprocessing module can perform Fourier transform on the time domain audio stream (first voice data) to obtain frequency domain features:

[0122]

[0123] Wherein k is the frequency index, |X[k]| is the amplitude-frequency characteristic data, ∠X[k] is the phase-frequency characteristic data, x[n] is the time domain audio stream (first voice data), Re is the real part of the data, and Im is the imaginary part of the data.

[0124] Next, the decryption / preprocessing module divides the amplitude-frequency characteristic data |X[k]| into multiple equal parts according to the frequency bands and calculates the resolution of each frequency band:

[0125]

[0126] The F-radio calculated by the above formula iis the F ratio of the ith frequency band, which can be used to measure the speaker's discrimination score in each frequency band. i The larger the value is, the larger the value obtained in the ith frequency band is, which means that the frequency band contains more speaker information and the frequency band is more distinguishable. Audio authentication research has found that compared with the relatively uniform energy of mechanical sound in the whole frequency band, the information of real people in different frequency bands is not uniform.

[0127] Next, the decryption / preprocessing module performs calculations according to the following formula:

[0128]

[0129]

[0130] Δf(n)=f0(n+1)-f0(n)

[0131]

[0132] Where k is the delay and N is the total number of audio frames.

[0133] E(Δf(n)) calculated by the above formula is the average pitch change of the user. Compared with the flat pitch of fake audio, the pitch of real people often has more or less pitch fluctuations due to subtle emotional changes or habits.

[0134] Next, the decryption / preprocessing module performs calculations according to the following formula:

[0135]

[0136] Res[i]=t,iff t <τ

[0137]

[0138] X[k] is the Fourier transform amplitude-frequency data of the audio time domain data in the time window [t, t+H), f s is the sampling frequency, Res[i] indicates the i-th breathing moment, τ is the breathing frequency threshold, and M represents the total number of breaths.

[0139] The RI calculated by the above formula represents the average interval of breathing of the “speaker” (which may be a real human being or not) of the first speech data detected based on the first speech data.

[0140] The F-ratio obtained by the above execution is i The sum of the pitch variation mean E(Δf(n)) can be used as the first statistical feature data obtained by executing step S3, and the average breathing interval RI can be used as the first biometric feature data obtained by executing step S3.

[0141] After the decryption / preprocessing module executes step S3, the decryption / preprocessing module executes step S4 to convert the first statistical feature data [F ratio F-radio i and the mean pitch change E(Δf(n))] and the first biometric data (average breathing interval RI) are folded and transformed into the first input tensor, and the first input tensor is sent to the neural network analysis module in AAV_AS.

[0142] In this embodiment, the neural network analysis module runs an artificial intelligence model, which can be a CNN convolutional neural network. The structure of the CNN convolutional neural network used in this embodiment is as follows: Fig.11 As shown. The output of the CNN convolutional neural network is The data label is y and the loss function is:

[0143]

[0144] Update the network in the following ways:

[0145]

[0146] Before using the CNN convolutional neural network, the CNN convolutional neural network can be trained using a gradient update algorithm, that is, the trained CNN convolutional neural network is used in step S5.

[0147] Reference Fig.11 The neural network analysis module executes step S5, runs the CNN convolutional neural network to extract effective features from the first statistical feature data, and merges them into the first biometric feature data. Specifically, Fig.11 The dimension of the first statistical feature data in is 3*8*8, and the dimension of the first biometric feature data is 10. If the first statistical feature data is less than 64 frames, it can be padded to 64 frames by 0. The CNN convolutional neural network uses a fully connected layer to map the first input tensor, and finally obtains the classification result of the first input tensor.

[0148] Specifically, the classification result of the first input tensor can indicate whether the first voice data used to generate the first input tensor is fake telecommunication audio (that is, the first voice data is generated by a computer program) or is not fake telecommunication audio (that is, the first voice data is formed by the sound spoken by a real human being), that is, the classification result of the first input tensor can be used as the first detection result.

[0149] In this embodiment, by executing steps S1-S5, the following technical effects can be achieved:

[0150] Comprehensive detection technology: By integrating audio signal processing technology and artificial intelligence models, a more comprehensive audio analysis method is provided. It can not only extract the biological feature points and statistical features of the audio, but also identify subtle features that are difficult to detect with traditional methods, such as simulated emotional expressions and breathing pauses, through the analysis of artificial intelligence models, greatly improving the accuracy and reliability of detection;

[0151] Real-time detection and deployment: AAV_AS deployed in the IMS network can achieve real-time audio forgery detection without the need for additional hardware or software support; the direct embedding of AAV_AS provides a built-in security solution for the telecom network, which not only enhances the overall security of the network, but also enables rapid response, effectively reducing the potential losses suffered by users due to forged audio;

[0152] Data security and privacy protection: Encryption technology is used to collect and store audio data packets to ensure the privacy and security of user data. The application of this encryption technology not only protects user data from unauthorized access, but also meets the current strict requirements for data privacy protection;

[0153] Modularity and Scalability: The modular architecture design of AAV_AS provides flexibility and scalability, allowing the system to be upgraded and maintained as needed. This design can adapt to the rapidly changing technological environment and growing security needs.

[0154] Comprehensive benefits: In summary, the counterfeit telecommunications audio detection method in this embodiment has shown obvious advantages in many aspects such as technical implementation, security, real-time, user warning and system stability. It not only provides an innovative solution to address audio security challenges in telecommunications networks, but also brings higher value and better experience to users and operators through its comprehensive benefits.

[0155] In this embodiment, the AAV_AS may further perform the following steps when executing the forged telecommunication audio detection method:

[0156] S6. When the first detection result indicates that the first voice data does not belong to forged telecommunications audio, the first voice data is released;

[0157] S7. When the first detection result indicates that the first voice data is forged telecommunication audio, a warning message is generated, the warning message is sent to the second voice communication party, selection information of the second voice communication party is obtained, and according to the selection information, the first voice data is released or the voice call process is hung up.

[0158] The process of steps S6-S7 is as follows: Fig.12 See Fig.12 , the process of steps S6-S7 includes:

[0159] 1. Analyze the neural network results: The service control module receives the analysis results of the neural network module. If the result shows that the caller has a real human voice, go to step 3. If the result shows that the caller has an AI-forged audio, go to step 2.

[0160] 2. Select a reminder strategy: If the analysis result shows that the caller is an AI forged audio, determine whether the current called party has hung up the call. If the call is still on hold, AAV_AS sends a playback request to MRF to play a reminder ringback tone to remind the called party to be careful of fraud. If the call has been hung up, AAV_AS sends a message to S-CSCF and sends a reminder SMS or pop-up window to the user.

[0161] 3. Exit: AAV_AS sends a request to the database to delete the user's audio data and exits the service after successful deletion.

[0162] Reference Fig.12 If the first detection result indicates that the first voice data is not a forged telecommunications audio, then the AAV_AS executes step S6 to release the first voice data, that is, the IMS sends the first voice data to the second voice communication party, so that the second voice communication party plays the first voice data for people to listen to, so as to carry out the voice call process normally; if the first detection result indicates that the first voice data is a forged telecommunications audio, then the AAV_AS executes step S7 to not release the first voice data, and quickly sends a warning message to the second voice communication party through various methods such as voice warning and SMS notification. The second voice communication party can return a message to the AAV_AS indicating "maintain If the selection information is "maintain the voice call process", AAV_AS sends the first voice data to the second voice communication party, so that the second voice communication party plays the first voice data for people to listen to, thereby conducting the voice call process normally. If the selection information is "hang up the voice call process", AAV_AS hangs up the voice call process, thereby preventing the first voice data belonging to forged telecommunications audio from being sent to the second voice communication party, providing an extra layer of security for users, so that users can be aware of potential risks in the first time and take corresponding preventive measures.

[0163] In this embodiment, if AAV_AS releases the first voice data according to the selection information of "maintaining the voice call process" when executing step S7, then AAV_AS can back up the first voice data and the warning information; after the voice call process ends (for example, the first voice communication party or the second voice communication party hangs up), AAV_AS sends the first voice data and the warning information to the second voice communication party. Through this backup message forwarding mechanism, a forged audio warning message can be sent to the user after the call ends, and the user can receive notifications about potential risks even after the call ends.

[0164] In this embodiment, if the first detection result indicates that the first voice data is a forged telecommunications audio, the AAV_AS may also obtain user attribute information of the first voice communication party. The user attribute information indicates whether the first voice communication party is a user who legally uses mechanical voice to make calls, such as a customer service account. If the user attribute information indicates that the first voice communication party is such a special user, the AAV_AS releases the first voice data, otherwise the voice call process is terminated, thereby ensuring that special users are provided with legal voice services normally.

[0165] In this embodiment, AAV_AS can also perform the following Fig.13 The disaster recovery process shown in the figure. Fig.13 The disaster recovery process specifically includes:

[0166] 1. Abnormal restart processing: When AAV_AS restarts abnormally due to external uncontrollable factors, the system will automatically switch to the disaster recovery module to ensure that the system status after the restart can be processed in a timely and effective manner.

[0167] 2. Database query: AAV_AS queries the database to check whether there is any unprocessed data. If there is unprocessed data, it enters the business processing stage; otherwise, it directly enters the exit stage.

[0168] 3. Business processing mechanism: AAV_AS extracts a piece of unprocessed business data from the database. Performs detailed analysis and processing on the extracted data. After processing, the processed data is deleted from the database. Repeat the above steps until all unprocessed business data has been processed.

[0169] 4. System exit: When all data that has not been processed in time has been processed, AAV_AS exits the disaster recovery module and resumes normal operation mode.

[0170] By executing the disaster recovery process, it is possible to ensure that unfinished audio analysis tasks can be restored after the system restarts abnormally. The establishment of this disaster recovery mechanism ensures that the system can maintain continuity and reliability even in the face of unexpected interruptions, ensuring the stability of the service and the user experience.

[0171] In this embodiment, if according to Figure 7 If the first voice collection period is determined in the manner shown in part (b) of the present invention, the AAV_AS may further perform the following steps when executing the forged telecommunication audio detection method:

[0172] S8. When the first detection result indicates that the first voice data belongs to forged telecommunications audio, the voice call process is interrupted;

[0173] S9. After the voice call process is interrupted, the voice call process is resumed;

[0174] S10. After the voice call process is restored, determine the second voice collection period;

[0175] S11. During the second voice collection period, obtaining the second voice data of the first voice communication party;

[0176] S12 extracting at least one second statistical feature data and at least one second biometric feature data from the second voice data;

[0177] S13. Compare the second statistical feature data and the second biometric feature data with the first statistical feature data and the first biometric feature data to obtain a comparison similarity;

[0178] S14. Based on the comparison similarity, the second voice data is released or the voice call process is terminated.

[0179] The principle of steps S8-S10 is as follows Fig.14 shown.

[0180] Reference Fig.14 In step S8, the AAV_AS interrupts the voice call process between the first voice communication party and the second voice communication party for a period of time. Specifically, the AAV_AS can directly hang up the voice call process, or shield the data sent by the first voice communication party and the second voice communication party to each other, or shield the data sent by either the first voice communication party or the second voice communication party to the other party, so as to achieve the effect of interrupting the voice call process.

[0181] Reference Fig.14 In step S9, the AAV_AS performs the opposite operation to step S8, thereby resuming the voice call process between the first voice communication party and the second voice communication party.

[0182] Reference Fig.14 In step S10, t seconds after the voice call process is resumed are determined as the second voice collection period.

[0183] In this embodiment, the principles of steps S11-S12 are the same as those of steps S2-S3. That is, the second voice data is the same as the first voice data, which is voice data obtained by recording the voice emitted by the first voice communication party, but the voice collection period is different; the second statistical feature data is the same type of data as the first statistical feature data, and the second biometric feature data is the same type of data as the first biometric feature data.

[0184] In step S13, AAV_AS compares the second statistical feature data and the second biometric data with the first statistical feature data and the first biometric data to obtain a comparative similarity. Specifically, since the first statistical feature data, the first biometric data, the second statistical feature data, and the second biometric data can all be expressed in the form of vectors, AAV_AS can execute a vector similarity algorithm to calculate the similarity between the second statistical feature data and the second biometric data, as well as the similarity between the first statistical feature data and the first biometric data, and then obtain the comparative similarity by summing or multiplying the two similarities; AAV_AS can also splice the second statistical feature data and the second biometric data into one vector, splice the first statistical feature data and the first biometric data into another vector, and execute a vector similarity algorithm to calculate the similarity between the two vectors to obtain a comparative similarity.

[0185] In step S14, AAV_AS may set a contrast threshold. If the contrast similarity obtained in step S13 is greater than the contrast threshold, it may be determined that the contrast similarity is relatively large. Conversely, if the contrast similarity is less than or equal to the contrast threshold, it may be determined that the contrast similarity is relatively small.

[0186] In step S14, AAV_AS may release the second voice data when it is determined that the comparison similarity is relatively small, and terminate the voice call process when it is determined that the comparison similarity is relatively large.

[0187] In this embodiment, the principle of executing steps S8-S14 is that the second voice data is recorded in the second voice collection period after the voice call process is interrupted and resumed; if the speaker of the first voice communication party is a real human, then the interruption and resumption of the voice call process is likely to affect the emotions and other factors of the speaker of the first voice communication party, so that the second statistical feature data and the second biometric data contained in the second voice data are greatly different from the first statistical feature data and the first biometric data contained in the first voice data, which is manifested as a relatively small comparison similarity; on the contrary, if the "speaker" of the first voice communication party is a computer program, it is not easily affected by the interruption and resumption of the voice call process, so that The difference between the second statistical feature data and the second biometric data and the first statistical feature data and the first biometric data is small, which is manifested as a relatively large comparison similarity; therefore, when executing steps S8-S14, when it is determined that the comparison similarity is relatively small, it can be determined that the speaker of the first voice communication party is more likely to be a real human, that is, the second voice data is more likely not to be forged telecommunications audio, thereby releasing the second voice data to ensure the normal call; on the contrary, when it is determined that the comparison similarity is relatively large, it can be determined that the "speaker" of the first voice communication party is more likely to be a computer program, that is, the second voice data is more likely to be forged telecommunications audio, thereby hanging up the voice call process to ensure the safety of the call.

[0188] By executing steps S8-S14, forged telecommunication audio can be detected according to the reaction of the first voice communication party during the voice call process, thereby improving the reliability of detecting forged telecommunication audio.

[0189] In this embodiment, the operation of "hanging up the voice call process" taken by AAV_AS can also be replaced by "shielding the first voice data", that is, the voice call process between the first voice communication party and the second voice communication party is not hung up, but the first voice data is not sent to the second voice communication party, thereby shielding harmful information and maintaining the continuity of the voice call process.

[0190] In summary, the forged telecommunication audio detection method in this embodiment has the following advantages:

[0191] 1. Comprehensive audio forgery detection method: Combining traditional audio signal processing technology with deep learning framework can improve the accuracy of detecting deep forged audio.

[0192] 2. AAV_AS deployment in IMS network: AAV_AS is deployed directly in the IMS network, enabling audio forgery detection to be performed in real time without the need for additional hardware or software. The personalized feature of this deployment method is that it provides a built-in security solution for the telecommunications network, enhancing the overall security of the network.

[0193] 3. Secure collection and preprocessing of audio data: By adopting a secure data collection and preprocessing method, including encrypting the audio data packets and storing them in the database, the privacy and security of user data can be guaranteed, and it will not be leaked even during data transmission and storage.

[0194] 4. Deep neural network audio analysis: By using deep neural networks to analyze audio data, it is possible to identify forged audio. Specifically, it is able to identify subtle features in the audio to accurately determine whether it is forged audio.

[0195] 5. Multi-channel user warning system: When forged audio is detected, it can quickly warn users through multiple channels (such as instant voice warnings, SMS notifications, etc.), thereby providing a rapid response mechanism to reduce the potential losses suffered by users due to forged audio.

[0196] 6. Backup message forwarding mechanism: This implementation designs a backup message forwarding mechanism to send forged audio warning information to users after the call ends, which can ensure that users can receive notifications about potential risks even after the call ends.

[0197] 7. Modular architecture of AAV_AS: The architecture of AAV_AS consists of multiple key modules, each with specific functions and responsibilities, which can provide flexibility and scalability, allowing the system to be upgraded and maintained as needed.

[0198] 8. Disaster recovery and system recovery: The disaster recovery module in this embodiment can recover unfinished audio analysis tasks after the system restarts abnormally, so that the system can maintain continuity and reliability even in the face of unexpected interruptions.

[0199] A computer program for executing the counterfeit telecommunications audio detection method in this embodiment can be written and written into a computer device or storage medium. When the computer program is read out and run, the counterfeit telecommunications audio detection method in this embodiment is executed, thereby achieving the same technical effect as the counterfeit telecommunications audio detection method in the embodiment.

[0200] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in the present disclosure are only relative to the relative positional relationship of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "" and "the" used in the present disclosure are also intended to include the plural forms, unless the context clearly indicates other meanings. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by those skilled in the art. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.

[0201] It should be understood that, although the term first, second, third etc. may be adopted to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish the same type of elements from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as" etc.) provided by the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, the scope of the present invention will not be limited.

[0202] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed dedicated integrated circuit for this purpose.

[0203] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed on one or more processors in common, by hardware or a combination thereof. A computer program includes a plurality of instructions that may be executed by one or more processors.

[0204] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0205] The computer program can be applied to input data to perform the functions of the present embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0206] The above are only preferred embodiments of the present invention. The present invention is not limited to the above embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical solutions and / or implementation methods may have various modifications and changes.

Claims

1. A method for detecting counterfeit telecommunication audio, characterized in that: The forged telecommunication audio detection method comprises: determining a first speech collection period; During the first voice collection period, obtaining first voice data of a first voice communication party; extracting at least one first statistical feature data and at least one first biometric feature data from the first voice data; Processing the first statistical feature data and the first biometric feature data to obtain a first input tensor; The first input tensor is input into an artificial intelligence model for processing to obtain a first detection result of the artificial intelligence model; the first detection result indicates whether the first voice data belongs to forged telecommunication audio or not.

2. The method for detecting counterfeit telecommunication audio according to claim 1, characterized in that: The step of determining the first speech collection period comprises: Obtaining a call initiation request; the call initiation request is used to request to establish a voice call process between a first voice communication party and a second voice communication party; In response to the call initiation request, a time period before the voice call process is established is determined as the first voice collection period.

3. The method for detecting counterfeit telecommunication audio according to claim 1, characterized in that: The step of extracting at least one first statistical feature data and at least one first biometric feature data from the first voice data comprises: Acquire frequency domain features of the first speech data; According to the frequency domain characteristics, obtaining the F ratio of each frequency band; Obtaining a mean value of pitch variation of the first speech data; Acquire the average breathing interval of the first voice data; The F ratio and the pitch variation mean are used as the first statistical feature data, and the average breathing interval is used as the first biometric feature data.

4. The method for detecting counterfeit telecommunication audio according to claim 3, characterized in that: The step of processing the first statistical feature data and the first biometric feature data to obtain a first input tensor includes: The first statistical feature data and the first biometric feature data are folded to obtain the first input tensor.

5. The method for detecting counterfeit telecommunication audio according to claim 1, characterized in that: The counterfeit telecommunication audio detection method further comprises: When the first detection result indicates that the first voice data is not forged telecommunication audio, releasing the first voice data; When the first detection result indicates that the first voice data is forged telecommunication audio, a warning message is generated, the warning message is sent to the second voice communication party, selection information of the second voice communication party is obtained, and according to the selection information, the first voice data is released or the voice call process is hung up; or When the first detection result indicates that the first voice data is forged telecommunication audio, user attribute information of the first voice communication party is obtained, and the first voice data is released or the voice call process is hung up according to the user attribute information.

6. The method for detecting counterfeit telecommunication audio according to claim 5, characterized in that: The counterfeit telecommunication audio detection method further comprises: When the first voice data is released according to the selection information, backing up the first voice data and the warning information; After the voice call process ends, the first voice data and the warning information are sent to the second voice communication party.

7. The method for detecting counterfeit telecommunication audio according to claim 1, characterized in that: The step of determining the first speech collection period comprises: Obtaining a call initiation request; the call initiation request is used to request to establish a voice call process between a first voice communication party and a second voice communication party; In response to the call initiation request, establishing the voice call process; A time period during which the voice call is in progress is determined as the first voice collection period.

8. The method for detecting counterfeit telecommunication audio according to claim 7, characterized in that: The counterfeit telecommunication audio detection method further comprises: When the first detection result indicates that the first voice data is forged telecommunication audio, interrupting the voice call process; After interrupting the voice call process, resuming the voice call process; Determining a second voice collection period during a time period after the voice call process is resumed; During the second voice collection period, obtaining second voice data of the first voice communication party; extracting at least one second statistical feature data and at least one second biometric feature data from the second voice data; Comparing the second statistical feature data and the second biometric feature data with the first statistical feature data and the first biometric feature data to obtain a comparison similarity; According to the comparison similarity, the second voice data is released or the voice call process is terminated.

9. A computer device, characterized in that: The device comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load at least one program to execute the counterfeit telecommunication audio detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to perform the counterfeit telecommunication audio detection method according to any one of claims 1 to 8 when executed by the processor.

Images