Audio detection method and device, electronic equipment and storage medium

Abstract

The invention relates to an audio detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring target audio data in a current call process; the target audio data is used for representing conversation content in the current conversation process; obtaining target text data corresponding to the target audio data; the target text data is text data obtained by converting the target audio data; calling a dialogue detection model, and detecting the target text data to obtain a detection result of the target audio data; the dialogue detection model is used for detecting whether the dialogue content has a fraud risk, and the detection result is used for representing whether the dialogue content represented by the target audio data has the fraud risk. The method does not depend on the preset blacklist, can effectively intercept the call with the fraud risk, has high timeliness and application range, can better protect the property and safety of the user, and improves the use experience of the user.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an audio detection method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI technology has the ability to generate highly realistic audio content. While AI technology brings convenience to users' lives, it also brings a series of security risks, such as misleading information, false news, and fraud. Take fraud as an example. In recent years, cybercriminals have used AI deep fake audio to call users in order to defraud them, posing a serious threat to their property and safety.

[0003] In the related art, a blacklist of fraudulent numbers is currently preset, and the blacklist of fraudulent numbers is compared with the current call number to determine whether the current call number is a fraudulent number, and then determine whether the current call has a fraud risk. However, in actual fraud scenarios, only the call numbers in the blacklist can be blocked, and the call numbers that are not added to the blacklist cannot be effectively blocked, and the timeliness is poor. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an audio detection method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an audio detection method, including: Acquire target audio data during a current call; the target audio data is used to represent the conversation content during the current call; Acquire target text data corresponding to the target audio data; the target text data is text data converted from the target audio data; A conversation detection model is called to detect the target text data to obtain a detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

[0006] In some embodiments, obtaining target audio data during the current call includes: A preset sliding window is used to intercept the audio data in the current call process to obtain the target audio data; the preset sliding window is used to intercept the audio data of a preset duration, and the duration of the target audio data is the preset duration.

[0007] In some embodiments, the obtaining target text data corresponding to the target audio data includes: A preset conversion model is called to convert the target audio data into the target text data; the preset conversion model is used to convert data in audio form into data in text form.

[0008] In some embodiments, when the dialogue detection model is a large language model, calling the dialogue detection model to detect the target text data to obtain the detection result of the target audio data includes: The target text data and preset instructions are input into the large language model, and the detection result corresponding to the preset instruction is output; the preset instruction is used to instruct the large language model on the detection content of the target text data.

[0009] In some embodiments, the training process of the dialogue detection model includes: Acquire training data, the training data comprising first sample audio data, a first label corresponding to the first sample audio data, second sample audio data, and a second label corresponding to the second sample audio data, the first sample audio data being audio data without fraud risk, the first label being used to characterize that the conversation content represented by the first sample audio data does not have fraud risk, the second sample audio data being audio data with fraud risk, and the second label being used to characterize that the conversation content represented by the second sample audio data has fraud risk; Based on the training data, an initial dialogue detection model to be trained is trained to obtain the dialogue detection model.

[0010] In some embodiments, the training of the initial dialogue detection model to be trained based on the training data to obtain the dialogue detection model includes: Convert the first sample audio data into first sample text data, and convert the second sample audio data into second sample text data; Inputting the first sample text data and the second sample text data into the initial dialogue detection model respectively, and outputting a first prediction detection result of the first sample text data and a second prediction detection result of the second sample text data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial dialogue detection model are adjusted to obtain the dialogue detection model.

[0011] In some embodiments, the method further comprises: In response to the detection result indicating that the conversation content represented by the target audio data has a risk of fraud, a prompt message is issued; the prompt message is used to prompt the target object that there is a risk of fraud in the current call.

[0012] According to a second aspect of an embodiment of the present disclosure, there is provided an audio detection device, including: A first acquisition module is configured to acquire target audio data during a current call; the target audio data is used to represent the conversation content during the current call; A second acquisition module is configured to acquire target text data corresponding to the target audio data; the target text data is text data converted from the target audio data; The processing module is configured to call a conversation detection model to detect the target text data and obtain a detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

[0013] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to execute the audio detection method as described in the first aspect of the present disclosure.

[0014] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the audio detection method as described in the first aspect of the present disclosure.

[0015] The above method of the present disclosure has the following beneficial effects: The method provided by the embodiment of the present disclosure can analyze the conversation content during the current call in real time, so as to determine whether there is a risk of fraud during the call. It does not rely on a preset blacklist and can effectively intercept calls with a risk of fraud. It has high timeliness and scope of application, so as to better protect the property and safety of users and improve the user experience.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] Figure 1 is a flow chart of an audio detection method according to an exemplary embodiment.

[0019] Figure 2 is a schematic diagram showing a training process according to an exemplary embodiment.

[0020] Figure 3 It is a schematic diagram showing a prompt message according to an exemplary embodiment.

[0021] Figure 4 is a flow chart of an audio detection method according to an exemplary embodiment.

[0022] Figure 5 is a flow chart of an audio detection method according to an exemplary embodiment.

[0023] Figure 6 is a flow chart of an audio detection method according to an exemplary embodiment.

[0024] Figure 7 is a flow chart of an audio detection method according to an exemplary embodiment.

[0025] Figure 8 is a schematic diagram showing an audio detection method according to an exemplary embodiment.

[0026] Fig. 9 It is a block diagram of an audio detection device according to an exemplary embodiment.

[0027] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] With the rapid development of artificial intelligence technology, AI technology has the ability to generate highly realistic audio content. While AI technology brings convenience to users' lives, it also brings a series of security risks, such as misleading information, false news, and fraud. Take fraud as an example. In recent years, cybercriminals have used AI deep fake audio to call users in order to defraud them, posing a serious threat to their property and safety.

[0030] In order to solve this problem, a blacklist of fraudulent numbers is currently preset, and the blacklist of fraudulent numbers is compared with the current call number to determine whether the current call number is a fraudulent number, and then determine whether the current call has a fraud risk. In this way, when the current call number is a fraudulent number, the electronic device can intercept the call, thereby effectively protecting the user's property and safety.

[0031] However, in actual fraud scenarios, if the current call number is added to the blacklist of fraudulent numbers, criminals can change the call number and call the user based on the changed call number. In this way, when the criminal's call number changes all the time, comparing the call number based on the preset blacklist of fraudulent numbers will not effectively intercept the call, and criminals will still commit fraud against users, thus posing a serious threat to the user's property and safety, and affecting the user's experience.

[0032] In order to solve the above problems, the present disclosure provides an audio detection method, which can obtain target audio data during the current call; the target audio data is used to characterize the conversation content during the current call; the target audio data is converted into target text data; the conversation detection model is called to detect the target text data to obtain the detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data. The method provided by the present disclosure can analyze the conversation content during the current call process in real time, so as to determine whether there is a fraud risk in the conversation content during the call process, and does not rely on a preset blacklist, and can also effectively intercept calls with fraud risks, with high timeliness and scope of application, so as to better protect the property and safety of users and improve the user experience.

[0033] The method provided by the embodiments of the present disclosure can be applied to a variety of scenarios, for example, to scenarios involving calls, such as video call scenarios and voice call scenarios.

[0034] Taking the voice call scenario as an example, when the user is on a voice call, the terminal device obtains the audio data of the call in real time, then converts the audio data into text data, and then calls the conversation detection model to detect the text data to determine whether there is a fraud risk in the current conversation content. If it is determined that there is a fraud risk, it means that there is a high possibility of fraud risk in the current voice call. If it is determined that there is no fraud risk, it means that there is no fraud risk in the current voice call or the possibility of fraud risk is very low.

[0035] The audio detection method provided in the embodiment of the present disclosure is performed by an electronic device, which may be a terminal device, such as a mobile phone, a tablet computer, a notebook, an intelligent robot, an intelligent wearable device, etc. The electronic device may also be a cloud device, such as a cloud server, etc.

[0036] Figure 1 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by an electronic device, see Figure 1 , the method comprises the following steps.

[0037] Step S101, obtaining target audio data during the current call; the target audio data is used to represent the conversation content during the current call.

[0038] The current call may be a voice call, such as a telephone voice call or an Internet voice call, or a video call. It should be noted that when the current call is a video call, this step obtains audio data in the video call, and does not limit whether to obtain image data and / or video data during the video call.

[0039] In some embodiments, the target audio data is in audio form, for example, the target audio data can be an audio stream. During a call, the terminal device can collect the target audio data in the audio form of the current call in real time in the background. Optionally, the terminal device can collect the target audio data in the current call in a variety of ways, for example, by collecting the target audio data through an application with audio capture capability, which is not limited in the embodiments of the present disclosure.

[0040] In some embodiments, the conversation content may include the conversation content of two or more parties, for example, the conversation content includes the content stated by target object 1 and target object 2 during the conversation. In addition, the conversation content may be distinguished item by item according to different speakers, for example, Speaker 1: Hello, who are you? Speaker 2: Hello, I am from the hospital, are you a relative of someone? Speaker 1: Yes, I am, what can I do for you? Speaker 2: Hello, he needs to have an operation and you need to pay a sum of money.

[0041] Step S102, obtaining target text data corresponding to the target audio data; the target text data is text data converted from the target audio data.

[0042] The conversation detection model provided in the embodiment of the present disclosure is used to identify and detect text data, so the target audio data is converted into target text data, and the target text data is applied to the subsequent detection process. Among them, the target text data includes the text content corresponding to the conversation content during the current call.

[0043] It should be noted that the target audio data and the target text data correspond to the same conversation content. The difference between the two is that the target audio data is the conversation content in audio form, while the target text data is the conversation content in text form.

[0044] The embodiment of the present disclosure does not limit the conversion method between audio data and text data, and the conversion method between the two can be set and selected based on actual needs.

[0045] Step S103, calling the conversation detection model to detect the target text data and obtain the detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

[0046] The detection result may indicate that the conversation content represented by the target audio data has a fraud risk, or that the conversation content represented by the target audio data does not have a fraud risk. When the conversation content represented by the target audio data has a fraud risk, it indicates that the possibility of the current call having a fraud risk is high.

[0047] It should be noted that the dialogue detection model can be deployed on a terminal device or on a cloud device. Accordingly, when the dialogue detection model is deployed on a terminal device, steps S101-S103 are executed by the terminal device, and when the dialogue detection model is deployed on a cloud device, steps S101-S103 are executed by the cloud device.

[0048] The method provided by the embodiment of the present disclosure can analyze the conversation content during the current call in real time, so as to determine whether there is a risk of fraud during the call. It does not rely on a preset blacklist and can effectively intercept calls with a risk of fraud. It has high timeliness and scope of application, so as to better protect the property and safety of users and improve the user experience.

[0049] In some embodiments, calling the dialog detection model to detect the target text data may include: inputting the target text data into the dialog detection model to obtain a detection result of the target text data.

[0050] In some embodiments, the conversation detection model may be a pre-trained detection model. Figure 2 The schematic diagram of the training process shown is used to discuss the training process of the dialogue detection model.

[0051] Acquire training data, and train the initial conversation detection model to be trained based on the training data to obtain a conversation detection model. The training data includes first sample audio data, a first label corresponding to the first sample audio data, second sample audio data, and a second label corresponding to the second sample audio data, the first sample audio data is audio data without fraud risk, the first label is used to characterize that the conversation content represented by the first sample audio data does not have fraud risk, the second sample audio data is audio data with fraud risk, and the second label is used to characterize that the conversation content represented by the second sample audio data has fraud risk.

[0052] Optionally, the first sample audio data may include multiple types of audio data, for example, the first sample audio data may be audio data of small talk between multiple objects, audio data of multiple objects in a dialogue challenge, audio data generated by AI without fraud risk, etc. Similarly, optionally, the second sample audio data may also include multiple types of audio data, for example, audio data with fraud risk generated by AI based on case descriptions, or audio data included in actual fraud cases, etc. In addition, it should be noted that the first sample audio data and the second sample audio data may include other irrelevant data, such as data for jumping to a certain application or dirty data. Therefore, in some embodiments, the sample audio data may be cleaned to remove audio data in the audio data that is not related to fraud, thereby obtaining clean audio data. In this way, the detection accuracy of the trained dialogue detection model can be improved.

[0053] In some embodiments, the first label corresponding to the first sample audio data and the second label corresponding to the second sample audio data can be manually marked by the user, which can improve the accuracy of the training data and further improve the accuracy of the trained dialogue detection model.

[0054] In some embodiments, corresponding to the above steps S101 to S103, the first sample audio data and the second sample audio data can be further processed to obtain sample text data corresponding to the above sample audio data, and the initial dialogue detection model can be trained based on the sample text data.

[0055] In some embodiments, based on the training data, an initial conversation detection model to be trained is trained to obtain the conversation detection model, including: converting the first sample audio data into the first sample text data, and converting the second sample audio data into the second sample text data; inputting the first sample text data and the second sample text data into the initial conversation detection model respectively, and outputting a first predicted detection result of the first sample text data and a second predicted detection result of the second sample text data; based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, adjusting the model parameters of the initial conversation detection model to obtain the conversation detection model.

[0056] The method of converting the first sample audio data into the first sample text data and the method of converting the second sample audio data into the second sample text data may refer to the method of converting the target audio data into the target text data in the above step S102.

[0057] It should be noted that the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label can indicate whether the output of the current initial dialogue detection model is accurate, so that the model parameters of the initial dialogue detection model can be adjusted based on the difference between the predicted detection result and the label. For example, when the first predicted detection result and the first label are different, and / or the second predicted detection result and the second label are different, it can be determined that the model parameters of the initial dialogue detection model are not appropriate, and therefore, the model parameters of the initial dialogue detection model need to be adjusted until the precision and recall of the trained initial dialogue detection model meet the requirements, and the trained initial dialogue detection model is used as the dialogue detection model.

[0058] In some embodiments, test data is also provided in the embodiments of the present disclosure. The test data may have the same composition as the training data, and both include audio data without fraud risk and audio data with fraud risk. In some embodiments, after the dialogue detection model is obtained, the dialogue detection model may be tested with test data to determine whether the dialogue detection model is qualified. The specific test process is not repeated here. Optionally, the accuracy, recall rate, F value (F value is used to verify the overall significance of the model) and other parameters of the dialogue detection model may be determined based on the test data. When at least one of the above parameters meets the preset conditions, it can be determined that the dialogue detection model meets the requirements.

[0059] In addition, it should be noted that the embodiments of the present disclosure do not limit the type of dialogue detection model. Optionally, the dialogue detection model can be a large language model, which is a deep learning model trained using a large amount of text data. The large language model can understand and generate human language.

[0060] The following describes the process of calling the dialogue detection model to detect the target text data and obtaining the detection results of the target audio data when the dialogue detection model is a large language model.

[0061] The target text data and the preset instructions are input into the large language model, and the detection result corresponding to the preset instructions is output. The preset instructions are used to indicate the detection content of the target text data by the large language model.

[0062] When the conversation detection model is a large language model, the reason for inputting preset instructions into the large language model is that the large language model is a general model that can determine multiple detection contents of the data and can output multiple detection results of the data. Therefore, in order to ensure that the large language model only detects whether the conversation content represented by the target audio data has a fraud risk, the target text data and the preset instructions can be input into the large language model at the same time to output a detection result for characterizing whether the conversation content represented by the target audio data has a fraud risk. The preset instructions can be used to instruct the large language model to detect whether the target text data has a fraud risk, such as "detect whether this set of text data has a fraud risk."

[0063] Correspondingly, when the dialogue detection model is a large language model, preset instructions can be added on the basis of the training process of the above dialogue detection model to obtain the large language model.

[0064] In addition, it should be noted that the initial dialogue detection model can be an existing detection model, such as an existing large language model, or an initial framework of a detection model. Accordingly, during the training process, the parameters of the initial dialogue detection model can be adjusted based on the training data to obtain a dialogue detection model.

[0065] In the disclosed embodiment, the training data of the conversation detection model includes audio data with fraud risks and audio data without fraud risks. Thus, the conversation detection model trained with the above training data has a high accuracy rate, thereby being able to accurately detect whether the conversation content represented by the target audio data has a fraud risk.

[0066] In some embodiments, for the terminal device, after obtaining the detection result, it is necessary to feed back the detection result to the user. In response to the detection result indicating that the conversation content represented by the target audio data has a risk of fraud, the terminal device issues a prompt message, and the prompt message is used to remind the target object that there may be a risk of fraud in the current call. Optionally, the prompt message can be a text message, such as a text message displayed on a display screen saying "The current call may have a risk of fraud, please hang up in time", or a voice message, such as a voice message saying "The current call may have a risk of fraud, please hang up in time". In one example, Figure 3The schematic diagram of a prompt information shown in FIG. 1 is a schematic diagram of a prompt information, where the prompt information can be displayed on the display screen of the terminal device in the form of a pop-up window, and at the same time, an operation button can be displayed in the pop-up window, such as Figure 3 The "Ignore" and "Hang Up" operation buttons are shown for the user to choose. For the user, the prompt information sent by the terminal device is only used to remind the user that there may be fraud risks in the current call. When the user receives the prompt information, the user can perform corresponding operations based on the actual situation.

[0067] In some embodiments, the terminal device responds to the detection result indicating that the conversation content represented by the target audio data does not have a fraud risk, and does not issue a prompt message, and continues to maintain the call. Optionally, the call can be maintained until the call is hung up, such as the user manually hangs up the call, or the call is automatically hung up due to external factors.

[0068] In some embodiments, there are limitations on the size of data that can be processed by the conversation detection model. For example, when the conversation detection model is deployed on a cloud device, the conversation detection model can continuously receive target text data and perform detection on the target text data. However, when the conversation detection model is deployed on a terminal device, it is limited by the hardware capabilities of the terminal device, such as the computing power of the terminal device and / or the storage space of the terminal. The conversation detection model cannot receive the conversation content involved in the entire call process.

[0069] Therefore, in some embodiments, acquiring the target audio data may include acquiring all audio data during the current call, or acquiring part of the audio data during the current call.

[0070] In some embodiments, obtaining part of the audio data during the current call includes: using a preset sliding window to intercept the audio data during the current call to obtain target audio data. The preset sliding window is used to intercept audio data of a preset duration, and the duration of the target audio data is the preset duration. That is, the preset sliding window is used to intercept the audio data of the preset duration during the current call to obtain the target audio data.

[0071] Optionally, a preset sliding window may be used to intercept audio data of a preset duration closest to the current time during the current call, and the audio data of the preset duration may be determined as the target audio data. Moreover, the size of the preset duration may be set and selected based on actual needs.

[0072] In some embodiments, the process of acquiring the target text data is described below.

[0073] In some embodiments, the target audio data can be converted into target text data by using a preset conversion model, wherein the preset conversion model is used to convert audio data in audio form into audio data in text form.

[0074] It should be noted that the conversion of the target audio data into the target text data by the preset conversion model can be performed by the terminal device or by the cloud device. When the preset conversion model is deployed on the terminal device, it is performed by the terminal device; when the preset conversion model is deployed on the cloud device, it is performed by the cloud device.

[0075] In some embodiments, when the preset conversion model is deployed on a cloud device, the cloud device can obtain the target audio data sent by the terminal device, and use the preset conversion model to convert the target audio data between audio form and text form to obtain target text data.

[0076] Optionally, the data format and data parameters of the target audio data may correspond to the data format and data parameters of the data that can be processed by the preset conversion model. For example, when the data format of the data that can be processed by the preset conversion model is the Pulse Code Modulation (PCM) format, and the bit rate of the data that can be processed is the preset bit rate, and the number of channels of the data that can be processed is the preset number of channels, then correspondingly, the target audio data may be audio data in the PCM format with the preset bit rate and the preset number of channels. Alternatively, the data format and data parameters of the target audio data may not correspond to the data format and data parameters of the data that can be processed by the preset conversion model. Before the preset conversion model processes the target audio data, the target audio data is firstly subjected to initial data processing so that the preset conversion model can process the target audio data.

[0077] In some embodiments, the preset conversion model is an Automatic Speech Recognition (ASR) model, and the ASR model can process a real-time audio stream and convert it into text. The specific process of the ASR model processing the target audio data is not described here.

[0078] The disclosed embodiment provides a specific method for converting target audio data into target text data. By converting audio data from audio form to text form, it can ensure that the target text data is accurately identified and detected by the conversation detection model, thereby obtaining accurate detection results. When the detection result shows that the conversation content represented by the target audio data has a risk of fraud, the user can be promptly prompted, thereby protecting the user's property and safety.

[0079] It should be noted that the deployment locations of the dialogue detection model and the preset conversion model can be selected according to the actual situation and actual needs. For example, when deployed on a terminal device, there is no need for network computing, and the protection effect of privacy data is good and the cost is low; while when deployed on a cloud device, the hardware requirements for the terminal device are low, and the computing pressure and memory usage of the terminal device are small. In some embodiments, the dialogue detection model can be deployed on a cloud device, or it can be deployed on a terminal device. Similarly, the preset conversion model can be deployed on a cloud device, or it can be deployed on a terminal. That is, there are the following four ways to deploy the dialogue detection model and the preset conversion model.

[0080] In the first method, the dialogue detection model is deployed on the cloud device, and the preset conversion model is deployed on the cloud device.

[0081] In the second method, the dialogue detection model is deployed on the terminal device, and the preset conversion model is deployed on the terminal device.

[0082] That is, in the first method and the second method mentioned above, the dialogue detection model and the preset conversion model are deployed on the same device.

[0083] In the third method, the dialogue detection model is deployed on the cloud device, and the preset conversion model is deployed on the terminal device.

[0084] In the fourth method, the dialogue detection model is deployed on the terminal device, and the preset conversion model is deployed on the cloud device.

[0085] That is, in the third and fourth methods mentioned above, the dialogue detection model and the preset conversion model are deployed on different devices, and the two are cross-deployed.

[0086] In order to make this solution clearer, the audio detection process is explained below by taking the deployment method of the dialogue detection model and the preset conversion model on the same device and the deployment method of the dialogue detection model and the preset conversion model on different devices as examples.

[0087] In an example, the following describes the audio detection process when a conversation detection model is deployed on a cloud device and a preset conversion model is deployed on a cloud device.

[0088] Figure 4 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by a terminal device and a cloud device, see Figure 4 , the method comprises the following steps.

[0089] Step S401: The terminal device obtains target audio data during the current call.

[0090] Step S402: The terminal device sends target audio data to the cloud device.

[0091] In step S403, the cloud device receives the target audio data and calls a preset conversion model to convert the target audio data into target text data.

[0092] In step S404, the cloud device calls the dialogue detection model to detect the target text data and obtain the detection result of the target audio data.

[0093] Step S405: The cloud device sends the detection result to the terminal device.

[0094] Step S406: In response to the detection result indicating that the conversation content represented by the target audio data has a fraud risk, the terminal device issues a prompt message.

[0095] Step S407, in response to the detection result indicating that the conversation content represented by the target audio data does not have a fraud risk, the terminal device does not issue a prompt message and continues the call.

[0096] In an example, the following describes an audio detection process when a conversation detection model is deployed on a terminal device and a preset conversion model is deployed on the terminal device.

[0097] Figure 5 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by a terminal device, see Figure 5 , the method comprises the following steps.

[0098] Step S501: The terminal device uses a preset sliding window to intercept the audio data in the current call process to obtain target audio data.

[0099] Step S502: The terminal device calls a preset conversion model to convert the target audio data into target text data.

[0100] Step S503: The terminal device calls the dialogue detection model to detect the target text data and obtain the detection result of the target audio data.

[0101] Step S504: In response to the detection result indicating that the conversation content represented by the target audio data has a fraud risk, the terminal device issues a prompt message.

[0102] Step S505, in response to the detection result indicating that the conversation content represented by the target audio data does not have a fraud risk, the terminal device does not issue a prompt message and continues the call.

[0103] In an example, the following describes the audio detection process when the conversation detection model is deployed on a cloud device and the preset conversion model is deployed on a terminal device.

[0104] Figure 6 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by a terminal device and a cloud device, see Figure 6 , the method comprises the following steps.

[0105] Step S601: The terminal device uses a preset sliding window to intercept the audio data in the current call process to obtain target audio data.

[0106] Step S602: The terminal device calls a preset conversion model to convert the target audio data into target text data.

[0107] Step S603: the terminal device sends the target text data to the cloud device.

[0108] In step S604, the cloud device receives the target text data and calls the dialogue detection model to detect the target text data to obtain the detection result of the target audio data.

[0109] Step S605: The cloud device sends the detection result to the terminal device.

[0110] Step S606: In response to the detection result indicating that the conversation content represented by the target audio data has a fraud risk, the terminal device issues a prompt message.

[0111] Step S607, in response to the detection result indicating that the conversation content represented by the target audio data does not have a fraud risk, the terminal device does not issue a prompt message and continues the call.

[0112] In an example, the following describes the audio detection process when the conversation detection model is deployed on a terminal device and the preset conversion model is deployed on a cloud device.

[0113] Figure 7 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by a terminal device and a cloud device, see Figure 7 , the method comprises the following steps.

[0114] Step S701: The terminal device obtains target audio data during the current call.

[0115] Step S702: The terminal device sends target audio data to the cloud device.

[0116] In step S703, the cloud device receives the target audio data and calls a preset conversion model to convert the target audio data into target text data.

[0117] Step S704: the cloud device sends the target text data to the terminal device.

[0118] Step S705: The terminal device receives the target text data, and calls the dialogue detection model to detect the target text data to obtain the detection result of the target audio data.

[0119] Step S706: In response to the detection result indicating that the conversation content represented by the target audio data has a fraud risk, the terminal device issues a prompt message.

[0120] Step S707, in response to the detection result indicating that the conversation content represented by the target audio data does not have a fraud risk, the terminal device does not issue a prompt message and continues the call.

[0121] In some embodiments, Figure 8 A schematic diagram of an audio detection method is given. During a current call, the electronic device can obtain the voice stream of the call (i.e., target audio data) in real time in the background, and then use the ASR model to convert the target audio data in audio form into target text data. Then, the target text data and preset instructions are input into the large language model that has been trained offline to obtain the detection result of the target audio data. Finally, based on the detection result of the target audio data, the operation of the current call is determined, such as hanging up the call or keeping the call. In addition, the offline training process of the large language model can refer to Figure 2 The process shown, for example, may include operations such as obtaining training data, processing training data (performing data cleaning operations on training data), fine-tuning the model (adjusting parameters of initial training data based on training data), etc., which will not be repeated here.

[0122] It should be noted that the audio detection method shown in the above embodiments can be a function set in the electronic device, and the user can choose to turn on or off the function based on the needs; or, the function can be turned on by default. In addition, the above audio detection method can be run in the background of the electronic device throughout the process, so that audio detection can be achieved without the user's perception.

[0123] Fig. 9 is a block diagram of an audio detection device according to an exemplary embodiment, which is configured in an electronic device, see Fig. 9 , the device comprises: The first acquisition module 901 is configured to acquire target audio data during the current call; the target audio data is used to represent the conversation content during the current call; The second acquisition module 902 is configured to acquire target text data corresponding to the target audio data; the target text data is text data converted from the target audio data; Processing module 903 is configured to call a conversation detection model to detect the target text data and obtain a detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

[0124] In some embodiments, the first acquisition module 901 is configured to: A preset sliding window is used to intercept the audio data in the current call process to obtain target audio data; the preset sliding window is used to intercept audio data of a preset duration, and the duration of the target audio data is the preset duration.

[0125] In some embodiments, the second acquisition module 902 is configured to: The preset conversion model is called to convert the target audio data into the target text data; the preset conversion model is used to convert the data in the audio form into the data in the text form.

[0126] In some embodiments, when the dialog detection model is a large language model, the processing module 903 is configured to: The target text data and preset instructions are input into the large language model, and the detection results corresponding to the preset instructions are output; the preset instructions are used to instruct the large language model on the detection content of the target text data.

[0127] In some embodiments, the processing module 903 is configured to: Acquire training data, the training data including first sample audio data, a first label corresponding to the first sample audio data, second sample audio data, and a second label corresponding to the second sample audio data, the first sample audio data is audio data without fraud risk, the first label is used to characterize that the conversation content represented by the first sample audio data does not have fraud risk, the second sample audio data is audio data with fraud risk, and the second label is used to characterize that the conversation content represented by the second sample audio data has fraud risk; Based on the training data, an initial dialogue detection model to be trained is trained to obtain a dialogue detection model.

[0128] In some embodiments, the processing module 903 is configured to: Convert the first sample audio data into first sample text data, and convert the second sample audio data into second sample text data; Inputting the first sample text data and the second sample text data into the initial dialogue detection model respectively, and outputting a first prediction detection result of the first sample text data and a second prediction detection result of the second sample text data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial dialogue detection model are adjusted to obtain the dialogue detection model.

[0129] In some embodiments, the processing module 903 is configured to: In response to the detection result indicating that the conversation content represented by the target audio data has a risk of fraud, a prompt message is issued; the prompt message is used to prompt the target object that there is a risk of fraud in the current call.

[0130] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0131] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the audio detection method in the above embodiment.

[0132] Fig.10 is a block diagram of an electronic device 1000 according to an exemplary embodiment.

[0133] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0134] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0135] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0136] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1000.

[0137] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0138] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0139] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0140] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0141] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0142] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0143] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0144] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the audio detection method provided by an exemplary embodiment of the present disclosure.

[0145] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0146] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio detection method, characterized in that: include: Get the target audio data during the current call; The target audio data is used to represent the conversation content during the current call; Acquire target text data corresponding to the target audio data; the target text data is text data converted from the target audio data; A conversation detection model is called to detect the target text data to obtain a detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

2. The audio detection method according to claim 1, characterized in that: The step of obtaining target audio data during the current call includes: A preset sliding window is used to intercept the audio data in the current call process to obtain the target audio data; the preset sliding window is used to intercept the audio data of a preset duration, and the duration of the target audio data is the preset duration.

3. The audio detection method according to claim 1, characterized in that: The step of obtaining target text data corresponding to the target audio data includes: A preset conversion model is called to convert the target audio data into the target text data; the preset conversion model is used to convert data in audio form into data in text form.

4. The audio detection method according to claim 1, characterized in that: When the dialogue detection model is a large language model, calling the dialogue detection model to detect the target text data to obtain the detection result of the target audio data includes: The target text data and preset instructions are input into the large language model, and the detection result corresponding to the preset instruction is output; the preset instruction is used to instruct the large language model on the detection content of the target text data.

5. The audio detection method according to claim 1, characterized in that: The training process of the dialogue detection model includes: Acquire training data, the training data comprising first sample audio data, a first label corresponding to the first sample audio data, second sample audio data, and a second label corresponding to the second sample audio data, the first sample audio data being audio data without fraud risk, the first label being used to characterize that the conversation content represented by the first sample audio data does not have fraud risk, the second sample audio data being audio data with fraud risk, and the second label being used to characterize that the conversation content represented by the second sample audio data has fraud risk; Based on the training data, an initial dialogue detection model to be trained is trained to obtain the dialogue detection model.

6. The audio detection method according to claim 5, characterized in that: The step of training the initial dialogue detection model to be trained based on the training data to obtain the dialogue detection model comprises: Convert the first sample audio data into first sample text data, and convert the second sample audio data into second sample text data; Inputting the first sample text data and the second sample text data into the initial dialogue detection model respectively, and outputting a first prediction detection result of the first sample text data and a second prediction detection result of the second sample text data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial dialogue detection model are adjusted to obtain the dialogue detection model.

7. The audio detection method according to claim 1, characterized in that: The method further comprises: In response to the detection result indicating that the conversation content represented by the target audio data has a risk of fraud, a prompt message is issued; the prompt message is used to prompt the target object that there is a risk of fraud in the current call.

8. An audio detection device, characterized in that: include: A first acquisition module is configured to acquire target audio data during a current call; The target audio data is used to represent the conversation content during the current call; A second acquisition module is configured to acquire target text data corresponding to the target audio data; The target text data is text data converted from the target audio data; The processing module is configured to call a conversation detection model to detect the target text data and obtain a detection result of the target audio data; the conversation detection model is used to detect whether there is a fraud risk in the conversation content, and the detection result is used to characterize whether there is a fraud risk in the conversation content represented by the target audio data.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the audio detection method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the audio detection method according to any one of claims 1 to 7.

Images