System and method for determining call-related risk based on voiceprint feature recognition

Abstract

The present invention relates to the field of voiceprint recognition, and discloses a system and method for realizing call-related risk judgment based on voiceprint feature recognition, including the following steps: Step 1: record and monitor the call audio, and upload it to a database; Step 2: separate the channels of the dual-channel audio and extract the user-side call; Step 3: remove the silent audio segments through intelligent de-muting means, and re-merge other valid audios into new audio; Step 4: extract the voiceprint features of the user's call audio on a trained model through an artificial intelligence deep learning algorithm; Step 5: upload the call audio to a database. The invention can improve the efficiency of the voiceprint feature recognition process, timely monitor and discover risks in the call process, can perform secondary manual verification on questionable data, improve the accuracy of judgment, and can continuously compare the voiceprint data of the same user to avoid misjudgment.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a system and method for determining call-related risk based on voiceprint feature recognition. Background Art

[0002] Voiceprint recognition is a type of biometric technology, also known as speaker recognition, which includes speaker identification and speaker confirmation. The process of voiceprint recognition is to convert sound signals into electrical signals, and then use computers to identify them. Different tasks and applications will use different voiceprint recognition technologies. For example, during a transaction, it is necessary to use speaker confirmation technology to determine the risk of a call and whether the transaction is normal.

[0003] The so-called voiceprint is the sound wave spectrum that carries speech information displayed by electroacoustic instruments. The production of human language is a complex physiological and physical process between the human body's language center and the vocal organs. The vocal organs used by people when speaking - tongue, teeth, larynx, lungs, and nasal cavity - vary greatly in size and shape from person to person, so the voiceprints of any two people are different. The acoustic characteristics of each person's voice are both relatively stable and variable, and are not absolute and unchanging. This variation can come from physiology, pathology, psychology, simulation, camouflage, and is also related to environmental interference;

[0004] However, the timeliness and efficiency of existing voiceprint feature recognition in phone calls are mostly average. It is difficult to detect the risks of monitoring phone calls in a timely manner, and it is easy to make misjudgments due to poor call quality, which affects the accuracy of judgment and the user experience. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a system and method for determining call-related risks based on voiceprint feature recognition, which can effectively solve the problems that the timeliness and efficiency of the prior art voiceprint feature recognition in the process of being applied in calls are mostly general, and it is difficult to detect the risks of the call process in time. It is easy to make misjudgments due to poor call quality, which affects the accuracy of the judgment and the user experience.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] The present invention discloses a method for realizing call-related risk judgment based on voiceprint feature recognition, comprising the following steps:

[0010] Step 1: Record and monitor the call audio and upload it to the database;

[0011] Step 2: Separate the dual-channel audio channels and extract the user-side conversation;

[0012] Step 3: Remove the silent audio segments through intelligent de-mute means and merge other valid audios into new audio;

[0013] Step 4: Extract the voiceprint features of the user's call audio based on the trained model through artificial intelligence deep learning algorithm;

[0014] Step 5: Upload the call audio to the database;

[0015] Step 6: Screen the retained call audio and evaluate and grade it based on quality factors;

[0016] Step 7: Prioritize high-quality call audio;

[0017] Step 8: Based on cosine similarity, the difference in voiceprint features between different rounds of calls is expressed by relative similarity. The similarity and the total number of times that the two-way comparison is below the threshold are analyzed. The call information with high similarity and a small number of times below the threshold is more likely to be effective.

[0018] Step 9: Determine whether this call and other calls are made by the same user;

[0019] Step 10: The decision is passed and the judgment is normal;

[0020] Step 11: The decision was not passed and was judged to be abnormal;

[0021] Step 12: Retain questionable data;

[0022] Step 13: After a second manual verification, resubmit.

[0023] Furthermore, in the process of extracting the user's side call in Step 2, it is necessary to convert the audio format through Python and adjust it to an adaptive format to separate the dual-channel audio and distinguish the two parties of the conversation.

[0024] Furthermore, the intelligent de-mute method in Step 3 is based on the waveform diagram of the vad mute detection. After audio cutting and valid audio merging, the valid audio data of a call is finally intercepted.

[0025] Furthermore, the artificial intelligence in Step 4 is based on the Resnet deep learning algorithm and the feature extraction capability of the multi-layer neural network. It obtains the voiceprint feature-related parameters through data training, and can eventually extract the voiceprint features corresponding to the call. Different speakers can be distinguished through voiceprint features based on the differences in each person's inherent characteristics such as voice and color.

[0026] Furthermore, the quality factors in step Step 6 include: audio clarity, audio data size, and audio data storage time.

[0027] Furthermore, the manual verification method in Step 13 includes: comparing historical audio information and checking system failures.

[0028] Furthermore, the screening trigger factor in step 6 is: whenever the call information of the same user is updated and added, the audio information of the same user can be screened again, that is, the addition of new call information will be compared and screened with the historical information one by one.

[0029] The system for determining call-related risk based on voiceprint feature recognition includes:

[0030] The central control terminal is used to control the operation of the overall system and send operating instructions;

[0031] An information collection module is used to collect call audio data during a call;

[0032] The storage module is used as a general storage location for data and can read and write information;

[0033] A processing module, used to process call audio information, de-mute the call audio, and merge valid call audio;

[0034] An extraction module is used to extract the voiceprint features of the user's call audio;

[0035] The comparison module is used to analyze the user's real-time call audio and compare it with the historical voiceprint information;

[0036] The temporary storage module is used to record and transfer the audio of calls that have not passed the decision;

[0037] The review module is used to manually check the audio of calls that have not passed the decision and analyze whether there is any interference information that affects the decision result;

[0038] A screening module is used to screen the retained historical audio data;

[0039] The grading module is used to grade the filtered historical audio data and use high-quality audio data as priority reference information.

[0040] Furthermore, the verification module and the storage module are interactively connected via a wireless network, and the verification information is submitted for preservation and read into storage.

[0041] Furthermore, the comparison module is interactively connected to a warning module via a wireless network, which is used to give an alarm reminder to the staff when the voiceprint information is inconsistent.

[0042] (III) Beneficial effects

[0043] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0044] 1. The present invention can greatly improve the efficiency and timeliness of the voiceprint feature recognition process, timely monitor and discover the risks in the call process, improve security, reduce hidden dangers, and is conducive to protecting the property safety of users and reducing the company's losses in risk events.

[0045] 2. The present invention can conduct a second manual verification on questionable data, reduce system misjudgment caused by environmental factors, improve the accuracy of judgment, avoid misjudgment and cause great losses, and can continuously compare the voiceprint data of the same user, using high-quality voiceprint data as a priority reference basis to further improve the accuracy of judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A flowchart of a method for realizing call-related risk judgment based on voiceprint feature recognition;

[0048] Figure 2 A schematic diagram of the structure of a system for determining call-related risk based on voiceprint feature recognition;

[0049] Figure 3 A schematic diagram showing the implementation process of a method for determining call-related risk based on voiceprint feature recognition;

[0050] The numbers in the figure represent: 1. Central control terminal; 2. Information collection module; 3. Storage module; 4. Processing module; 5. Extraction module; 6. Comparison module; 7. Warning module; 8. Temporary storage module; 9. Verification module; 10. Screening module; 11. Classification module. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The present invention will be further described below in conjunction with the embodiments.

[0053] Example 1

[0054] This embodiment provides a method for realizing a system for judging call-related risks based on voiceprint feature recognition. Figure 1 and 3 As shown, the following steps are included:

[0055] Step 1: Record and monitor the call audio and upload it to the database;

[0056] Step 2: Separate the dual-channel audio channels and extract the user-side conversation;

[0057] Step 3: Remove the silent audio segments through intelligent de-mute means and merge other valid audios into new audio;

[0058] Step 4: Extract the voiceprint features of the user's call audio based on the trained model through artificial intelligence deep learning algorithm;

[0059] Step 5: Upload the call audio to the database;

[0060] Step 6: Screen the retained call audio and evaluate and grade it based on quality factors;

[0061] Step 7: Prioritize high-quality call audio;

[0062] Step 8: Based on cosine similarity, the difference in voiceprint features between different rounds of calls is represented by presenting relative similarity, and the similarity and the total number of times below the threshold are analyzed;

[0063] Step 9: Determine whether this call and other calls are made by the same user;

[0064] Step 10: The decision is passed and the judgment is normal;

[0065] Step 11: The decision was not passed and was judged to be abnormal;

[0066] Step 12: Retain questionable data;

[0067] Step 13: After a second manual verification, resubmit.

[0068] like Figure 1 As shown, in the process of extracting the user's call in Step 2, the audio format needs to be converted by python and adjusted to an adaptive format to separate the dual-channel audio and distinguish the two parties of the conversation.

[0069] like Figure 1 As shown, the intelligent mute removal method in step Step 3 is based on the waveform diagram of the vad mute detection. After audio cutting and valid audio merging, the valid audio data of a call is finally intercepted.

[0070] like Figure 1 As shown, the quality factors in step Step 6 include: audio clarity, audio data size, and audio data storage time.

[0071] like Figure 1 As shown, the manual verification method in step Step 13 includes: comparing historical audio information and checking system failures.

[0072] like Figure 1 As shown, the screening trigger factor in step 6 is: whenever the call information of the same user is updated and added, the audio information of the same user can be screened again.

[0073] Through this setting, the system can greatly improve the efficiency and timeliness of voiceprint feature recognition, timely monitor and discover risks in the call process, improve security, reduce hidden dangers, and help protect the property safety of users and reduce the company's losses in risk events;

[0074] It can conduct a second manual verification of questionable data, reduce system misjudgments caused by environmental factors, improve the accuracy of judgments, avoid misjudgments and cause major losses, and continuously compare the voiceprint data of the same user, using high-quality voiceprint data as a priority reference to further improve the accuracy of judgments.

[0075] Example 2

[0076] This embodiment provides a system for determining call-related risk based on voiceprint feature recognition. Figure 2 As shown, including:

[0077] Central control terminal 1, used for overall control of the system operation and sending operation instructions;

[0078] Information collection module 2, used to collect call audio data during a call;

[0079] Storage module 3, used as a general storage place for data, and capable of reading and writing information;

[0080] Processing module 4, used to process call audio information, de-mute the call audio, and merge valid call audio;

[0081] Extraction module 5, used to extract voiceprint features of user call audio;

[0082] Comparison module 6, used to analyze the user's real-time call audio and compare it with the historically stored voiceprint information;

[0083] The temporary storage module 8 is used to record and transfer the call audio that has not passed the decision;

[0084] The review module 9 is used to manually check the audio of calls that have not passed the decision and analyze whether there is any interference information that affects the decision result;

[0085] A screening module 10 is used to screen the retained historical audio data;

[0086] The grading module 11 is used to grade the filtered historical audio data and use the audio data with high quality as priority reference information.

[0087] like Figure 2 As shown, the verification module 9 is interactively connected with the storage module 3 via a wireless network, and the verification information is submitted for storage and read into the storage.

[0088] like Figure 2 As shown, the comparison module 6 is interactively connected to the warning module 7 via a wireless network: it is used to give an alarm reminder to the staff when the voiceprint information is inconsistent.

[0089] When the system is installed, the central control terminal 1 performs overall control operations. The information collection module 2 collects call audio data during the call and stores it through the storage module 3. The processing module 4 removes the mute on the audio data and merges the effective call audio. The extraction module 5 extracts the user's voiceprint features. After that, the comparison module 6 analyzes and compares the user's real-time call audio. When the voiceprint information is inconsistent, the warning module 7 issues an alarm and transfers and records the problem data through the temporary storage module 8. Finally, the grading module 11 grades the filtered historical audio data and uses high-quality audio data as priority reference information.

[0090] Example 3

[0091] like Figure 1As shown, this embodiment is based on the artificial intelligence in step Step 4 of the above embodiment, and provides a Resnet-based deep learning algorithm. According to the feature extraction capability of the multi-layer neural network, the voiceprint feature-related parameters are obtained through data training, and finally the voiceprint features corresponding to the call can be extracted. According to the differences in each person's inherent characteristics such as voice color, different speakers can be distinguished through voiceprint features.

[0092] Use Python to create a data list. The format of the data list is: voice file path\t voice classification label. The voice classification label is the unique ID of the speaker. Different voice data sets can be written into the same data list by writing corresponding functions to generate data lists.

[0093] Convert all mp3 audio to wav format, delete the wrong data after creating the data list, create the data list and mean standard value for training reading, convert the speech data into the amplitude spectrum of short-time Fourier transform, and use librosa to calculate the audio features;

[0094] The training model uses a modified resnet34 model. After setting the data input layer, its size is used as the shape of the amplitude spectrum of the short-time Fourier transform. After each round of training, a model evaluation is performed to calculate the accuracy of the model to observe the convergence of the model. Similarly, the model is saved once after each round of training. The model parameters that can be restored for training are saved separately, which can also be used as pre-training model parameters. The prediction model is also saved for subsequent predictions;

[0095] After training, save the prediction model and use it to predict the audio features in the test set. Then use the audio features for pairwise comparison. The threshold is controlled from 0 to 1 with a step size of 0.01 to find the best threshold and calculate the accuracy.

[0096] In summary, the central control terminal 1 performs the overall control operation, the information collection module 2 collects the call audio data during the call, stores it through the storage module 3, the processing module 4 removes the mute of the audio data, merges the effective call audio, and extracts the user's voiceprint features through the extraction module 5. After that, the user's real-time call audio is analyzed and compared through the comparison module 6. When the voiceprint information is inconsistent, the warning module 7 issues an alarm, and the problem data is transferred and recorded through the temporary storage module 8. Finally, the filtered historical audio data is graded through the classification module 11, and the audio data with high quality is used as priority reference information;

[0097] The use of this system can greatly improve the efficiency and timeliness of the voiceprint feature recognition process, timely monitor and discover risks in the call process, improve security, reduce hidden dangers, which is beneficial to protecting the property safety of users and reducing the company's losses in risk events. It can conduct a second manual verification of questionable data, reduce system misjudgments caused by environmental factors, improve the accuracy of judgments, avoid misjudgments and cause large losses, and continuously compare the voiceprint data of the same user, using high-quality voiceprint data as a priority reference to further improve the accuracy of judgments.

[0098] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining call-related risk based on voiceprint feature recognition, characterized in that: The following steps are involved: Step 1: Record and monitor the call audio and upload it to the database; Step 2: Separate the dual-channel audio channels and extract the user-side conversation; Step 3: Remove the silent audio segments through intelligent de-mute means and merge other valid audios into new audio; Step 4: Extract the voiceprint features of the user's call audio based on the trained model through artificial intelligence deep learning algorithm; Step 5: Upload the call audio to the database; Step 6: Screen the retained call audio and evaluate and grade it based on quality factors; Step 7: Prioritize high-quality call audio; Step 8: Based on cosine similarity, the difference in voiceprint features between different rounds of calls is expressed by relative similarity. The similarity and the total number of times below the threshold in pairwise comparison are analyzed. The call information with high similarity and a small total number of times below the threshold is more likely to be effective. Step 9: Determine whether this call and other calls are made by the same user; Step 10: The decision is passed and the judgment is normal; Step 11: The decision was not passed and was judged to be abnormal; Step 12: Retain questionable data; Step 13: After a second manual verification, resubmit; The intelligent mute removal method in Step 3 is based on the waveform diagram of the vad mute detection. After audio cutting and valid audio merging, the valid audio data of a call is finally intercepted; The screening trigger factor in step 6 is: whenever the call information of the same user is updated and added, the audio information of the same user can be screened again.

2. The method for determining call-related risk based on voiceprint feature recognition according to claim 1, characterized in that: During the extraction of the user-side call in Step 2, the audio format needs to be converted by Python and adjusted to an adaptive format to separate the dual-channel audio and distinguish the two parties of the conversation.

3. The method for determining call-related risk based on voiceprint feature recognition according to claim 1, characterized in that: In the step 4, based on the Resnet deep learning algorithm, according to the feature extraction capability of the multi-layer neural network, the voiceprint feature related parameters are obtained by training the model, and finally the voiceprint features corresponding to the call can be extracted. According to the differences in the inherent characteristics of each person's voice, different speakers can be distinguished by the voiceprint features.

4. The method for determining call-related risk based on voiceprint feature recognition according to claim 1, characterized in that: The quality factors in Step 6 include: audio clarity, audio data size, and audio data storage time.

5. The method for determining call-related risk based on voiceprint feature recognition according to claim 1, characterized in that: The manual verification method in Step 13 includes: comparing historical audio information and checking system failures.

6. A system for determining call-related risk based on voiceprint feature recognition, the system being an implementation system of the method for determining call-related risk based on voiceprint feature recognition as claimed in any one of claims 1 to 5, characterized in that: include: The central control terminal (1) is used to control the operation of the overall system and send operating instructions; The information collection module (2) is used to collect call audio data during a call; A storage module (3) is used as a general storage location for data and can read and write information; The processing module (4) is used to process the call audio information, remove the muting of the call audio, and merge the valid call audio; An extraction module (5), used to extract voiceprint features of the user's call audio; A comparison module (6) is used to analyze the user's real-time call audio and compare it with the historically stored voiceprint information; A temporary storage module (8) is used to record and transfer the audio of calls that have not passed the decision; The review module (9) is used to manually check the audio of calls that have not passed the decision and analyze whether there is any interference information that affects the decision result; A screening module (10), used for screening the retained historical audio data; A grading module (11) is used to grade the filtered historical audio data and use high-quality audio data as priority reference information; The verification module (9) is interactively connected with the storage module (3) via a wireless network, and the verification information is submitted for storage and read into the storage; The comparison module (6) is interactively connected to a warning module (7) via a wireless network, and is used to issue an alarm to the staff when the voiceprint information is inconsistent.

Images