Voiceprint recognition method and device based on clustering analysis, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a voiceprint recognition method based on clustering analysis, and the method comprises the steps: obtaining to-be-detected first voice data and to-be-detected second voice data, carrying out the voice activity detection, recognizing a voice activity region, and segmenting the voice data into a plurality of voice segments; performing clustering analysis on the voice segments to generate a plurality of new voice sequences, and respectively generating corresponding voiceprint portraits; and carrying out weighting processing on the voiceprint portraits based on the weighting parameters, analyzing the similarity between the weighted voiceprint portraits, and finally judging whether the two sections of voice data are from the same speaker or not. According to the voiceprint recognition method, the voice segment clustering analysis and the weighting processing are combined, so that the influence of non-essential difference caused by environmental noise, emotion change and the like on voiceprint recognition is effectively reduced, the accuracy and the stability of voiceprint recognition are improved, and particularly, better robustness is achieved in a complex environment.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a voiceprint recognition method, device, equipment and storage medium based on cluster analysis. Background Art

[0002] In the financial field, voiceprint recognition technology is gradually applied to scenarios such as identity authentication and risk control, such as customer identity verification, voice payment, and transaction security verification. Compared with traditional password verification or biometric recognition (such as fingerprint, facial recognition) technology, voiceprint recognition technology is convenient and contactless, which can improve user experience and authentication efficiency. However, the existing voiceprint recognition technology still has many shortcomings, which limits its wide application in financial scenarios.

[0003] First of all, human voice characteristics are not static and are easily affected by the external environment and the speaker's own state. For example, background noise interference, changes in transmission channels, fluctuations in voice characteristics caused by illness or emotional changes of the speaker, and deliberate imitation of disguised voices can all cause the same person to show significant differences in voiceprint characteristics in different situations. Although these differences are "non-essential differences", that is, they do not change the structure of the vocal organs, existing voiceprint recognition systems find it difficult to accurately distinguish these non-essential differences from the speaker's true acoustic characteristics, thus affecting recognition accuracy and stability.

[0004] Secondly, current voiceprint recognition technology mostly relies on repeated sampling and preprocessing technology to improve accuracy, which may have a certain effect in static detection. However, in dynamic detection and real-time application scenarios, existing methods are difficult to cope with the changes in voiceprint features over time and in context. In particular, when dealing with continuous scenarios involving variable voice signals, the recognition system is easily disturbed, resulting in reduced reliability of identity authentication.

[0005] In addition, the existing voiceprint recognition system lacks robustness in dealing with complex environments (such as noisy environments and multi-channel interference), and common noise reduction and channel compensation technologies are difficult to adapt to the ever-changing financial application scenarios, affecting the actual application effect. This deficiency is particularly prominent in financial transaction scenarios, because transaction verification requires high precision and high stability to ensure the security of user funds and data. Summary of the invention

[0006] The main purpose of the present invention is to provide a voiceprint recognition method, device, equipment and storage medium based on cluster analysis, aiming to solve the technical problem that the existing technology cannot effectively distinguish non-essential differences from the essential characteristics of the speaker in voiceprint feature detection, resulting in insufficient accuracy and stability of voiceprint recognition in complex environments.

[0007] To achieve the above object, the present invention provides a voiceprint recognition method based on cluster analysis, comprising:

[0008] Acquire first voice data and second voice data to be detected;

[0009] Performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data;

[0010] dividing the first voice data and the second voice data into a plurality of first voice segments and a plurality of second voice segments respectively according to the voice activity area;

[0011] Performing cluster analysis on the first speech segment and the second speech segment respectively to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data;

[0012] splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0013] Based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generate corresponding first voiceprint portraits and second voiceprint portraits;

[0014] Performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to weighted parameters;

[0015] Analyzing the similarity between the weighted first voiceprint image and the second voiceprint image;

[0016] It is identified based on the similarity whether the first voice data and the second voice data belong to the same speaker.

[0017] Furthermore, in order to achieve the above object, the present invention provides a voiceprint recognition device based on cluster analysis, comprising:

[0018] A voice data acquisition module, used to acquire first voice data and second voice data to be detected;

[0019] A voice activity detection module, configured to perform voice activity detection on the first voice data and the second voice data, and identify and mark voice activity areas in the first voice data and the second voice data;

[0020] A speech segmentation module, used for segmenting the first speech data and the second speech data into a plurality of first speech segments and a plurality of second speech segments respectively according to the speech activity area;

[0021] A cluster analysis module, used to perform cluster analysis on the first speech segment and the second speech segment respectively, to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data;

[0022] A speech sequence generation module, used for splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0023] A voiceprint portrait generation module, used to generate corresponding first voiceprint portraits and second voiceprint portraits based on the multiple new first voice sequences and the multiple new second voice sequences;

[0024] A weighted processing module, used for performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to a weighted parameter;

[0025] A similarity analysis module, used to analyze the similarity between the first voiceprint image and the second voiceprint image after weighted processing;

[0026] The voiceprint recognition module is used to identify whether the first voice data and the second voice data belong to the same speaker according to the similarity.

[0027] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a voiceprint recognition program based on clustering analysis stored in the memory and executable on the processor, wherein the voiceprint recognition program based on clustering analysis, when executed by the processor, implements the steps of the voiceprint recognition method based on clustering analysis as described above.

[0028] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a voiceprint recognition program based on cluster analysis is stored, and when the voiceprint recognition program based on cluster analysis is executed by a processor, the steps of the voiceprint recognition method based on cluster analysis as described above are implemented.

[0029] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a voiceprint recognition method based on cluster analysis. By acquiring the first voice data and the second voice data to be detected, voice activity detection is performed and the voice activity area is identified, and the voice data is divided into multiple voice segments respectively; multiple new voice sequences are generated by clustering analysis of the voice segments, and corresponding voiceprint portraits are generated respectively; the voiceprint portraits are weighted based on weighting parameters, and the similarity between the weighted voiceprint portraits is analyzed, and finally it is determined whether the two voice data are from the same speaker. The present invention effectively reduces the impact of non-essential differences caused by environmental noise, emotional changes, etc. on voiceprint recognition by combining voice segment cluster analysis with weighted processing, thereby improving the accuracy and stability of voiceprint recognition, especially having better robustness in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0031] Figure 1 A schematic diagram of an application environment of a voiceprint recognition method based on cluster analysis in an embodiment of the present invention;

[0032] Figure 2 It is a flow chart of an embodiment of a voiceprint recognition method based on cluster analysis of the present invention;

[0033] Figure 3 It is a functional module diagram of a preferred embodiment of a voiceprint recognition device based on cluster analysis of the present invention;

[0034] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0035] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0037] The voiceprint recognition method based on cluster analysis provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the user terminal communicates with the server terminal through a network. The server terminal can obtain the first voice data and the second voice data to be detected through the user terminal, perform voice activity detection and identify the voice activity area, and divide the voice data into multiple voice segments respectively; generate multiple new voice sequences by clustering analysis of the voice segments, and generate corresponding voiceprint portraits respectively; perform weighted processing on the voiceprint portraits based on weighted parameters, analyze the similarity between the weighted voiceprint portraits, and finally determine whether the two voice data are from the same speaker. The present invention combines voice segment clustering analysis with weighted processing to effectively reduce the impact of non-essential differences caused by environmental noise, emotional changes, etc. on voiceprint recognition, improve the accuracy and stability of voiceprint recognition, and especially have better robustness in complex environments. Among them, the user terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server terminal can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0038] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a voiceprint recognition method based on cluster analysis provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0039] like Figure 2 As shown, the voiceprint recognition method based on cluster analysis proposed by the present invention includes the following steps:

[0040] S10, obtaining first voice data and second voice data to be detected;

[0041] In this embodiment, two segments of voice data to be detected are collected from the audio data source, which are marked as "first voice data" and "second voice data". These data may come from real-time voice input or pre-stored audio files, and are used in the subsequent voiceprint analysis and recognition process.

[0042] Real-time voice input can be obtained through a microphone device, sampled using an audio interface, and the collected audio signal is converted into a processable digital audio data format (such as PCM format or WAV format).

[0043] Pre-stored audio files can be read through the audio processing program, supporting multiple audio formats (such as MP3, WAV, FLAC, etc.). The audio reading module will parse the file header information, extract the audio data and convert it into a format suitable for subsequent processing.

[0044] Assign labels to the collected speech data so that different speech data sources can be distinguished in the subsequent processing. This labeling operation helps to keep the data processing orderly, especially in the comparative analysis of multiple speech data.

[0045] Each piece of speech data may be stored using a specific data structure (such as a data object or a data frame), and a tag attribute (such as "speech 1", "speech 2") may be added thereto.

[0046] The labeled speech data can be stored in memory or written to a database, ensuring that it can be easily accessed and distinguished during subsequent analysis.

[0047] Example description: When a customer calls the bank customer service hotline, the system can collect the customer's voice data in real time through the microphone as the first voice data. At the same time, the system can call the customer's historical voice record from the database as the second voice data. In this way, by acquiring and marking these two pieces of voice data, the system can lay the foundation for subsequent voiceprint identification, thereby achieving efficient and accurate customer identity authentication and improving the security and user experience of banking services.

[0048] By acquiring the first voice data and the second voice data to be detected, basic audio data for voiceprint recognition can be effectively prepared, and the flexibility and reliability of data collection can be ensured.

[0049] S20, performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data;

[0050] In this embodiment, voice activity detection refers to the process of distinguishing speech and non-speech parts from an audio stream. By analyzing the characteristics of the audio signal (such as energy, spectrum, zero crossing rate, etc.), VAD can identify speech segments and exclude non-speech segments (such as background noise or silence). The core purpose of this step is to improve the accuracy of subsequent voiceprint processing and ensure that only the parts containing valid speech features are used.

[0051] The short-term energy of the audio signal can be calculated and compared with a preset energy threshold. If the energy of the audio frame exceeds the threshold, it is determined to be a speech activity area, otherwise it is a non-speech part.

[0052] Fast Fourier Transform (FFT) can also be used to perform spectrum analysis on audio signals and detect the presence of high-frequency components to determine voice activity. Voice usually has a strong high-frequency component, while background noise or silence has a relatively flat spectrum.

[0053] You can also use pre-trained machine learning models (such as deep neural networks or support vector machines) to detect speech activity areas. The models are trained using a set of labeled audio data and perform well in more complex audio environments.

[0054] Identifying speech activity regions refers to marking segments of audio data that contain speech, usually by separating speech from non-speech parts to determine when speech activity starts and ends. This marking helps accurately extract speech features and exclude unnecessary noise or silence interference.

[0055] The start and end time points of each identified speech activity area can be marked to form a timestamp. This function can be implemented using an audio analysis library (such as PyDub or librosa). The identified speech activity area is divided into several segments and stored as independent fragments to facilitate subsequent clustering analysis and feature extraction.

[0056] Marking speech activity regions means marking the recognized speech segments in the audio data for subsequent processing. The marker of each speech activity region usually contains a timestamp, segment number, and related attribute information. Use a data structure (such as a dictionary or list) to store detailed information of each speech activity region, including the start and end time of the segment, volume characteristics, spectral characteristics, etc. Insert annotation information into the audio file or generate a separate marker file (such as JSON format) to record the location and attributes of the speech activity region.

[0057] Example: When a customer calls a bank for identity verification, the system collects voice data in real time and performs voice activity detection. The system can automatically identify and mark the voice activity area, remove the silent part and background noise, and provide accurate voice segments for subsequent voiceprint comparison. This process can significantly improve the accuracy of identity verification, reduce the error rate caused by environmental noise, and ensure the security of financial transactions.

[0058] By performing voice activity detection on the first voice data and the second voice data and accurately identifying and marking the voice activity area, it is possible to effectively extract important segments containing voice features and reduce the impact of noise and silence on voiceprint analysis.

[0059] S30, dividing the first voice data and the second voice data into a plurality of first voice segments and a plurality of second voice segments respectively according to the voice activity area;

[0060] In this embodiment, the continuous audio data is segmented into several independent voice segments using the previously identified and marked voice activity areas. Each voice segment contains a complete voice activity interval, and the non-voice part is excluded. The purpose of segmentation is to organize the audio data into units that are easier to analyze and process, so as to improve the efficiency and accuracy of subsequent voiceprint recognition. Using the start and end time of the voice activity area as boundaries, the first voice data and the second voice data are respectively segmented into multiple voice segments. Segmentation can be achieved through an audio processing library (such as PyDub or librosa) to generate multiple audio segment files or audio segment objects. Call the audio cutting function to segment the original audio data according to the time axis based on the timestamp of each voice activity area to form multiple voice segments. The segmented voice segments can be saved in memory or local files for subsequent analysis.

[0061] The segmented speech data is organized into multiple segments, namely, "first speech segment" and "second speech segment". These segments are independent speech activity intervals extracted from the first speech data and the second speech data, ensuring the integrity and coherence of the speech data.

[0062] Assign a unique number to each generated speech segment and store the segments in a data structure (such as a list or array). Each segment can contain metadata information, such as the start time, end time, and duration of the segment, to facilitate subsequent processing.

[0063] During the segmentation process, you can choose to convert the audio format of each voice segment (such as WAV or MP3 format) and apply simple audio optimization techniques (such as noise reduction or volume adjustment) to improve data quality.

[0064] Example description: When the customer's voice is collected in real time, the system will divide the customer's voice data into multiple voice segments according to the voice activity area. In this way, the system can split the customer's continuous voice stream into independent segments, thereby improving the efficiency and accuracy of voiceprint recognition. The segmented voice segments can be used to compare the customer's voiceprint features to ensure the accuracy of identity verification and reduce the risk of misjudgment caused by background noise or non-voice parts.

[0065] By dividing the first voice data and the second voice data into multiple voice segments according to the voice activity area, the voice data can be effectively organized and managed, and the efficiency and accuracy of subsequent processing can be improved. The interference of irrelevant audio data can be significantly reduced, making the subsequent voiceprint feature extraction and analysis more accurate, especially showing high robustness in dynamic and noisy environments.

[0066] S40, performing cluster analysis on the first voice segment and the second voice segment respectively to obtain a plurality of cluster categories of the first voice data and a plurality of cluster categories of the second voice data;

[0067] In this embodiment, cluster analysis is an unsupervised machine learning method that aims to classify objects with similar features into the same category. In this step, cluster analysis is performed on the first voice segment and the second voice segment respectively to classify each voice segment into different cluster categories according to its voice features. This process can bring similar voice segments together, reduce the impact of background noise and voice feature changes on voiceprint analysis, and lay the foundation for subsequent voiceprint portrait generation.

[0068] Extract features for clustering analysis from each speech segment, such as frequency features, pitch features, energy distribution, etc. Audio processing libraries such as librosa can be used to extract features such as Mel-frequency cepstral coefficients (MFCC), pitch, volume, etc.

[0069] Use appropriate clustering algorithms, such as K-means clustering, hierarchical clustering, or Gaussian mixture model (GMM), to classify speech segments into multiple categories. Select an appropriate clustering algorithm based on data characteristics and application requirements to ensure the accuracy and stability of clustering results.

[0070] Before clustering analysis, the extracted speech features are standardized to eliminate the scale differences between different features and improve the effect of the clustering algorithm.

[0071] After cluster analysis, the first voice data and the second voice data are divided into multiple cluster categories, each of which contains segments with similar voice features. This can effectively organize and classify voice data, facilitating subsequent voiceprint portrait generation and similarity analysis.

[0072] The clustering category of each speech segment is stored in a data structure, such as a dictionary or an array, and the category label, feature vector and other information of each segment are recorded to facilitate subsequent processing.

[0073] Data visualization tools (such as t-SNE or PCA) can be used to visualize the clustering results to verify the clustering effect and adjust and optimize the algorithm parameters.

[0074] Example: When a customer interacts with a bank over the phone, the collected voice data is divided into multiple segments and clustered. The system can classify voice segments into different categories based on features such as pitch, speaking speed, and energy distribution, thereby eliminating abnormal segments caused by noise or emotional changes and retaining only stable and reliable voice features for voiceprint recognition. This can significantly improve the accuracy of identity authentication and reduce the risk of fraud.

[0075] By performing cluster analysis on the first voice segment and the second voice segment respectively, the voice data can be effectively organized and segments with similar voice features can be classified into the same category. The interference of background noise and voice feature changes on voiceprint analysis can be reduced, and the accuracy and stability of voiceprint image generation can be improved. In addition, cluster analysis can also enhance the system's anti-interference ability, making voiceprint recognition more robust in complex environments.

[0076] S50, splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0077] In this embodiment, in cluster analysis, speech segments in the same cluster category are considered to have similar speech features. By splicing these speech segments belonging to the same category together, a longer speech sequence with consistent features can be generated. The purpose of this is to retain the coherence of similar speech features while reducing interference with the analysis results.

[0078] First, the speech segments in the first speech data are sorted according to the clustering categories to ensure that all segments belonging to the same category are arranged in chronological order to avoid incoherent speech features after splicing.

[0079] Use an audio processing library (such as PyDub, FFmpeg, or librosa) to splice speech fragments of the same category into a complete speech sequence. Volume balancing and transition optimization can be performed during the splicing process to improve the naturalness and coherence of the speech sequence.

[0080] After the splicing is completed, all cluster categories in the first voice data are respectively generated into new voice sequences, and each sequence contains all voice segments of the same category. The generated multiple new voice sequences are used for subsequent voiceprint portrait generation and feature analysis. Each new first voice sequence is stored in memory or a local file, and a unique identifier is assigned to each sequence for subsequent processing and access. Batch processing can be used to store all sequences at once to improve processing efficiency.

[0081] Perform the same processing on the second speech data, and stitch the speech segments within the same cluster category into a new speech sequence. This allows you to maintain the same processing flow and logic as the first speech data in subsequent analysis. Sort the segments in the second speech data by cluster category to ensure the correct stitching order. Sort them in the same way as the first speech data to maintain consistency. Use the same audio processing tools to stitch the speech segments into a new speech sequence, making necessary volume adjustments and transition optimizations.

[0082] The spliced ​​second voice data segments are used to generate multiple new voice sequences, each of which represents a coherent voice feature within a cluster category. These new voice sequences will be used for subsequent voiceprint image generation and similarity analysis.

[0083] The new second speech sequence is also stored in a suitable data structure, and a unique identifier is assigned to each sequence for subsequent use and management.

[0084] By splicing the voice segments belonging to the same cluster category in the first voice data and the second voice data into new voice sequences, similar voice features can be effectively organized and integrated, making the subsequent voiceprint image generation and similarity analysis more stable and accurate. The inconsistency of voice features is reduced, the robustness of voiceprint recognition is improved, and it shows better adaptability, especially when dealing with complex or noisy environments.

[0085] S60, based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generate corresponding first voiceprint portraits and second voiceprint portraits;

[0086] In this embodiment, multiple new first speech sequences are used as input, speech features are extracted and processed through a specific voiceprint image generation method, and finally a first voiceprint image is generated. A voiceprint image is a visual representation of speech signal features, usually presented in the form of a spectrogram, a Mel-frequency cepstral coefficient (MFCC) graph, or other speech feature graphs, and is used to analyze and compare the speech features of the speaker.

[0087] Extract key speech features from each new first speech sequence, such as MFCC, speech spectrum, fundamental frequency, etc. An audio processing library (such as librosa or Kaldi) can be used to perform feature extraction on the audio data to convert the speech sequence into a two-dimensional feature matrix.

[0088] Map the extracted features to the voiceprint image to generate a feature representation in the form of an image. For example, when generating a spectrogram, the short-time Fourier transform (STFT) can be used to decompose the voice data into spectral components and then visualized as a color or grayscale image. The generated first voiceprint image is stored in the form of an image file or matrix to facilitate subsequent analysis and comparison.

[0089] Similar to the generation process of the first voiceprint portrait, the second voiceprint portrait is generated based on the new second speech sequence, which contains the key speech features in the second speech data. This step is intended to provide a basis for subsequent voiceprint comparison and similarity analysis.

[0090] The feature extraction is performed on each new second speech sequence in the same way as the first speech sequence to ensure the comparability and consistency of the features.

[0091] Map the features to the voiceprint image to generate the corresponding spectrogram or other speech feature graph. The same visualization technique can be used to convert the extracted features into a two-dimensional image to ensure that the generated voiceprint images are comparable.

[0092] The second voiceprint image is stored in a suitable data structure for subsequent similarity analysis and comparison.

[0093] By generating the corresponding first voiceprint portrait and second voiceprint portrait based on multiple new first speech sequences and second speech sequences, the key features of the speech data can be effectively captured and represented. The generated voiceprint portrait can more accurately reflect the speech characteristics of the speaker, and improve the reliability and accuracy of subsequent voiceprint comparison. Especially in complex speech environments, it can enhance the robustness of voiceprint recognition and effectively reduce the interference caused by noise or changes in speech characteristics.

[0094] S70, performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to a weighted parameter;

[0095] In this embodiment, the generated first voiceprint image and second voiceprint image are weighted. The purpose of weighted processing is to adjust the voiceprint features according to different influencing factors (such as age, gender, emotional state, etc.) to reduce the impact of non-essential differences on the voiceprint comparison results. By weighting parameters, the accuracy and robustness of voiceprint recognition can be improved, especially in a changing audio environment, enhancing the system's ability to cope with complex scenarios.

[0096] First, we identify several key factors that affect voiceprint images, such as background noise level, voice emotion state, pronunciation speed, etc. These factors have different degrees of influence on voiceprint features.

[0097] Based on historical voice data and statistical analysis, the weight parameters of each influencing factor are calculated, and the weight value is used to adjust the importance of the voiceprint feature. For example, by analyzing a large-scale voice data set, the degree of influence of each influencing factor on the result can be determined, and the corresponding weight can be assigned to it.

[0098] Apply the determined weight parameters to weight each feature in the voiceprint image. The feature matrix can be weighted using matrix multiplication, or weights can be applied to different areas of the voiceprint image through convolution operations. The weighted image will more accurately reflect the essential characteristics of the speaker and reduce the impact of non-essential differences.

[0099] By weighting the first voiceprint image and the second voiceprint image according to the weighting parameters, the interference of non-essential differences (such as background noise and emotional changes) on voiceprint recognition can be effectively reduced, thereby improving the accuracy and robustness of voiceprint comparison. It helps to maintain the stability of voiceprint recognition in complex voice environments and can more accurately capture the essential voice characteristics of the speaker.

[0100] S80, analyzing the similarity between the first voiceprint portrait and the second voiceprint portrait after weighted processing;

[0101] In this embodiment, the first voiceprint image and the second voiceprint image that have been weighted are subjected to similarity analysis to determine whether the two voice data segments are from the same speaker. Similarity analysis is usually based on a mathematical measurement method between feature vectors to calculate the similarity between the two voiceprint images, thereby providing a basis for the final identity verification.

[0102] Extract feature vectors for similarity analysis from the weighted voiceprint image. These feature vectors can be feature descriptors of the image, such as frequency distribution, texture pattern, or other key features. These features can be extracted using audio processing and image processing tools such as librosa, OpenCV, or TensorFlow.

[0103] Use an appropriate similarity metric for analysis, such as cosine similarity, Euclidean distance, or Mahalanobis distance. Different metrics are suitable for different scenarios, and the specific choice depends on the nature of the voiceprint features. Cosine similarity is often used to calculate the similarity of high-dimensional feature vectors, while Euclidean distance is suitable for calculating the geometric distance between feature points.

[0104] According to the measurement method, the similarity score between the two feature vectors is calculated. The score value is usually within a predetermined range, such as between 0 and 1, and the higher the value, the higher the similarity. This score will be used for the final voiceprint comparison and judgment.

[0105] Example: In telephone banking services, the customer's voice is collected and converted into a voiceprint image, which is then analyzed for similarity with the customer's voiceprint image stored in the database. By calculating the similarity score, the system can accurately determine the customer's identity and ensure the security of transactions and access. For example, when a customer makes a bank transfer call in a public place, even if the environment is noisy, the system can still provide reliable identity authentication services through weighted processing and accurate similarity analysis. This mechanism can effectively reduce the risk of fraud, improve customer experience, and ensure the security of financial transactions.

[0106] By performing similarity analysis on the weighted first and second voiceprint images, the similarity between the two voiceprint images can be accurately evaluated, thereby determining whether the two voice data segments come from the same speaker. This effectively reduces the impact of environmental noise and non-essential differences, and improves the accuracy and robustness of voiceprint recognition. In particular, it can still maintain a high level of recognition accuracy in a changing voice environment.

[0107] S90: Identify, based on the similarity, whether the first voice data and the second voice data belong to the same speaker.

[0108] In this embodiment, based on the previously calculated similarity score, it is determined whether the two pieces of speech data are produced by the same speaker. The determination process relies on a preset similarity threshold as a basis for distinguishing the same speaker from different speakers. If the similarity score is higher than or equal to the threshold, it is determined that the two pieces of speech data belong to the same speaker; otherwise, it is determined that they are different speakers.

[0109] Set a reasonable similarity threshold based on experimental data or actual application scenarios. The choice of threshold can be optimized by testing different data sets and evaluating their accuracy to ensure that the system can provide efficient judgment in various environments.

[0110] The calculated similarity score is compared with the preset threshold. If the score is greater than or equal to the threshold, the output judgment result is "same speaker"; if the score is less than the threshold, the output judgment result is "different speaker".

[0111] The determination result may be output in the form of a Boolean value (such as “yes” or “no”) or presented through a user interface for further use by the system or user.

[0112] Example: When a customer makes a high-value transfer over the phone, the system collects and processes the customer's voice data and compares it with the voiceprint image stored in the database. If the similarity score reaches the preset security threshold, the system will automatically determine that the customer's identity is successfully authenticated and allow the transaction. If the score is lower than the threshold, the system will trigger further security verification steps, such as requiring the customer to answer security questions or confirm through other identity verification methods. This mechanism not only improves the security of financial transactions, but also prevents potential fraud in a timely manner when the customer's identity is abnormal, ensuring the safety of financial assets.

[0113] By determining whether the first voice data and the second voice data belong to the same speaker based on similarity, efficient and accurate identity authentication can be achieved. By using objective similarity scores and preset thresholds, the possibility of human misjudgment is greatly reduced, and it can adapt to voice environments of different complexities. Whether in a quiet environment or a noisy scene, the system can maintain high accuracy and robustness, thereby improving the reliability of voice recognition and identity authentication.

[0114] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a voiceprint recognition method based on cluster analysis, which obtains first voice data and second voice data to be detected, performs voice activity detection and identifies voice activity areas, and divides the voice data into multiple voice segments; generates multiple new voice sequences by clustering the voice segments, and generates corresponding voiceprint portraits respectively; performs weighted processing on the voiceprint portraits based on weighted parameters, analyzes the similarity between the weighted voiceprint portraits, and finally determines whether the two voice data are from the same speaker. The present invention combines voice segment cluster analysis with weighted processing to effectively reduce the impact of non-essential differences caused by environmental noise, emotional changes, etc. on voiceprint recognition, thereby improving the accuracy and stability of voiceprint recognition, especially having better robustness in complex environments.

[0115] In one embodiment, the above S40 includes:

[0116] S401, extracting a first speech feature of each first speech segment and a second speech feature of each second speech segment;

[0117] S402, performing standardization processing on the first speech feature and the second speech feature respectively;

[0118] S403: Use a clustering module to perform cluster analysis on the first speech feature after the standardization processing and the second speech feature after the standardization processing, respectively, to obtain multiple cluster categories of the first speech data and multiple cluster categories of the second speech data.

[0119] In this embodiment, speech features are extracted from each first speech segment, and these features are used to describe the characteristics of the speech signal, which is convenient for subsequent cluster analysis. Speech features generally include frequency features, time domain features, energy features, etc., such as Mel Frequency Cepstral Coefficients (MFCC), pitch, volume changes, etc.

[0120] Features such as MFCC, zero-crossing rate, pitch, etc. can be extracted using an audio processing library such as librosa. Each speech segment is processed through a feature extraction module to generate a set of feature vectors that describe the key characteristics of the speech signal.

[0121] Standardization aims to eliminate the scale differences between different features so that all features are within the same scale range to improve the effect of clustering analysis. Standardization is usually achieved by calculating the mean and standard deviation of the features and then adjusting each feature value. Use the formula (x-mean) / std, where x is the feature value, mean is the mean of the feature, and std is the standard deviation of the feature. The standardized feature values ​​are converted to a range with a mean of 0 and a standard deviation of 1, which makes it easier for the clustering algorithm to better separate the features.

[0122] Cluster analysis refers to the classification of similar speech features into the same category. The purpose is to classify speech segments with similar attributes for subsequent voiceprint portrait generation. Common clustering algorithms include K-means clustering, hierarchical clustering, and Gaussian mixture model (GMM). When using the K-means algorithm, first set the number of cluster categories and randomly select the initial cluster center. Iterate to update the cluster center and assign feature vectors until the cluster center converges. The standardized feature vectors can be clustered using the KMeans method or other clustering modules in a machine learning library (such as scikit-learn).

[0123] Similar to the feature extraction process of the first speech segment, speech features are extracted from each second speech segment to describe the key characteristics of the speech signal. The feature extraction method is the same to ensure that the features are comparable and consistent.

[0124] The same techniques and tools as those used for the first speech segment are used to extract features from the second speech segment and generate a set of feature vectors, keeping the two sets of data processed in the same way.

[0125] The features of the second speech segment are normalized to ensure that they are analyzed within the same scale range. The purpose of normalization is to balance the influence of different features so that the clustering algorithm can group the features more accurately.

[0126] The (x-mean) / std) is also used for standardization. The standardized features are stored in a data structure to facilitate the input of clustering analysis.

[0127] The standardized second speech features are input into the clustering module for cluster analysis to obtain multiple cluster categories of the second speech data. The analysis process is the same as the cluster analysis of the first speech features, ensuring that the analysis results are consistent and comparable.

[0128] Use the same clustering algorithm and parameter configuration to ensure that the clustering analysis results of the two groups of speech data are compared and analyzed under the same conditions. The consistency of the results can be maintained by setting the random seed.

[0129] This embodiment can effectively organize and classify voice data by performing cluster analysis on the first voice segment and the second voice segment, and classify similar voice segments into the same category. The randomness and interference of voice features are reduced, and the stability and accuracy of voiceprint analysis are improved. Cluster analysis can also improve the manageability of data, provide a clearer feature distribution for subsequent voiceprint portrait generation and identity verification, and show higher robustness, especially in noisy or complex voice environments.

[0130] In one embodiment, the above S403 includes:

[0131] S4031, determining the number of clustering categories according to preset parameters;

[0132] S4032, randomly selecting an initial cluster center for each cluster category;

[0133] S4033, analyzing the distance between each first speech feature and each second speech feature and each initial cluster center based on a distance measurement method, and assigning each first speech feature and each second speech feature to the initial cluster center with the closest distance;

[0134] S4034, respectively analyzing the average values ​​of all first speech features and all second speech features in each cluster category, and using the average values ​​to update the cluster center corresponding to the cluster category;

[0135] S4035, repeating the allocation operation and the update operation, when the change range of all cluster centers is less than the preset threshold, ending the cluster analysis, and obtaining multiple cluster categories of the first voice data and the second voice data respectively.

[0136] In this embodiment, when cluster analysis begins, it is necessary to set the number of cluster categories (i.e., the number of clusters) based on the complexity of the data features or experimental experience. This parameter determines how many categories the speech features are divided into. Usually, a reasonable number of clusters is selected based on data analysis or prior knowledge to better describe the distribution of speech features. The number of clusters is determined by analyzing the distribution of data or conducting preliminary experiments. For example, the elbow method or silhouette coefficient is used to determine the appropriate number of cluster categories. Model selection or cross-validation techniques are used to automatically select the optimal number of categories based on the clustering effects of different numbers of clusters.

[0137] In K-means clustering, the selection of the initial cluster center affects the final clustering results and convergence speed. To initialize the clustering process, several points are randomly selected from the standardized speech features as the initial cluster centers, each center corresponding to a cluster category. A random number generator is used to select the initial center points in the dataset to ensure that each center point covers different areas of the entire feature space. Optimization strategies (such as K-means++ initialization) can be used to improve the selection effect of the initial cluster center to reduce the number of iterations of the clustering algorithm and improve accuracy.

[0138] Calculate the distance between each speech feature and all initial cluster centers, usually using a metric such as Euclidean distance or Manhattan distance. Assign each feature to the nearest cluster center to form an initial cluster grouping. Use the Euclidean distance formula to calculate the distance between each feature vector and the cluster center. Traverse all speech features, assign each feature to the nearest cluster center, and update the list of feature vectors contained in each cluster category.

[0139] The cluster center is updated by calculating the average value of all speech features in each category. The new cluster center is determined by the average position of all feature vectors in the category to make the center point closer to the actual feature distribution. All feature vectors in each cluster category are averaged dimension by dimension to obtain the new cluster center coordinates. The cluster center position is updated to the calculated average value and the next iteration is continued.

[0140] The clustering process is an iterative process that requires continuous updating of feature allocation and cluster centers until the position change of all cluster centers is less than the preset convergence threshold. At this point, the clustering is considered stable, the clustering analysis is completed, and multiple cluster categories are generated. Repeat the calculation of feature allocation and center update, and record the change of each cluster center. When the change is less than the threshold, stop the iteration. Use the preset threshold (such as 0.001) as the condition for clustering convergence to ensure that the algorithm can complete the clustering task within a reasonable time.

[0141] This embodiment uses a clustering module to perform cluster analysis on the first and second speech features after standardization, so as to classify speech segments with similar features and significantly improve the organization and processing efficiency of speech data. Cluster analysis helps reduce the interference of noise and non-essential features on voiceprint recognition, and multiple rounds of iterative updates ensure the accuracy and stability of clustering results, so that the system can still maintain efficient recognition performance in complex speech environments.

[0142] In one embodiment, the above S20 includes:

[0143] S201, performing volume normalization processing on the first voice data and the second voice data, and removing background noise;

[0144] S202, extracting key speech features from the processed first speech data and the second speech data respectively;

[0145] S203, using a voice activity detection module to respectively identify voice activity areas in the first voice data and the second voice data according to the key voice features;

[0146] S204, marking the start and end time of the corresponding voice activity area in the first voice data and the second voice data, and generating an identifier of the voice activity area.

[0147] In this embodiment, volume normalization refers to adjusting the overall volume of the audio signal to a consistent level, thereby reducing the volume differences caused by different recording environments or devices. Removing background noise refers to filtering out noise interference in the audio that does not belong to the voice signal, such as wind, ambient sound, etc., to improve the clarity and analyzability of the voice signal. Use an audio processing algorithm (such as RMS normalization) to adjust the volume to a preset target level. The volume normalization function in an audio processing library (such as librosa, PyDub) can be called to implement it. Use spectral subtraction or other noise suppression techniques to remove background noise. Noise features can be extracted by analyzing the silent part of the audio, and then these noise features are subtracted from the overall signal to improve signal quality.

[0148] Extract key features related to speech activity from the preprocessed audio data for subsequent speech activity detection. These features may include energy changes, spectral features, zero crossing rate, etc. of the audio signal to distinguish speech activity from silence. Use audio processing tools to extract features such as energy features, spectral envelopes, or MFCCs for each audio frame. These features can help identify whether there is speech activity in the audio signal.

[0149] The Voice Activity Detection (VAD) module uses the extracted speech features to automatically identify the speech activity area in the audio signal. The VAD module determines whether speech exists by analyzing the changes in signal features in order to separate the speech activity area from the silent part. Set a threshold for energy or spectrum changes. When the speech feature exceeds the threshold, it is judged as a speech activity area; otherwise, it is judged as a silent part. The threshold can be adjusted according to the experiment to adapt to different speech environments. Use a trained classification model (such as a support vector machine or a neural network) to classify the audio frames and distinguish between speech activity areas and silent areas. This method is suitable for more complex audio environments.

[0150] Each detected speech activity region is marked with its start and end time in the audio stream to accurately locate the speech activity. The generated speech activity region identifier includes these timestamps for subsequent speech segmentation and processing. The start and end time of each speech activity region is recorded and stored in a data structure, such as a dictionary or a time series list. These identifiers can be used in subsequent processing steps for speech segmentation or further analysis.

[0151] This embodiment significantly improves the clarity and detectability of the voice signal by normalizing the volume and removing noise from the first voice data and the second voice data. The voice activity detection module can accurately identify and mark the voice activity area, reduce the interference of the non-voice part, and provide accurate time marking for subsequent voice segmentation and analysis.

[0152] In one embodiment, the above S60 includes:

[0153] S601, inputting the multiple new first speech sequences and the multiple new second speech sequences into a voiceprint portrait generation model;

[0154] S602, processing each speech sequence according to the voiceprint portrait generation model to generate a corresponding first voiceprint portrait and a second voiceprint portrait;

[0155] S603, optimizing the first voiceprint portrait and the second voiceprint portrait by using noise suppression and image smoothing technology to obtain optimized first voiceprint portrait and second voiceprint portrait.

[0156] In this embodiment, a new speech sequence is taken as input and passed to a pre-trained voiceprint portrait generation model. The voiceprint portrait generation model can convert audio signals into voiceprint portraits in the form of images. These portraits usually show the spectral characteristics of the speech, such as energy distribution, frequency changes, etc. Through this visual expression, the characteristics of different speech can be analyzed and compared more clearly. Each new speech sequence (audio signal) is input into the voiceprint portrait generation model. The model can be a short-time Fourier transform (STFT) model based on traditional audio analysis, or it can be a deep learning model (such as a convolutional neural network, CNN) for generating spectrograms or other voiceprint portraits. The voiceprint portrait generation model needs to be configured according to the application scenario, such as adjusting parameters such as frame length, frame shift, and frequency band division to optimize the clarity and resolution of the voiceprint portrait.

[0157] The voiceprint portrait generation model processes each input speech sequence, extracts key speech features, and maps these features into the image space to generate the first voiceprint portrait and the second voiceprint portrait. The voiceprint portrait is an intuitive expression of speech features, which facilitates subsequent voiceprint comparison and analysis. The audio signal is converted into a two-dimensional feature map, such as a spectrogram, Mel-frequency cepstral coefficient (MFCC) map, etc. STFT can be used to generate a spectrogram to show the frequency distribution of the speech signal over time; or a deep learning model can be used to extract more complex voiceprint features. The features are converted into image format and the generated voiceprint portrait is stored for further optimization and analysis.

[0158] After generating the preliminary voiceprint image, perform image optimization processing. Noise suppression technology is used to remove random noise or artifacts that may exist in the image, and image smoothing technology is used to reduce burrs and abruptness on the edges of the image and enhance the overall visual effect of the image. The optimized voiceprint image is clearer and more stable, which helps to improve the accuracy of voiceprint matching. Use image processing methods (such as Gaussian filtering or median filtering) to denoise the voiceprint image and eliminate random noise in the image. Apply image smoothing technology (such as bilateral filtering or convolution operation) to smooth the edges of the image, making the image features more coherent and natural, and improving the effect of subsequent analysis.

[0159] This embodiment generates a voiceprint image based on multiple new first speech sequences and second speech sequences, and performs noise suppression and image smoothing optimization, which can significantly improve the clarity and stability of the voiceprint image. The optimized voiceprint image can more accurately reflect the voice characteristics of the speaker, reduce the impact of environmental noise and image artifacts on voiceprint analysis, and thus improve the accuracy and robustness of voiceprint comparison.

[0160] In one embodiment, the above S70 includes:

[0161] S701, identifying factors affecting voiceprint features from the first voiceprint portrait and the second voiceprint portrait respectively;

[0162] S702, determining the influence degree of the influencing factors on the voiceprint features based on historical voice data and historical statistical analysis;

[0163] S703, establishing a mapping relationship between the influencing factors and the voiceprint features, and setting corresponding weight parameters for the influencing factors according to the degree of influence on the voiceprint features;

[0164] S704, using corresponding weight parameters to perform weighted processing on the voiceprint features corresponding to the influencing factors in the first voiceprint portrait and the second voiceprint portrait.

[0165] In this embodiment, the features in the voiceprint image are analyzed to identify external or internal factors that may affect the voiceprint features, such as age, gender, emotional state, background noise, etc. These factors can affect the expression of voiceprint features and need to be identified and adjusted in the voiceprint image to improve the accuracy of voiceprint comparison. Use image processing algorithms or feature analysis models to identify and mark areas in the voiceprint image that are related to different influencing factors. For example, use frequency distribution patterns to identify the impact of high-frequency noise, or use machine learning models to automatically classify the source of voice features. Assign labels to each identified influencing factor to facilitate subsequent weight adjustment.

[0166] Use historical voice data for statistical analysis to quantify the specific degree of influence of each influencing factor on the voiceprint features. This includes evaluating the impact of different factors on the voiceprint portrait and assigning a weight value to each factor to indicate its importance and influence. By analyzing a large amount of historical voice data, the correlation between different influencing factors and changes in voiceprint features is calculated. Statistical analysis tools (such as Python's pandas library) and machine learning models (such as linear regression) can be used to determine the degree of influence of each influencing factor. Based on the results of the correlation analysis, the weight value of each influencing factor is generated and recorded in the data table or model parameters.

[0167] Build a mapping relationship to associate each influencing factor with its related voiceprint feature. Assign corresponding weight parameters to each influencing factor so that it can be adjusted in the voiceprint portrait. This step helps to build a mathematical model that can accurately describe and correct the deviation of voiceprint features. Use a data structure (such as a dictionary or matrix) to store the mapping relationship between influencing factors and voiceprint features, and define corresponding weight parameters for each factor. This relationship can be generated through rule mapping or automated modeling. Assign appropriate weight parameters to each influencing factor to ensure that the weight value accurately reflects the impact of the factor on the voiceprint feature.

[0168] Apply the determined weight parameters to weight the features in the voiceprint portrait. This process is to correct or reduce the impact of non-essential differences (such as noise and emotional changes) on the voiceprint features, so that the voiceprint portrait can better represent the essential characteristics of the speaker. Use matrix operations or convolution operations to apply weight parameters to relevant feature areas in the voiceprint portrait to adjust their intensity or form of expression. For example, the eigenvalues ​​can be weighted by matrix multiplication to make the adjusted features closer to the real voiceprint features of the speaker. The weighted voiceprint portrait is stored as an optimized feature map, ready for subsequent similarity analysis or comparison.

[0169] Example: Customer A calls the bank customer service hotline and hopes to make a high-value transfer. Customer A is in a noisy airport environment and speaks faster due to nervousness. The system needs to accurately identify Customer A's voiceprint to ensure the security of identity verification.

[0170] The system collects the voice data of customer A and normalizes its volume to ensure that the audio signal is at a consistent volume level for easy analysis. The system uses a noise suppression algorithm to remove background noise, such as airport broadcasts and crowd noise.

[0171] The system analyzes the generated preliminary voiceprint image and identifies factors that affect the voiceprint characteristics, including:

[0172] Background noise: The ambient noise of the airport is identified as the main interference factor, and the system assigns a weight parameter of 0.4 to it.

[0173] Emotional fluctuations: Customer A’s speaking speed and tone change due to nervousness. The system recognizes the impact of emotional fluctuations and assigns a weight parameter of 0.3.

[0174] Speech speed change: Since customer A's speech speed has increased significantly, speech speed change has become an influencing factor, and the system assigns it a weight parameter of 0.2.

[0175] Device differences: The customer uses a mobile phone to make a call, and the impact of the recording device on the sound quality is identified as a secondary factor, and the system assigns a weight parameter of 0.1.

[0176] Therefore, the areas affected by background noise in the voiceprint image are downgraded by 40% to reduce the interference of noise on the voiceprint features. The voiceprint features affected by emotional fluctuations are adjusted by 30% to compensate for the feature offset caused by emotional changes. The features of speech speed changes are smoothed by 20% to make the voiceprint image more reflective of the features at normal speech speed. The slight feature offset caused by equipment differences is corrected by 10% to ensure that the voiceprint image is consistent with the image under standard equipment recording conditions.

[0177] The system uses the above-mentioned weight parameters to perform weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively. The optimized voiceprint portrait can more accurately reflect the essential voiceprint characteristics of customer A, rather than the characteristics affected by external factors.

[0178] The optimized voiceprint image is analyzed for similarity with the standard voiceprint image of customer A stored in the bank database. If the similarity score is higher than 0.85 (the security threshold set by the system), the system confirms that customer A's identity verification is successful and allows high-value transfer operations.

[0179] If the similarity score is below the threshold, the system will require customer A to take additional verification measures, such as answering security questions or completing verification through multi-factor authentication.

[0180] This embodiment can effectively reduce the impact of non-essential differences (such as background noise and emotional fluctuations) on voiceprint features through weighted processing based on weight parameters, thereby improving the accuracy of identity verification. Even if the customer is in a complex and noisy environment, the system can still reliably identify his or her identity, ensuring the security of financial transactions, while improving customer experience and reducing misjudgments or verification failures caused by external interference.

[0181] In one embodiment, the above S80 includes:

[0182] S801, inputting the first voiceprint portrait and the second voiceprint portrait into a similarity analysis module;

[0183] S802, extracting a first feature vector and a second feature vector for similarity analysis from the first voiceprint portrait and the second voiceprint portrait respectively according to the similarity analysis module;

[0184] S803: Analyze the first feature vector and the second feature vector using a similarity measurement method to generate a similarity score.

[0185] In this embodiment, the weighted first voiceprint portrait and the second voiceprint portrait are input into a dedicated similarity analysis module. The module analyzes the similarity between the two portraits by extracting feature vectors from the image. The similarity analysis module can use image processing and machine learning algorithms to complete feature extraction and similarity calculation. The optimized voiceprint portrait is converted into a format suitable for input into the similarity analysis module, such as a matrix form or a tensor format, so as to extract feature vectors. A deep learning model (such as a convolutional neural network, CNN) or a preprocessing algorithm can be used specifically for extracting features from voiceprint portraits for comparison.

[0186] The similarity analysis module extracts feature vectors by processing voiceprint portraits. These feature vectors are high-dimensional representations that describe the main features of voiceprint portraits and capture key information related to voiceprint features, such as frequency distribution, energy concentration areas, etc. Feature vectors can be extracted using pre-trained models or custom models in deep learning frameworks (such as TensorFlow or PyTorch). The extracted feature vectors are usually high-dimensional vectors of fixed length, which facilitates subsequent similarity measurement. Use batch processing technology or multi-threaded processing to improve feature extraction efficiency, especially when processing large-scale voiceprint portrait data.

[0187] The similarity between two sets of feature vectors is calculated using a similarity measurement method to generate a similarity score. Commonly used similarity measurement methods include cosine similarity, Euclidean distance, inner product, etc. The similarity score is used to quantify the similarity between two voiceprint images. The higher the score, the more similar the two are.

[0188] This embodiment can effectively quantify the similarity between different voice data and ensure the accuracy of identity authentication by inputting the weighted voiceprint portrait into the similarity analysis module and extracting the feature vector for similarity calculation. The use of a suitable similarity measurement method can better capture the key features in the voiceprint portrait, especially in a complex audio environment, showing high robustness and reliability. The generated similarity score provides a reliable basis for subsequent identity authentication and decision-making, ensuring the security of financial transactions.

[0189] In one embodiment, a voiceprint recognition device based on cluster analysis is provided, and the voiceprint recognition device based on cluster analysis corresponds one-to-one to the voiceprint recognition method based on cluster analysis in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of a voiceprint recognition device based on cluster analysis of the present invention. Voice data acquisition module 10, voice activity detection module 20, voice division module 30, cluster analysis module 40, voice sequence generation module 50, voiceprint image generation module 60, weighted processing module 70, similarity analysis module 80 and voiceprint recognition module 90. The functional modules are described in detail as follows:

[0190] A voice data acquisition module 10, used to acquire first voice data and second voice data to be detected;

[0191] A voice activity detection module 20, configured to perform voice activity detection on the first voice data and the second voice data, and identify and mark voice activity areas in the first voice data and the second voice data;

[0192] A speech segmentation module 30, configured to segment the first speech data and the second speech data into a plurality of first speech segments and a plurality of second speech segments respectively according to the speech activity area;

[0193] A cluster analysis module 40, configured to perform cluster analysis on the first speech segment and the second speech segment respectively, to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data;

[0194] A speech sequence generation module 50 is used to splice first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and to splice second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0195] A voiceprint portrait generating module 60 is used to generate corresponding first voiceprint portraits and second voiceprint portraits based on the multiple new first speech sequences and the multiple new second speech sequences;

[0196] A weighted processing module 70, configured to perform weighted processing on the first voiceprint image and the second voiceprint image according to weighted parameters;

[0197] A similarity analysis module 80, configured to analyze the similarity between the first voiceprint image and the second voiceprint image after weighted processing;

[0198] The voiceprint recognition module 90 is used to identify whether the first voice data and the second voice data belong to the same speaker according to the similarity.

[0199] In one embodiment, the cluster analysis module 40 is specifically used for:

[0200] Extracting a first speech feature of each first speech segment and a second speech feature of each second speech segment;

[0201] Performing standardization processing on the first speech feature and the second speech feature respectively;

[0202] A clustering module is used to perform cluster analysis on the first speech feature after the standardization processing and the second speech feature after the standardization processing, respectively, to obtain multiple cluster categories of the first speech data and multiple cluster categories of the second speech data.

[0203] In one embodiment, the cluster analysis module 40 is specifically used for:

[0204] Determine the number of clustering categories according to preset parameters;

[0205] Randomly select an initial cluster center for each cluster category;

[0206] Analyzing the distance between each first speech feature and each second speech feature and each initial cluster center based on a distance measurement method, and assigning each first speech feature and each second speech feature to the initial cluster center with the closest distance;

[0207] Analyzing the average values ​​of all first speech features and all second speech features in each cluster category respectively, and using the average values ​​to update the cluster center corresponding to the cluster category;

[0208] The allocation operation and the update operation are repeated, and when the change ranges of all cluster centers are less than a preset threshold, the cluster analysis is terminated to obtain a plurality of cluster categories of the first speech data and the second speech data respectively.

[0209] In one embodiment, the voice activity detection module 20 is specifically configured to:

[0210] Performing volume normalization processing on the first voice data and the second voice data, and removing background noise;

[0211] Extracting key speech features from the processed first speech data and the second speech data respectively;

[0212] Using a voice activity detection module, respectively identifying voice activity areas in the first voice data and the second voice data according to the key voice features;

[0213] The start and end times of the corresponding voice activity areas are marked in the first voice data and the second voice data, and an identifier of the voice activity area is generated.

[0214] In one embodiment, the voiceprint image generation module 60 is specifically used to:

[0215] Inputting the multiple new first speech sequences and the multiple new second speech sequences into a voiceprint portrait generation model;

[0216] Process each speech sequence according to the voiceprint portrait generation model to generate a corresponding first voiceprint portrait and a second voiceprint portrait;

[0217] The first voiceprint portrait and the second voiceprint portrait are optimized by using noise suppression and image smoothing technology to obtain optimized first voiceprint portrait and second voiceprint portrait.

[0218] In one embodiment, the weighted processing module 70 is specifically configured to:

[0219] Identifying factors affecting voiceprint features from the first voiceprint portrait and the second voiceprint portrait respectively;

[0220] Determine the degree of influence of influencing factors on voiceprint features based on historical voice data and historical statistical analysis;

[0221] Establishing a mapping relationship between influencing factors and voiceprint features, and setting corresponding weight parameters for influencing factors according to the degree of influence on the voiceprint features;

[0222] The voiceprint features corresponding to the influencing factors in the first voiceprint portrait and the second voiceprint portrait are weighted using corresponding weight parameters.

[0223] In one embodiment, the similarity analysis module 80 is specifically used for:

[0224] Inputting the first voiceprint image and the second voiceprint image into a similarity analysis module;

[0225] Extracting a first feature vector and a second feature vector for similarity analysis from the first voiceprint portrait and the second voiceprint portrait respectively according to the similarity analysis module;

[0226] The first feature vector and the second feature vector are analyzed using a similarity measurement method to generate a similarity score.

[0227] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a voiceprint recognition method based on clustering analysis on the server side.

[0228] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a user-side voiceprint recognition method based on cluster analysis.

[0229] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0230] Acquire first voice data and second voice data to be detected;

[0231] Performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data;

[0232] dividing the first voice data and the second voice data into a plurality of first voice segments and a plurality of second voice segments respectively according to the voice activity area;

[0233] Performing cluster analysis on the first speech segment and the second speech segment respectively to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data;

[0234] splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0235] Based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generate corresponding first voiceprint portraits and second voiceprint portraits;

[0236] Performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to weighted parameters;

[0237] Analyzing the similarity between the weighted first voiceprint image and the second voiceprint image;

[0238] It is identified based on the similarity whether the first voice data and the second voice data belong to the same speaker.

[0239] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0240] Acquire first voice data and second voice data to be detected;

[0241] Performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data;

[0242] dividing the first voice data and the second voice data into a plurality of first voice segments and a plurality of second voice segments respectively according to the voice activity area;

[0243] Performing cluster analysis on the first speech segment and the second speech segment respectively to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data;

[0244] splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data;

[0245] Based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generate corresponding first voiceprint portraits and second voiceprint portraits;

[0246] Performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to weighted parameters;

[0247] Analyzing the similarity between the weighted first voiceprint image and the second voiceprint image;

[0248] It is identified based on the similarity whether the first voice data and the second voice data belong to the same speaker.

[0249] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0250] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0251] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0252] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A voiceprint recognition method based on cluster analysis, characterized in that: The following steps are involved: Acquire first voice data and second voice data to be detected; Performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data; dividing the first voice data and the second voice data into a plurality of first voice segments and a plurality of second voice segments respectively according to the voice activity area; Performing cluster analysis on the first speech segment and the second speech segment respectively to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data; splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data; Based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generate corresponding first voiceprint portraits and second voiceprint portraits; Performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to weighted parameters; Analyzing the similarity between the weighted first voiceprint image and the second voiceprint image; It is identified based on the similarity whether the first voice data and the second voice data belong to the same speaker.

2. The voiceprint recognition method based on cluster analysis according to claim 1, characterized in that: Performing cluster analysis on the first voice segment and the second voice segment respectively to obtain multiple cluster categories of the first voice data and multiple cluster categories of the second voice data, including: Extracting a first speech feature of each first speech segment and a second speech feature of each second speech segment; Performing standardization processing on the first speech feature and the second speech feature respectively; A clustering module is used to perform cluster analysis on the first speech feature after the standardization processing and the second speech feature after the standardization processing, respectively, to obtain multiple cluster categories of the first speech data and multiple cluster categories of the second speech data.

3. The voiceprint recognition method based on cluster analysis according to claim 2, characterized in that: Using a clustering module to perform cluster analysis on the first speech feature after the standardization processing and the second speech feature after the standardization processing, respectively, to obtain multiple cluster categories of the first speech data and multiple cluster categories of the second speech data, including: Determine the number of clustering categories according to preset parameters; Randomly select an initial cluster center for each cluster category; Analyzing the distance between each first speech feature and each second speech feature and each initial cluster center based on a distance measurement method, and assigning each first speech feature and each second speech feature to the initial cluster center with the closest distance; Analyzing the average values ​​of all first speech features and all second speech features in each cluster category respectively, and using the average values ​​to update the cluster center corresponding to the cluster category; The allocation operation and the update operation are repeated, and when the change ranges of all cluster centers are less than a preset threshold, the cluster analysis is terminated to obtain a plurality of cluster categories of the first speech data and the second speech data respectively.

4. The voiceprint recognition method based on cluster analysis according to claim 1, characterized in that: Performing voice activity detection on the first voice data and the second voice data, identifying and marking voice activity areas in the first voice data and the second voice data, includes: Performing volume normalization processing on the first voice data and the second voice data, and removing background noise; Extracting key speech features from the processed first speech data and the second speech data respectively; Using a voice activity detection module, respectively identifying voice activity areas in the first voice data and the second voice data according to the key voice features; The start and end times of the corresponding voice activity areas are marked in the first voice data and the second voice data, and an identifier of the voice activity area is generated.

5. The voiceprint recognition method based on cluster analysis according to claim 1, characterized in that: Based on the multiple new first voice sequences and the multiple new second voice sequences, respectively generating corresponding first voiceprint portraits and second voiceprint portraits, including: Inputting the multiple new first speech sequences and the multiple new second speech sequences into a voiceprint portrait generation model; Process each speech sequence according to the voiceprint portrait generation model to generate a corresponding first voiceprint portrait and a second voiceprint portrait; The first voiceprint portrait and the second voiceprint portrait are optimized by using noise suppression and image smoothing technology to obtain optimized first voiceprint portrait and second voiceprint portrait.

6. The voiceprint recognition method based on cluster analysis as claimed in claim 1, characterized in that: The first voiceprint image and the second voiceprint image are weighted according to the weighting parameters, including: Identifying factors affecting voiceprint features from the first voiceprint portrait and the second voiceprint portrait respectively; Determine the degree of influence of influencing factors on voiceprint features based on historical voice data and historical statistical analysis; Establishing a mapping relationship between influencing factors and voiceprint features, and setting corresponding weight parameters for influencing factors according to the degree of influence on the voiceprint features; The voiceprint features corresponding to the influencing factors in the first voiceprint portrait and the second voiceprint portrait are weighted using corresponding weight parameters.

7. The voiceprint recognition method based on cluster analysis according to claim 1, characterized in that: Analyze the similarity between the weighted first voiceprint image and the second voiceprint image, including: Inputting the first voiceprint image and the second voiceprint image into a similarity analysis module; Extracting a first feature vector and a second feature vector for similarity analysis from the first voiceprint portrait and the second voiceprint portrait respectively according to the similarity analysis module; The first feature vector and the second feature vector are analyzed using a similarity measurement method to generate a similarity score.

8. A voiceprint recognition device based on cluster analysis, characterized in that: The voiceprint recognition device based on cluster analysis includes: A voice data acquisition module, used to acquire first voice data and second voice data to be detected; A voice activity detection module, configured to perform voice activity detection on the first voice data and the second voice data, and identify and mark voice activity areas in the first voice data and the second voice data; A speech segmentation module, used for segmenting the first speech data and the second speech data into a plurality of first speech segments and a plurality of second speech segments respectively according to the speech activity area; A cluster analysis module, used to perform cluster analysis on the first speech segment and the second speech segment respectively, to obtain a plurality of cluster categories of the first speech data and a plurality of cluster categories of the second speech data; A speech sequence generation module, used for splicing first speech segments belonging to the same cluster category in the first speech data to generate multiple new first speech sequences of the first speech data, and splicing second speech segments belonging to the same cluster category in the second speech data to generate multiple new second speech sequences of the second speech data; A voiceprint portrait generation module, used to generate corresponding first voiceprint portraits and second voiceprint portraits based on the multiple new first voice sequences and the multiple new second voice sequences; A weighted processing module, used for performing weighted processing on the first voiceprint portrait and the second voiceprint portrait respectively according to a weighted parameter; A similarity analysis module, used to analyze the similarity between the first voiceprint image and the second voiceprint image after weighted processing; The voiceprint recognition module is used to identify whether the first voice data and the second voice data belong to the same speaker according to the similarity.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a voiceprint recognition program based on clustering analysis stored in the memory and executable on the processor. When the voiceprint recognition program based on clustering analysis is executed by the processor, the steps of the voiceprint recognition method based on clustering analysis as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a voiceprint recognition program based on cluster analysis, and when the voiceprint recognition program based on cluster analysis is executed by the processor, the steps of the voiceprint recognition method based on cluster analysis as described in any one of claims 1 to 7 are implemented.

Images