Voice recognition method and device used in dialogue, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a voice recognition method and device used in dialogues, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps of taking original speaker voice in a conversation, performing voiceprint feature extraction on the original speaker voice to obtain original voiceprint features, clustering sample voiceprint features of a pre-constructed voiceprint feature database to obtain a voiceprint feature cluster, determining a reference voiceprint feature of a cluster center according to the sample voiceprint features of the voiceprint feature cluster, and performing voiceprint feature extraction on the reference voiceprint feature of the cluster center according to the reference voiceprint feature of the cluster center. The similarity between the original voiceprint feature and the reference voiceprint feature is calculated, the original speaker voice is subjected to voice recognition according to the similarity, the voice category is obtained, the voice category is used for indicating that the original speaker voice is abnormal voice or normal voice, and the voice can be recognized to resist voice attacks of the abnormal voice.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a method and device for speech recognition in a conversation, an electronic device, and a storage medium. Background Art

[0002] In the related art, financial institutions such as banks and insurance companies use intelligent voice assistants to remotely handle various services for users. For example, users can send voice messages to intelligent voice assistants to purchase financial products to handle the purchase of financial products. However, the voice recognition ability of intelligent voice assistants is low, and they cannot distinguish between normal and abnormal voices. When responding to abnormal voices, they are vulnerable to voice attacks from abnormal voices. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a method and device, an electronic device and a storage medium for speech recognition in a conversation, aiming to recognize speech to distinguish whether the speech is normal speech or abnormal speech.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for speech recognition in a conversation, the method comprising:

[0005] Get the original speaker voice in the conversation;

[0006] Extracting voiceprint features from the original speaker's speech to obtain original voiceprint features;

[0007] Clustering sample voiceprint features of a pre-constructed voiceprint feature database to obtain a voiceprint feature cluster; the voiceprint feature cluster has a cluster center;

[0008] Determining a reference voiceprint feature at the cluster center according to the sample voiceprint feature of the voiceprint feature cluster;

[0009] Calculating the similarity between the original voiceprint feature and the reference voiceprint feature;

[0010] Speech recognition is performed on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

[0011] In some embodiments, the conversation includes a speaking object, and the extracting voiceprint features from the original speaker's speech to obtain the original voiceprint features includes:

[0012] Performing voice conversion on the original speaker's voice to obtain a speaker text;

[0013] Performing word recognition on the speaker text to obtain target spoken words;

[0014] Performing speech segmentation on the original speaker's speech according to the target speech words to obtain speech segments of the speaker of the speaking object;

[0015] Extracting voiceprint features from the speaker's speech segment to obtain candidate voiceprint features;

[0016] The candidate voiceprint features are fused to obtain the original voiceprint features.

[0017] In some embodiments, extracting voiceprint features from the speaker's speech segment to obtain candidate voiceprint features includes:

[0018] Performing speech denoising on the speaker's speech segment to obtain a candidate speech segment;

[0019] Extracting spectral features of the candidate speech segment to obtain Mel-spectrogram features;

[0020] Performing a first voiceprint feature extraction on the Mel frequency spectrum feature to obtain a preliminary voiceprint feature;

[0021] A second voiceprint feature is extracted from the preliminary voiceprint feature to obtain the candidate voiceprint feature.

[0022] In some embodiments, the step of fusing the candidate voiceprint features to obtain the original voiceprint features includes:

[0023] Calculating feature weights of the candidate voiceprint features;

[0024] The candidate voiceprint features are feature-weighted according to the feature weights to obtain the original voiceprint features.

[0025] In some embodiments, calculating the feature weight of the candidate voiceprint feature includes:

[0026] Obtaining an average value of the candidate voiceprint features to obtain an average voiceprint feature;

[0027] Obtaining the standard deviation of the candidate voiceprint feature according to the average voiceprint feature to obtain a standard deviation voiceprint feature;

[0028] The feature weight is determined according to the average voiceprint feature and the standard deviation voiceprint feature.

[0029] In some embodiments, the step of fusing the candidate voiceprint features to obtain the original voiceprint features includes:

[0030] Calculating a feature score of the candidate voiceprint feature;

[0031] Screening the candidate voiceprint features according to the feature scores to obtain a benchmark voiceprint feature;

[0032] The reference voiceprint features are spliced ​​to obtain the original voiceprint features.

[0033] In some embodiments, before clustering the sample voiceprint features of the pre-built voiceprint feature database to obtain voiceprint feature clusters, the method further includes:

[0034] Obtain sample speaker speech;

[0035] Extracting features of the sample speaker's speech to obtain sample voiceprint features;

[0036] The voiceprint feature database is constructed according to the sample voiceprint features.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for speech recognition in a conversation, the device comprising:

[0038] An acquisition module, used to acquire the original speaker's voice in the conversation;

[0039] A feature extraction module, used to extract voiceprint features from the original speaker's speech to obtain original voiceprint features;

[0040] A clustering module, used for clustering sample voiceprint features of a pre-built voiceprint feature database to obtain voiceprint feature clusters; the voiceprint feature clusters have a cluster center;

[0041] A determination module, configured to determine a reference voiceprint feature of the cluster center according to the sample voiceprint feature of the voiceprint feature cluster;

[0042] A calculation module, used to calculate the similarity between the original voiceprint feature and the reference voiceprint feature;

[0043] The speech recognition module is used to perform speech recognition on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

[0044] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0046] The method for speech recognition in a conversation, the device for speech recognition in a conversation, the electronic device and the computer-readable storage medium proposed in the embodiments of the present application obtain the original speaker's speech in the conversation, extract the voiceprint feature of the original speaker's speech, so as to extract the distinguishing information in the original speaker's speech that plays a key role in speech recognition, and obtain the original voiceprint feature, so that the original speaker's speech can be distinguished as normal speech or abnormal speech according to the similarity between the original voiceprint feature and each sample voiceprint feature in the voiceprint feature database. In order to improve the efficiency of speech recognition and avoid the calculation of the similarity between the original voiceprint feature and each sample voiceprint feature in the voiceprint feature database, the sample voiceprint features in the voiceprint feature database are clustered to simplify a large number of sample voiceprint features into a small number of clusters to obtain voiceprint feature clusters. The features at the center of the cluster can be used as representatives of all sample voiceprint features in the voiceprint feature cluster. In order to simplify the feature representation and reduce the computational complexity, the reference voiceprint features at the center of the cluster are determined according to the sample voiceprint features of the voiceprint feature cluster, so as to replace the features of the entire voiceprint feature cluster with the features at the center of the cluster. The similarity between the original voiceprint features and the reference voiceprint features is calculated, and speech recognition is performed on the original speaker's voice based on the similarity to obtain a speech category. The speech category is used to indicate whether the original speaker's voice is an abnormal voice or a normal voice, so that the original speaker's voice can be distinguished as a normal voice or an abnormal voice according to the speech category to resist voice attacks by abnormal voices. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for speech recognition in a conversation provided by an embodiment of the present application;

[0048] Figure 2 yes Figure 1 Flow chart of step S120 in FIG.

[0049] Figure 3 yes Figure 2 Flow chart of step S240 in FIG.

[0050] Figure 4 yes Figure 2 Flow chart of step S250 in FIG.

[0051] Figure 5 yes Figure 4 Flow chart of step S410 in FIG.

[0052] Figure 6 yes Figure 2 Another flow chart of step S250 in FIG.

[0053] Figure 7 is another flow chart of a method for speech recognition in a conversation provided by an embodiment of the present application;

[0054] Figure 8is a schematic diagram of the structure of a speech recognition device for conversation provided by an embodiment of the present application;

[0055] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] In the related art, financial institutions such as banks and insurance companies use intelligent voice assistants to remotely handle various services for users. For example, users can send voice messages to intelligent voice assistants to purchase financial products to handle the purchase of financial products. However, the voice recognition ability of intelligent voice assistants is low, and they cannot distinguish between normal and abnormal voices. When responding to abnormal voices, they are vulnerable to voice attacks from abnormal voices.

[0060] Based on this, the embodiments of the present application provide a method for speech recognition in conversation, a device for speech recognition in conversation, an electronic device and a computer-readable storage medium, which are intended to perform speech recognition and distinguish between normal speech and abnormal speech.

[0061] The method for speech recognition in conversation, the device for speech recognition in conversation, the electronic device and the computer-readable storage medium provided in the embodiments of the present application are specifically explained through the following embodiments. First, the method for speech recognition in conversation in the embodiments of the present application is described.

[0062] The method for speech recognition in conversation provided in the embodiment of the present application relates to the field of financial technology technology. The method for speech recognition in conversation provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for speech recognition in conversation, etc., but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0064] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0065] Figure 1 is an optional flowchart of a method for speech recognition in a conversation provided by an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S160.

[0066] Step S110, obtaining the original speaker's voice in the conversation;

[0067] Step S120, extracting voiceprint features from the original speaker's speech to obtain original voiceprint features;

[0068] Step S130, clustering the sample voiceprint features of the pre-constructed voiceprint feature database to obtain a voiceprint feature cluster; the voiceprint feature cluster has a cluster center;

[0069] Step S140, determining a reference voiceprint feature at the cluster center according to the sample voiceprint features of the voiceprint feature cluster;

[0070] Step S150, calculating the similarity between the original voiceprint feature and the reference voiceprint feature;

[0071] Step S160, performing speech recognition on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

[0072] In the steps S110 to S160 shown in the embodiment of the present application, the original voiceprint features of the original speaker's speech are extracted, the similarity between the original voiceprint features and the voiceprint features in the voiceprint feature database is calculated, and the original speaker's speech is distinguished as normal speech or abnormal speech according to the similarity, so as to resist the voice attack of abnormal speech. Furthermore, the sample voiceprint features in the voiceprint feature database are clustered, a large number of sample voiceprint features are clustered into voiceprint feature clusters, and the voiceprint features of the entire cluster are replaced by the voiceprint features at the center of the voiceprint feature cluster, which greatly improves the efficiency of similarity calculation, thereby improving the efficiency of speech recognition.

[0073] In step S110 of some embodiments, in order to better meet the needs of users, financial institutions such as banks and insurance companies use intelligent voice assistants to handle various financial services such as financial product purchases and insurance claims. When a user needs to handle financial services provided by a financial institution, such as the purchase of financial products and insurance claims, he or she can send voice instructions or voice questions to the voice assistant through a terminal device such as a smart phone, smart speaker, computer, etc. After the voice assistant performs business handling actions such as purchase actions and claims actions according to the voice instructions or voice questions, it will feedback the results of the business handling to the user through voice. The terminal device and the voice assistant handle business through voice interaction, and a dialogue will occur between the two parties during the process of handling business. The dialogue includes a speaking object, and the speaking object includes a first speaking object and a second speaking object. The first speaking object is a user who needs to handle business, and the second speaking object is an intelligent voice assistant used to handle business.

[0074] When using intelligent voice assistants to handle business, users do not need to go to the counter to handle the business for on-site confirmation, which makes it difficult for financial institutions to confirm the authenticity of the user's identity. If synthetic voice and voice changing technology are used to imitate real voices to make malicious complaints, insurance fraud, and malicious insurance cancellations, the intelligent voice assistant cannot distinguish whether the voice is normal or abnormal, and cannot guarantee the security of business handling. Normal voice is the real voice of the speaker, and abnormal voice is the voice that simulates the speaker and maliciously handles business. In order to ensure the authenticity of the user's identity and the security of business handling, the voice of the second speaker is removed from the conversation, and the voice of the first speaker is obtained. The voice of the first speaker is used as the original speaker's voice, and voice recognition is performed based on the original speaker's voice.

[0075] Voiceprints are relatively stable. The voiceprint changes of the same speaker remain relatively stable, which can be used as a representation and identification of the speaker. It is a voice feature that distinguishes the current speaker from other speakers. In order to distinguish the current speaker from the simulated speaker and ensure the authenticity of the current speaker's identity, voiceprint feature extraction is performed on the original speaker's voice to extract effective, stable and reliable features that uniquely represent the speaker's identity, and obtain the original voiceprint features. The following describes the process of voiceprint feature extraction.

[0076] See also Figure 2 In some embodiments, step S120 may include but is not limited to steps S210 to S250:

[0077] Step S210, performing voice conversion on the original speaker's voice to obtain the speaker's text;

[0078] Step S220, performing word recognition on the speaker text to obtain target speech words;

[0079] Step S230, performing speech segmentation on the original speaker's speech according to the target speech words to obtain the speaker's speech segments of the speaking object;

[0080] Step S240, extracting voiceprint features from the speaker's speech segment to obtain candidate voiceprint features;

[0081] Step S250, performing feature fusion on the candidate voiceprint features to obtain the original voiceprint features.

[0082] In step S210 of some embodiments, the original speaker's speech is divided into a plurality of speech frames according to a preset time interval, and for each speech frame, the speech features in the speech frame are extracted, and the one-dimensional signal is converted into a multi-dimensional vector. The speech features may be acoustic features commonly used in the field of speech recognition, such as Mel-scale Frequency Cepstral Coefficients (MFCCs), zero crossing rate, Linear Prediction Coefficients (LPCs), energy, fundamental frequency, etc. The speech features are converted into a recognizable word sequence through an acoustic model, and the word sequence is converted into a sentence through a language model to obtain a speaker's text. Among them, the acoustic model is used to capture the relationship between the physical properties of the sound and the vocabulary in the language, and the acoustic model may be a recurrent neural network, a long short-term memory network, a transformer network, etc. The language model is used to capture the relationship between vocabulary and sentence structure, and the language model may be a hidden Markov model, a long short-term memory network, etc. The original speaker's speech may also be converted into speech through a tool with a speech-to-text function to obtain a speaker's text.

[0083] In step S220 of some embodiments, daily chatting or non-transactional voices do not trigger the business processing flow of the intelligent voice assistant. In order to improve the security of business processing such as stock trading, it is only necessary to perform abnormal voice recognition on voices containing specific words. In order to determine whether the original speaker's voice contains specific words, the speaker's text can be segmented using a word segmentation tool, and the speaker's text can be divided into words or phrases to obtain target spoken words. The word segmentation tool can be jieba, HanLP, etc. Alternatively, the speaker's text is semantically analyzed through a word embedding model to obtain contextual semantic features. Contextual semantic features include the association relationship between each word in the speaker's text and the meaning of the word. The contextual semantic features are classified by intent through an intent recognition model to obtain an intent category, and the intent category is used as the target spoken word. The intent category is used to indicate the speaking intention of the first speaker, such as surrender. The word embedding model can be a word2vec model, a glove model, etc., and the intent recognition model can be a support vector machine, a random forest, etc.

[0084] In step S230 of some embodiments, the first speaking object includes at least one sub-object. In a single-user financial transaction scenario, the first speaking object only involves one sub-object. If the target speaking word is a preset specific word, the voiceprint feature of the original speaker can be directly extracted to obtain the original voiceprint feature. The specific word is a word that can trigger a business operation or a financial transaction operation, such as insurance cancellation, purchase of insurance products, insurance claims, loans, etc.

[0085] In a multi-user financial transaction scenario, the first speaker may involve multiple sub-objects. Intelligent voice assistants usually cannot know the identity tags of each sub-object. In the absence of speaker identity tags, in order to distinguish the speech of different sub-objects, if the target speech word is a preset specific word, the original speaker speech is divided into multiple speech frames, and the speech features of each speech frame are extracted. Determine the clustering algorithm, cluster the speech features according to the clustering algorithm, and obtain clusters. The speech segments corresponding to the speech features in the same cluster are spliced ​​to obtain the speaker speech segments of the cluster, and each cluster corresponds to a sub-object.

[0086] If the clustering algorithm is the K-means algorithm, K speech features are randomly selected as the K initialized cluster centers, where K is an integer greater than 1. For each speech feature, the Euclidean distance from the speech feature to each cluster center is calculated respectively, and the speech feature is classified into the cluster where the cluster center with the smallest Euclidean distance is located. For each cluster, the cluster center is updated according to the average value of all speech features in the cluster. Repeat the classification and cluster center update steps until the iteration stop condition is met. Take a cluster as a sub-object, and perform feature splicing on all speech features in the same cluster to obtain the speaker's speech segment corresponding to the sub-object of the cluster. A cluster identifier can be set for each cluster to distinguish different clusters. The iteration stop condition can be that the change of the cluster center between two adjacent iterations is small, the number of iterations reaches the maximum iteration limit, or the change of the speech features assigned in the cluster is small.

[0087] In step S240 of some embodiments, in a multi-user financial transaction scenario, it is necessary to verify the object identity of each sub-object to avoid reducing the security of financial transactions due to the inconsistency of the object identity of one sub-object. In order to verify whether the sub-object is an authorized user of the financial institution, it is necessary to extract the voice pattern of each sub-object. Specifically, for the speaker's voice segment of each sub-object, the voiceprint feature extraction is performed on the speaker's voice segment to obtain the voice representation of the sub-object and obtain the candidate voiceprint feature.

[0088] In step S250 of some embodiments, if the speaker voice segment of at least one sub-object is abnormal voice, the original speaker voice is abnormal voice, and if the speaker voice segments of all sub-objects are normal voice, the original speaker voice is normal voice, and whether the original speaker voice is abnormal or not is related to the candidate voiceprint features of each sub-object. In addition, the original speaker voice is a collection of speaker voice segments of different sub-objects, and the candidate voiceprint features of a single speaker voice segment contain less information. In order to improve the accuracy of speech anomaly recognition, feature fusion is performed on each candidate voiceprint feature to perform comprehensive and sufficient feature extraction on the original speaker voice, so as to obtain voiceprint feature information with high richness and obtain the original voiceprint feature.

[0089] Through the above steps S210 to S250, sufficient feature representation of the original speaker's voice can be obtained to improve the accuracy of voice recognition and achieve secure financial business transactions.

[0090] See also Figure 3 In some embodiments, step S240 may include but is not limited to steps S310 to S340:

[0091] Step S310, performing speech denoising on the speaker's speech segment to obtain a candidate speech segment;

[0092] Step S320, extracting spectrum features from the candidate speech segment to obtain Mel spectrum features;

[0093] Step S330, performing a first voiceprint feature extraction on the Mel spectrum feature to obtain a preliminary voiceprint feature;

[0094] Step S340: extract the second voiceprint feature from the preliminary voiceprint feature to obtain a candidate voiceprint feature.

[0095] In step S310 of some embodiments, when a user conducts business through voice, he or she may be in a noisy environment such as a street or an office area, so that the speaker's voice segment contains background noise. In order to improve the accuracy of voice recognition and avoid interference of background noise on voice recognition, the speaker's voice segment can be denoised by a denoising method to remove the non-human voice part (including silence and various noises) in the speaker's voice segment to improve the voice quality of the speaker's voice segment and obtain a candidate voice segment.

[0096] Specifically, Fourier transform is performed on the speaker's speech segment to obtain a spectrum representation of the speaker's speech segment. Voice activity detection is performed on the speaker's speech segment to obtain a silent segment in the speaker's speech segment. Fourier transform is performed on the silent segment to obtain a spectrum representation of the silent segment. The spectrum representations of each silent segment are averaged to obtain an average spectrum. The average spectrum is removed from the spectrum representation of the speaker's speech segment to obtain a denoised spectrum. Inverse Fourier transform is performed on the denoised spectrum to obtain a candidate speech segment.

[0097] The specific process of voice activity detection includes: dividing the speaker's voice segment into multiple frames. For each frame, extracting the voice features of the frame. Classifying the voice features through the set decision rules to obtain the frame category, the frame category is used to indicate whether the frame is a silent frame or a speech frame. A silent frame is a frame that does not contain any voice activity, and a speech frame is a frame that contains valid voice activity. Multiple consecutive silent frames form a silent segment. The decision rule can be a classification rule based on a threshold or a classification rule based on a machine learning model.

[0098] If the decision rule is a classification rule based on a threshold, the speech feature is compared with the set feature threshold. If the speech feature is greater than or equal to the feature threshold, the frame is regarded as a speech frame. If the speech feature is less than the feature threshold, the frame is regarded as a silent frame. If the decision rule is a classification rule based on a machine learning model, the speech feature is input into the machine learning model for classification to obtain a frame category. The machine learning model can be a support vector machine, a random forest, etc.

[0099] In step S320 of some embodiments, when the speech conditions such as the speaker, accent, and speech speed change, the candidate speech segment will also change. In order to extract the key feature information of the candidate speech segment and suppress unimportant changes, the candidate speech segment is subjected to a discrete Fourier transform to obtain a signal spectrum, and the signal spectrum is subjected to a logarithmic transform to obtain a logarithmic spectrum. The logarithmic spectrum is subjected to an inverse discrete Fourier transform to obtain a Mel-frequency cepstral coefficient, and the Mel-frequency cepstral coefficient is used as a Mel-frequency feature.

[0100] In step S330 of some embodiments, the Mel-spectrogram feature captures the spectral characteristics of the speech signal. In order to extract higher-level features related to the speaker's identity, the Mel-spectrogram feature is subjected to initial voiceprint feature extraction through a voiceprint model to obtain a preliminary voiceprint feature. The voiceprint model may be a Gaussian mixture model, a convolutional neural network, a recurrent neural network, an autoencoder, etc.

[0101] In order to enhance the representation capability of the speaker's voiceprint features, the voiceprint model can be used to extract multi-scale voiceprint features from the Mel spectrum features. The voiceprint model includes a convolution sampling layer, a series of conformer blocks, and an attention statistics pooling layer (ASP). The conformer block is used to learn global features and local features. Specifically, the Mel spectrum features are downsampled by the convolution sampling layer to obtain a first feature map, and the first conformer block is used to extract features from the first feature map to obtain a more robust speaker representation, thereby obtaining a second feature map output by the first conformer block. Taking the second conformer block as the starting point, the second feature map output by the previous conformer block is extracted by the current conformer block to obtain the second feature map output by the current conformer block. The steps of feature extraction by the conformer block are repeated until the second feature map output by the last conformer block is obtained. The second feature maps output by each conformer block are feature spliced ​​to obtain a spliced ​​feature map. The concatenated feature map is subjected to attention transformation through the attention statistics pooling layer to obtain the weight of the second feature map output by each conformer block. The second feature map is multiplied by the corresponding weight, and the multiplication results are added to obtain the preliminary voiceprint feature.

[0102] In step S340 of some embodiments, the speech of different speakers may contain common characteristics, such as the same accent and the same intonation of speech of different speakers, so that the candidate voiceprint features of different speakers contain common factors. In order to make the voiceprint features distinguish different speakers, the preliminary voiceprint features are re-extracted by a multi-layer perceptron to remove the common factors in the preliminary voiceprint features, extract the individual factors, and obtain the candidate voiceprint features. The multi-layer perceptron includes an input layer, a hidden layer and an output layer, and the hidden layer includes a nonlinear activation function, which can be a ReLU function, a sigmoid function or a Tanh function.

[0103] Through the above steps S310 to S340, voiceprint features for distinguishing different speakers can be extracted, thereby ensuring the accuracy of voiceprint feature extraction and further ensuring the accuracy of speech recognition.

[0104] Each candidate voiceprint feature is added together to perform feature fusion to obtain the original voiceprint feature. Different speakers have different candidate voiceprint features, and different candidate voiceprint features contribute to different degrees to speech recognition. In order to further improve the accuracy of speech recognition, each candidate voiceprint feature is fused in a weighted manner.

[0105] See also Figure 4 In some embodiments, step S250 may include but is not limited to step S410 or step S420:

[0106] Step S410, calculating the feature weight of the candidate voiceprint feature;

[0107] Step S420, performing feature weighting on the candidate voiceprint features according to the feature weights to obtain the original voiceprint features.

[0108] In step S410 of some embodiments, the feature weight may be the overall weight of the candidate voiceprint feature. The candidate voiceprint feature is a feature vector, and the feature weight may also be the weight of each element in the candidate voiceprint feature.

[0109] In step S420 of some embodiments, if the feature weight is an overall weight, each candidate voiceprint feature is multiplied by the corresponding feature weight to obtain multiple intermediate voiceprint features. Each intermediate voiceprint feature is added to obtain the original voiceprint feature. For example, the original speaker's speech is divided into 5 speaker's speech segments, and the candidate voiceprint features of these 5 speaker's speech segments are s 1 、s 2 、s 3 、s 4 and 5The feature weights corresponding to these five candidate voiceprint features are λ 1 , 2 , 3 , 4 and λ 5 , then the original voiceprint feature is expressed as: 1 *s 1 +λ 2 *s 2 +λ 3 *s 3 +λ 4 *s 4 +λ 5 *s 5 , * represents multiplication operation.

[0110] If the feature weight is the weight of each element in the candidate voiceprint feature, the weight of the jth element in the i-th candidate voiceprint feature is λ i,j , the jth element in the i-th candidate voiceprint feature is s i,j , then the jth element in the original voiceprint feature is:

[0111] m j =∑ i λ i,j *s i,j ,

[0112] Among them, m j is the jth element of the original voiceprint feature.

[0113] For example, the original speaker's speech is divided into three speaker's speech segments, and the candidate voiceprint features of these three speaker's speech segments are s 1 、s 2 、s 3 , each candidate voiceprint feature consists of three elements, namely s 1 =[s 1,1 ,s 1,2 ,s 1,3 ],s 2 =[s 2,1 ,s 2,2 ,s 2,3 ],s 3 =[s 3,1 ,s 3,2 ,s 3,3 ], element s i,j The weight is λ i,j . Then the first element in the original voiceprint feature is m 1 =λ 1,1 *s 1,1 +λ 2,1 *s 2,1 +λ 3,1 *s3,1 , the second element is m 2 =λ 1,2 *s 1,2 +λ 2,2 *s 2,2 +λ 3,2 *s 3,2 , the third element is m 3 =λ 1,3 *s 1,3 +λ 2,3 *s 2,3 +λ 3,3 *s 3,3 , the original voiceprint feature is [m 1 ,m 2 ,m 3 ].

[0114] In the above steps S410 to S420, the candidate voiceprint features are weighted according to their contribution to speech recognition, so as to integrate the candidate voiceprint features of different speakers and improve the accuracy of speech recognition.

[0115] You can use trained models such as the XGboost model and the lightGBM model to score each candidate voiceprint feature and get the feature importance score of each candidate voiceprint feature. Add the feature importance scores of each candidate voiceprint feature to get the total score. Divide the feature importance score of each candidate voiceprint feature by the total score to get the feature weight of each candidate voiceprint feature. At this time, the feature weight is the overall weight of the candidate voiceprint feature.

[0116] See also Figure 5 In some embodiments, step S410 may include but is not limited to steps S510 to S530:

[0117] Step S510, obtaining an average value of candidate voiceprint features to obtain an average voiceprint feature;

[0118] Step S520, obtaining the standard deviation of the candidate voiceprint feature according to the average voiceprint feature to obtain the standard deviation voiceprint feature;

[0119] Step S530, determining feature weights according to the average voiceprint features and the standard deviation voiceprint features.

[0120] In step S510 of some embodiments, the elements at the corresponding positions of each candidate voiceprint feature are added to obtain the element feature sum at the corresponding position, and the element feature sum at the corresponding position is divided by the number of features of the candidate voiceprint feature to obtain the average value of the corresponding position. The average value can reflect the contribution of the elements at the corresponding positions of the candidate voiceprint feature to speech recognition. Elements with larger mean values ​​have better classification performance in speech recognition tasks. The average voiceprint feature is obtained based on the average values ​​of each position. At this time, the feature length of the average voiceprint feature is M, that is, the number of elements in the average voiceprint feature is M, and the number of elements in the average voiceprint feature is equal to the number of elements in the candidate voiceprint feature. The calculation method of the average voiceprint feature is as follows:

[0121]

[0122] Among them, E j represents the jth element of the average voiceprint feature; N represents the number of candidate voiceprint features; s i,j Represents the jth element of the i-th candidate voiceprint feature.

[0123] Alternatively, the average value of the same candidate voiceprint feature is calculated, and the average value of each candidate voiceprint feature is used as the average voiceprint feature. At this time, the feature length of the average voiceprint feature is N, and the number of elements in the average voiceprint feature is equal to the number of features of the candidate voiceprint feature. The average value can reflect the contribution of different candidate voiceprint features to speech recognition. The candidate voiceprint feature with a larger mean value has better classification performance in the speech recognition task. The calculation method of the average voiceprint feature is as follows:

[0124]

[0125] Among them, E i is the average value of the i-th candidate voiceprint feature, that is, the i-th element of the average voiceprint feature; M is the feature length of the candidate voiceprint feature, that is, the number of elements in the candidate voiceprint feature; s i,j is the jth element of the i-th candidate voiceprint feature.

[0126] In step S520 of some embodiments, the standard deviation of the jth position is calculated based on the jth element of the average voiceprint feature and the jth element of each candidate voiceprint feature, and the standard deviation voiceprint feature is obtained based on the standard deviation of each position. The standard deviation can reflect the degree of deviation of the elements at the same position of different candidate voiceprint features from the average value of the position. The element with a smaller standard deviation has higher stability in the speech recognition task. At this time, the feature length of the standard deviation voiceprint feature is M. The standard deviation of the jth position is expressed as:

[0127]

[0128] Among them, σj Represents the jth element of the standard deviation voiceprint feature.

[0129] Alternatively, the standard deviation of the same candidate voiceprint feature is calculated, and the standard deviation voiceprint feature is obtained according to the standard deviation of each candidate voiceprint feature. At this time, the feature length of the standard deviation voiceprint feature is N. The standard deviation can reflect the degree of deviation of each element in the same candidate voiceprint feature from the average value of the candidate voiceprint feature. The smaller the standard deviation, the higher the stability of the candidate voiceprint feature in the speech recognition task. The standard deviation of the i-th candidate voiceprint feature is expressed as:

[0130]

[0131] Among them, σ i Represents the i-th element of the standard deviation voiceprint feature.

[0132] In step S530 of some embodiments, the average voiceprint feature and the standard deviation voiceprint feature with the same feature length are divided by position to obtain the feature weight. The feature weight is calculated as follows:

[0133]

[0134] When the characteristic length is M, λ j is the feature weight of the jth element of the candidate voiceprint feature. The jth elements of all candidate voiceprint features share the same feature weight. The feature weight is the weight of each element in the candidate voiceprint feature. The larger the feature weight, the better the classification performance and stability of the jth element in the speech recognition task. When the feature length is N, λ j is the feature weight of the jth candidate voiceprint feature. The feature weight is the overall weight of the candidate voiceprint feature. The larger the feature weight, the better the classification performance and stability of the jth candidate voiceprint feature in the speech recognition task.

[0135] Through the above steps S510 to S530, feature weights can be obtained to evaluate the importance of candidate voiceprint features in the speech recognition task based on the feature weights.

[0136] See also Figure 6 In some embodiments, step S250 may include but is not limited to steps S610 to S630:

[0137] Step S610, calculating the feature score of the candidate voiceprint feature;

[0138] Step S620, screening the candidate voiceprint features according to the feature scores to obtain the reference voiceprint features;

[0139] Step S630, performing feature splicing on the reference voiceprint features to obtain original voiceprint features.

[0140] In step S610 of some embodiments, the feature score is used to reflect the importance of the candidate voiceprint feature to the speech recognition task. The larger the feature score, the greater the contribution of the candidate voiceprint feature to speech recognition. Trained models such as XGboost model, lightGBM model, etc. can be used to score each candidate voiceprint feature to obtain a feature score for each candidate voiceprint feature. Alternatively, the self-attention mechanism is used to perform an attention transformation on the candidate voiceprint features to obtain a feature score. The self-attention mechanism includes a query weight matrix and a key weight matrix. The query weight matrix is ​​multiplied by the candidate voiceprint feature to obtain a query vector, and the key weight matrix is ​​multiplied by the candidate voiceprint feature to obtain a key vector. The dot product of the query vector and the key vector is calculated to obtain the feature score.

[0141] In step S620 of some embodiments, in order to improve the efficiency of speech recognition and enable the intelligent voice assistant to recognize the user voice from the terminal device in real time, candidate voiceprint features with feature scores greater than or equal to a preset gain threshold are selected as baseline voiceprint features, and candidate voiceprint features with feature scores less than the preset gain threshold are eliminated.

[0142] In step S630 of some embodiments, whether the original speaker's voice is abnormal is related to the voiceprint features of each speaker. While ensuring the efficiency of speech recognition, in order to improve the accuracy of speech recognition, the embodiments of the present application integrate the voiceprint features of different speakers, perform feature splicing on the reference voiceprint features, and obtain the original voiceprint features.

[0143] In the above steps S610 to S630, speech recognition is performed by splicing voiceprint features with higher importance, thereby improving the efficiency and accuracy of speech recognition to meet the business processing needs of users in real-time financial transaction scenarios.

[0144] See also Figure 7 In some embodiments, before step S130, the method may include but is not limited to steps S710 to S730:

[0145] Step S710, obtaining sample speaker speech;

[0146] Step S720, extracting features from the sample speaker's speech to obtain sample voiceprint features;

[0147] Step S730: construct a voiceprint feature database based on the sample voiceprint features.

[0148] In step S710 of some embodiments, a sample speaker voice is obtained, and the sample speaker voice is the real speaking voice of an authorized user. When the user handles a business, the user is guided to register a voiceprint, and the user is instructed to speak a specific text to obtain a sample speaker voice. The sample speaker voice has a user ID. In order to build a voiceprint feature database, sample speaker voices collected by voice services over a period of time and with user IDs can be obtained. The sample speaker voices are converted into a unified audio file format, such as converted into a waveform audio file format (WAV), and the sample speaker voices are cut into a unified time length, and the sample speaker voices after the time length is cut are sampled at a preset sampling rate to facilitate subsequent processing of voice signals. The preset sampling rate can be 8000Hz.

[0149] In step S720 of some embodiments, the sample speaker's speech is subjected to speech denoising to obtain pure speech, and the Mel-frequency cepstral coefficients of the pure speech are extracted. The Mel-frequency cepstral coefficients are subjected to voiceprint feature extraction through a pre-trained voiceprint model to obtain voiceprint feature information. The voiceprint feature information is subjected to voiceprint feature extraction through a multi-layer perceptron to remove common voiceprint information and obtain sample voiceprint features. The sample voiceprint features are used to verify the identity of the user. The voiceprint model and the multi-layer perceptron are both trained based on large-scale human voice data.

[0150] In step S730 of some embodiments, the user ID and the sample voiceprint feature are stored in a database to obtain a voiceprint feature database. The database may be an oracle database, a redis database, or the like.

[0151] Through the above steps S710 to S730, a voiceprint feature library can be constructed to perform speech recognition based on the voiceprint feature library.

[0152] In step S130 of some embodiments, a clustering algorithm is called to cluster the sample voiceprint features of the voiceprint feature database, and the sample voiceprint features of the same category are classified into one cluster, and the sample voiceprint features of different categories are classified into different clusters, to obtain multiple voiceprint feature clusters, each voiceprint feature cluster has a cluster center, and the cluster center is the cluster center.

[0153] In step S140 of some embodiments, the sample voiceprint features in the same voiceprint feature cluster are averaged to obtain the reference voiceprint feature at the cluster center of the voiceprint feature cluster. For example, a voiceprint feature cluster includes 3 sample voiceprint features, which are w1, w2 and w3 respectively. The reference voiceprint feature of the voiceprint feature cluster is expressed as (w1+w2+w3) / 3.

[0154] In step S150 of some embodiments, in order to verify the user identity of the original speaker's voice, the similarity between the original voiceprint feature and the reference voiceprint feature of each voiceprint feature cluster is calculated respectively. Specifically, the Euclidean distance or cosine distance between the original voiceprint feature and the reference voiceprint feature of each voiceprint feature cluster can be calculated, and the reciprocal of the Euclidean distance or the reciprocal of the cosine distance is used as the similarity. The smaller the Euclidean distance or cosine distance between the original voiceprint feature and the reference voiceprint feature, the greater the similarity between the original voiceprint feature and the reference voiceprint feature, the greater the possibility that the user identity corresponding to the original speaker's voice is the user ID corresponding to the reference voiceprint feature in the voiceprint feature database, and the greater the probability that the original speaker's voice is normal voice.

[0155] In step S160 of some embodiments, the maximum similarity is selected from multiple similarities to obtain a maximum similarity value. The original speaker's speech whose maximum similarity value is greater than or equal to a preset similarity threshold is regarded as normal speech, and the original speaker's speech whose maximum similarity value is less than the preset similarity threshold is regarded as abnormal speech.

[0156] A large amount of business call voice data accumulated over a period of time can be obtained, and voiceprint features of each business call voice data can be extracted to obtain the original voiceprint features of each business call voice data. For each original voiceprint feature, the similarity between the original voiceprint feature and each reference voiceprint feature in the voiceprint feature database is calculated, and the maximum similarity is taken as the target similarity. Business call voice data with a target similarity greater than or equal to a preset similarity threshold is regarded as normal voice, and business call voice data with a target similarity less than a preset similarity threshold is regarded as abnormal voice.

[0157] See also Figure 8 The embodiment of the present application further provides a device for speech recognition in a conversation, which can implement the above-mentioned method for speech recognition in a conversation, and the device includes:

[0158] An acquisition module 810 is used to acquire the original speaker's voice in the conversation;

[0159] The feature extraction module 820 is used to extract the voiceprint features of the original speaker's speech to obtain the original voiceprint features;

[0160] The clustering module 830 is used to cluster the sample voiceprint features of the pre-built voiceprint feature database to obtain voiceprint feature clusters; the voiceprint feature clusters have cluster centers;

[0161] A determination module 840, configured to determine a reference voiceprint feature at the center of a cluster according to the sample voiceprint features of the voiceprint feature cluster;

[0162] A calculation module 850 is used to calculate the similarity between the original voiceprint feature and the reference voiceprint feature;

[0163] The speech recognition module 860 is used to perform speech recognition on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

[0164] The specific implementation of the device for speech recognition in conversation is substantially the same as the specific implementation of the method for speech recognition in conversation described above, and will not be described in detail herein.

[0165] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned method for speech recognition in a conversation when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0166] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0167] The processor 910 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0168] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 920, and the processor 910 calls and executes the method for speech recognition in a conversation in the embodiments of this application;

[0169] Input / output interface 930, used to implement information input and output;

[0170] Communication interface 940, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0171] bus 950 , which transmits information between the various components of the device (e.g., processor 910 , memory 920 , input / output interface 930 , and communication interface 940 );

[0172] The processor 910 , the memory 920 , the input / output interface 930 , and the communication interface 940 are connected to each other in communication within the device via a bus 950 .

[0173] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for speech recognition in conversation is implemented.

[0174] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The method for speech recognition in conversation, the device for speech recognition in conversation, the electronic device and the computer storage medium provided by the embodiments of the present application extract the original voiceprint features of the original speaker's speech, calculate the similarity between the original voiceprint features and the voiceprint features in the voiceprint feature database, and distinguish the original speaker's speech as normal speech or abnormal speech according to the similarity, so as to resist the speech attack of abnormal speech. Furthermore, the sample voiceprint features in the voiceprint feature database are clustered, a large number of sample voiceprint features are clustered into voiceprint feature clusters, and the voiceprint features of the entire cluster are replaced by the voiceprint features at the center of the voiceprint feature cluster, which greatly improves the efficiency of similarity calculation, thereby improving the efficiency of speech recognition.

[0176] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0177] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0181] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0186] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for speech recognition in a conversation, characterized in that: The method comprises: Get the original speaker voice in the conversation; Extracting voiceprint features from the original speaker's speech to obtain original voiceprint features; Clustering sample voiceprint features of a pre-constructed voiceprint feature database to obtain a voiceprint feature cluster; the voiceprint feature cluster has a cluster center; Determining a reference voiceprint feature at the cluster center according to the sample voiceprint feature of the voiceprint feature cluster; Calculating the similarity between the original voiceprint feature and the reference voiceprint feature; Speech recognition is performed on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

2. The method according to claim 1, characterized in that The conversation includes a speaking object, and the extracting of voiceprint features from the original speaker's voice to obtain the original voiceprint features includes: Performing voice conversion on the original speaker's voice to obtain a speaker text; Performing word recognition on the speaker text to obtain target spoken words; Performing speech segmentation on the original speaker's speech according to the target speech words to obtain speech segments of the speaker of the speaking object; Extracting voiceprint features from the speaker's speech segment to obtain candidate voiceprint features; The candidate voiceprint features are fused to obtain the original voiceprint features.

3. The method according to claim 2, characterized in that The extracting voiceprint features from the speaker's speech segment to obtain candidate voiceprint features includes: Performing speech denoising on the speaker's speech segment to obtain a candidate speech segment; Extracting spectral features of the candidate speech segment to obtain Mel-spectrogram features; Performing a first voiceprint feature extraction on the Mel frequency spectrum feature to obtain a preliminary voiceprint feature; A second voiceprint feature is extracted from the preliminary voiceprint feature to obtain the candidate voiceprint feature.

4. The method according to claim 2, characterized in that: The step of fusing the candidate voiceprint features to obtain the original voiceprint features includes: Calculating feature weights of the candidate voiceprint features; The candidate voiceprint features are feature-weighted according to the feature weights to obtain the original voiceprint features.

5. The method according to claim 4, characterized in that The calculating the feature weight of the candidate voiceprint feature includes: Obtaining an average value of the candidate voiceprint features to obtain an average voiceprint feature; Obtaining the standard deviation of the candidate voiceprint feature according to the average voiceprint feature to obtain a standard deviation voiceprint feature; The feature weight is determined according to the average voiceprint feature and the standard deviation voiceprint feature.

6. The method according to claim 2, characterized in that The step of fusing the candidate voiceprint features to obtain the original voiceprint features includes: Calculating a feature score of the candidate voiceprint feature; Screening the candidate voiceprint features according to the feature scores to obtain a benchmark voiceprint feature; The reference voiceprint features are spliced ​​to obtain the original voiceprint features.

7. The method according to any one of claims 1 to 6, characterized in that: Before clustering the sample voiceprint features of the pre-built voiceprint feature database to obtain voiceprint feature clusters, the method further includes: Obtain sample speaker speech; Extracting features of the sample speaker's speech to obtain sample voiceprint features; The voiceprint feature database is constructed according to the sample voiceprint features.

8. A speech recognition device for conversation, characterized in that: The device comprises: An acquisition module, used to acquire the original speaker's voice in the conversation; A feature extraction module, used to extract voiceprint features from the original speaker's speech to obtain original voiceprint features; A clustering module, used for clustering sample voiceprint features of a pre-built voiceprint feature database to obtain voiceprint feature clusters; the voiceprint feature clusters have a cluster center; A determination module, configured to determine a reference voiceprint feature of the cluster center according to the sample voiceprint feature of the voiceprint feature cluster; A calculation module, used to calculate the similarity between the original voiceprint feature and the reference voiceprint feature; The speech recognition module is used to perform speech recognition on the original speaker's speech according to the similarity to obtain a speech category; the speech category is used to indicate whether the original speaker's speech is an abnormal speech or a normal speech.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images