Speech recognition method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of voice processing, and relates to a voice recognition method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining multi-language voice signal information, and carrying out the preprocessing of the multi-language voice signal information, and obtaining standard voice signal information; performing feature extraction on the standard voice signal information to obtain an acoustic feature vector sequence; training a voiceprint embedding model based on the acoustic feature vector sequence; acquiring multi-language voice data, and inputting the multi-language voice data into the voiceprint embedding model to obtain a voiceprint feature vector; constructing a language voiceprint database based on the voiceprint feature vectors; performing feature extraction on the to-be-recognized voice information based on the voiceprint embedding model to obtain a to-be-recognized voiceprint feature vector; and comparing and matching the to-be-recognized voiceprint feature vector with a language voiceprint database to obtain a voice recognition result. According to the invention, the voice object of the to-be-recognized voice information can be accurately recognized.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, specifically to the field of digital medicine, and in particular to a speech recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] In the field of digital medicine, voiceprint recognition technology is gradually showing its huge application potential, especially in remote medical consultation, patient identity authentication and disease auxiliary diagnosis. However, the significant differences in the phonemes, rhythms, and intonations of speech signals in different languages ​​have brought severe challenges to cross-language voiceprint recognition.

[0003] Traditional single-language voiceprint feature extraction methods are often difficult to directly apply to multilingual scenarios. This is because different languages ​​have significant differences in speech characteristics, such as the distribution of phonemes, changes in rhythm, and fluctuations in intonation. These differences make it difficult for traditional feature extraction methods to maintain stable recognition performance in multilingual environments.

[0004] In addition, what is more complicated is that in actual applications, the speaker's voice may be mixed with multiple languages, or code switching may occur. This complex language environment further increases the difficulty of cross-language voiceprint recognition. For example, in remote medical consultations, patients may use their native language or a mixture of multiple languages ​​to describe their symptoms. At this time, the output voice information is difficult to be effectively recognized, so it is impossible to effectively determine the patient's true identity, which brings certain difficulties to the effective voiceprint recognition of patients. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a speech recognition method, apparatus, computer device and storage medium to solve the problem that the speech object to be recognized cannot be accurately recognized.

[0006] In order to solve the above technical problems, the present application provides a speech recognition method, which adopts the following technical solution:

[0007] Acquiring multilingual voice signal information, and preprocessing the multilingual voice signal information to obtain standard voice signal information;

[0008] Extracting features from the standard speech signal information to obtain an acoustic feature vector sequence;

[0009] Training a voiceprint embedding model based on the acoustic feature vector sequence;

[0010] Acquire multilingual speech data, and input the multilingual speech data into the voiceprint embedding model to obtain a voiceprint feature vector;

[0011] Building a language voiceprint database based on the voiceprint feature vector;

[0012] Extracting features of the voice information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized;

[0013] The voiceprint feature vector to be recognized is compared and matched with the language voiceprint database to obtain a speech recognition result.

[0014] Furthermore, the step of acquiring multilingual voice signal information and preprocessing the multilingual voice signal information to obtain standard voice signal information specifically includes:

[0015] Acquire a signal information extraction identifier, and extract the multilingual voice signal information from a database according to the signal information extraction identifier;

[0016] Performing voice activity detection on the multilingual voice signal information to obtain valid voice signal information;

[0017] Performing noise reduction processing on the effective voice signal information to obtain noise-reduced voice signal information;

[0018] The noise reduction speech signal information is speech segmented according to a preset speech segmentation rule to obtain the standard speech signal information.

[0019] Furthermore, the step of extracting features from the standard speech signal information to obtain an acoustic feature vector sequence specifically includes:

[0020] Dividing the standard voice signal information into units to obtain language phoneme units;

[0021] Extracting features of the language phoneme unit to obtain a unit acoustic feature vector;

[0022] The unit acoustic feature vectors are concatenated according to the corresponding order of the standard speech signal information to obtain the acoustic feature vector sequence.

[0023] Furthermore, the step of training the voiceprint embedding model based on the acoustic feature vector sequence specifically includes:

[0024] Obtaining an initial model extraction identifier, and extracting an initial voiceprint embedding model from a database according to the initial model extraction identifier;

[0025] Inputting the acoustic feature vector sequence into the initial voiceprint embedding model to obtain an initial voiceprint feature vector;

[0026] A contrast loss calculation is performed on the initial voiceprint feature vector, and the initial voiceprint embedding model is optimized according to the calculated loss value to obtain an optimized voiceprint embedding model.

[0027] Furthermore, the step of acquiring multilingual speech data and inputting the multilingual speech data into the voiceprint embedding model to obtain the voiceprint feature vector specifically includes:

[0028] Acquire a voice data extraction identifier, and extract the multilingual voice data from a preset corpus according to the voice data extraction identifier;

[0029] Preprocessing the multilingual speech data to obtain multilingual speech segments;

[0030] The multilingual speech segment is input into the voiceprint embedding model to obtain the voiceprint feature vector.

[0031] Furthermore, the step of constructing a language voiceprint database based on the voiceprint feature vector specifically includes:

[0032] Acquire language tags corresponding to the multilingual speech data, classify the voiceprint feature vectors according to the language tags, and obtain a voiceprint feature vector library;

[0033] Clustering the voiceprint feature vectors in the voiceprint feature vector library according to a preset clustering algorithm to obtain voiceprint feature vector clusters;

[0034] Taking the cluster center of the voiceprint feature vector cluster as the voiceprint feature vector representative of the speech object, constructing a voiceprint dictionary of the speech object;

[0035] The voiceprint feature vector library and the speech object voiceprint dictionary are merged to obtain the language voiceprint database.

[0036] Furthermore, the step of comparing and matching the voiceprint feature vector to be identified with the language voiceprint database to obtain a speech recognition result specifically includes:

[0037] Calculating the similarity between the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database to obtain a similarity result;

[0038] Sorting the similarity results, and selecting the highest ranked similarity result as the comparison similarity value;

[0039] Determining whether the comparison similarity value exceeds a preset similarity threshold;

[0040] If the comparison similarity value exceeds the preset similarity threshold, the speech information to be recognized is determined as the speech object corresponding to the similarity result, and the speech object is output as the speech recognition result.

[0041] In order to solve the above technical problems, the embodiment of the present application further provides a speech recognition device, which adopts the following technical solution:

[0042] An information acquisition module, used to acquire multilingual voice signal information and pre-process the multilingual voice signal information to obtain standard voice signal information;

[0043] A sequence extraction module, used for extracting features from the standard speech signal information to obtain an acoustic feature vector sequence;

[0044] A model training module, used for training a voiceprint embedding model based on the acoustic feature vector sequence;

[0045] A model output module, used for acquiring multilingual speech data, and inputting the multilingual speech data into the voiceprint embedding model to obtain a voiceprint feature vector;

[0046] A database construction module, used for constructing a language voiceprint database based on the voiceprint feature vector;

[0047] A feature extraction module, used for extracting features of the speech information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized;

[0048] The result recognition module is used to compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a speech recognition result.

[0049] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0050] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above-mentioned speech recognition methods when executing the computer-readable instructions.

[0051] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0052] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the speech recognition method described in any one of the above items are implemented.

[0053] Compared with the prior art, the embodiment of the present application has the following beneficial effects: the embodiment obtains multilingual voice signal information and preprocesses the multilingual voice signal information to obtain standard voice signal information; extracts features from the standard voice signal information to obtain an acoustic feature vector sequence; trains a voiceprint embedding model based on the acoustic feature vector sequence; obtains multilingual voice data and inputs the multilingual voice data into the voiceprint embedding model to obtain a voiceprint feature vector; constructs a language voiceprint database based on the voiceprint feature vector; extracts features from the voice information to be recognized based on the voiceprint embedding model to obtain a voiceprint feature vector to be recognized; compares and matches the voiceprint feature vector to be recognized with the language voiceprint database to obtain a voice recognition result. This effectively realizes accurate recognition of the voice object of the voice information to be recognized, so as to improve the accuracy and security of the system voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0056] Figure 2 A flow chart of an embodiment of a speech recognition method according to the present application;

[0057] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;

[0058] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;

[0059] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S30;

[0060] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S40;

[0061] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S50;

[0062] Figure 8 yes Figure 2A flowchart of a specific implementation of step S70;

[0063] Fig. 9 is a structural schematic diagram of an embodiment of a speech recognition device according to the present application;

[0064] Fig.10 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0066] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to non-related or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0067] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0068] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0069] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0070] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0071] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0072] It should be noted that the speech recognition method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the speech recognition device is generally arranged in the server / terminal device.

[0073] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0074] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for speech recognition according to the present application. The speech recognition method comprises the following steps:

[0075] Step S10, acquiring multi-language voice signal information, and pre-processing the multi-language voice signal information to obtain standard voice signal information;

[0076] In this embodiment, the multilingual speech signal refers to a signal containing multiple different natural language speech information, which may come from different speakers, different environmental backgrounds, and exist in different language forms, and the above languages ​​belong to different language families or language groups, such as Chinese, English, French, German, Spanish, etc. The preprocessing includes speech activity detection, background noise suppression, and speech segmentation. By performing speech activity detection, background noise suppression, and speech segmentation preprocessing on the multilingual speech signal information, standard and effective standard speech signal information is obtained.

[0077] Step S20, extracting features from the standard speech signal information to obtain an acoustic feature vector sequence;

[0078] In this embodiment, the acoustic feature vector sequence is a sequence composed of multiple unit acoustic feature vectors, wherein the unit acoustic feature vector is a feature vector corresponding to a speech phoneme unit and is obtained by performing feature extraction on the speech phoneme unit.

[0079] Step S30, training a voiceprint embedding model based on the acoustic feature vector sequence;

[0080] In this embodiment, the voiceprint embedding model is used to identify the voiceprint feature vector in the input voice information or voice data, including a convolutional neural network layer, a recurrent neural network layer, and a fully connected layer. In this embodiment, the voiceprint embedding model can adopt a d-vector model, which uses a deep neural network (DNN) to learn the characteristic representation of the speaker.

[0081] Step S40, acquiring multi-language speech data, and inputting the multi-language speech data into the voiceprint embedding model to obtain a voiceprint feature vector;

[0082] In this embodiment, the multilingual voice data is voice data obtained from a preset corpus, which includes data of different speakers, different voice environments, and different voice contents. A matching query is performed in the preset corpus through the voice data extraction identifier to extract the corresponding multilingual voice data.

[0083] Step S50, constructing a language voiceprint database based on the voiceprint feature vector;

[0084] In this embodiment, the language voiceprint database includes a voiceprint feature vector library and a voice object voiceprint dictionary, wherein the voiceprint feature vector library stores voiceprint feature vectors classified according to language categories, and the voiceprint feature vectors are used to represent the unique voiceprint of a person speaking. The voice object voiceprint dictionary records the voice object information corresponding to each voiceprint feature vector, such as the speaker's identity information, the description of the voice content, etc. The language voiceprint database is constructed through the voiceprint feature vector library and the voice object voiceprint dictionary, thereby effectively facilitating the subsequent voice object recognition corresponding to the voice information to be recognized.

[0085] Step S60, extracting features of the speech information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized;

[0086] In this embodiment, the speech information to be recognized is preprocessed to obtain standard speech information to be recognized, and then the standard speech information to be recognized is framed and windowed to obtain an acoustic feature vector sequence to be recognized, and finally the acoustic feature vector sequence to be recognized is input into the voiceprint embedding model for feature extraction, thereby obtaining a voiceprint feature vector to be recognized. The preprocessing methods include voice activity detection, background noise suppression, and speech segmentation.

[0087] Step S70, comparing and matching the voiceprint feature vector to be recognized with the language voiceprint database to obtain a speech recognition result.

[0088] In this embodiment, the comparison and matching step is performed by calculating the similarity between the voiceprint feature vector to be identified and the voiceprint feature vector stored in the language voiceprint database, and the similarity can be calculated by a cosine similarity algorithm. The speech recognition result includes a corresponding speech object result and a no-speech object result, and the corresponding result display field can be preset for display on the system. For example, the display field of the speech object result is "Recognition successful, the current user is correct", and the display field of the no-speech object result is "Recognition failed, the system has no current user".

[0089] In this embodiment, the above method can be applied to a medical service system, in which the user's identity corresponding to the voice information to be recognized is verified and recognized in the medical service system, thereby realizing secure login of the user. Specifically, in this embodiment, the medical service system can be one or more of a medical insurance system and a disease insurance system, the voice information to be recognized is the user input voice collected by a mobile device or a detection device, the multilingual voice signal information, the voiceprint embedding model, and the language voiceprint database are all stored in the medical insurance system and the disease insurance system and obtained from the database of the above system, the voice recognition result is generated by the above system through the method of this embodiment, and stored in the system database for preservation, and the recognition result is displayed on the system display interface.

[0090] This embodiment obtains multilingual voice signal information and preprocesses the multilingual voice signal information to obtain standard voice signal information; extracts features from the standard voice signal information to obtain an acoustic feature vector sequence; trains a voiceprint embedding model based on the acoustic feature vector sequence; obtains multilingual voice data and inputs the multilingual voice data into the voiceprint embedding model to obtain a voiceprint feature vector; builds a language voiceprint database based on the voiceprint feature vector; extracts features from the voice information to be recognized based on the voiceprint embedding model to obtain a voiceprint feature vector to be recognized; compares and matches the voiceprint feature vector to be recognized with the language voiceprint database to obtain a voice recognition result. This effectively realizes accurate recognition of the voice object of the voice information to be recognized, so as to improve the accuracy and security of the system voice recognition.

[0091] refer to Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:

[0092] Step S101, obtaining a signal information extraction identifier, and extracting the multilingual speech signal information from a database according to the signal information extraction identifier;

[0093] In this embodiment, the signal information extraction identifier is unique identification information corresponding to the multi-language voice signal information, and a matching query is performed in the database through the signal information extraction identifier to obtain the corresponding multi-language voice signal information.

[0094] Step S102, performing voice activity detection on the multilingual voice signal information to obtain valid voice signal information;

[0095] In this embodiment, voice activity detection is a method of identifying the part actually containing voice activity from multilingual voice signal information and excluding non-voice parts such as silence and noise. The distinction between voice and non-voice signals can be achieved through signal processing technology, such as energy detection, zero-crossing rate detection, spectrum analysis, etc. In this embodiment, voice activity detection is performed through spectrum analysis.

[0096] Step S103, performing noise reduction processing on the effective voice signal information to obtain noise-reduced voice signal information;

[0097] In this embodiment, noise reduction processing is a method of removing or reducing background noise from effective speech signal information to improve the clarity and intelligibility of speech signals. Noise reduction processing can be achieved through noise reduction techniques such as spectrum subtraction, Wiener filtering, and adaptive filtering. In this embodiment, adaptive filtering is used for noise reduction processing.

[0098] Step S104: Segment the noise reduction speech signal information according to a preset speech segmentation rule to obtain the standard speech signal information.

[0099] In this embodiment, speech segmentation is a method of segmenting the noise-reduced speech signal information into a plurality of standard speech signal information segments according to a preset speech segmentation rule. The speech segmentation rule may be based on specific features of the speech signal, such as silence interval, duration of speech activity, etc. By comparing these features with a preset threshold, the noise-reduced speech signal information is segmented into a plurality of suitable segments. After segmentation, the obtained speech signal segments are combined to obtain standard speech signal information.

[0100] This embodiment obtains a signal information extraction identifier, extracts the multilingual voice signal information from a database according to the signal information extraction identifier; performs voice activity detection on the multilingual voice signal information to obtain valid voice signal information; performs noise reduction processing on the valid voice signal information to obtain noise-reduced voice signal information; and performs voice segmentation on the noise-reduced voice signal information according to a preset voice segmentation rule, thereby effectively obtaining standard voice signal information that meets subsequent processing requirements, so as to facilitate subsequent feature extraction processing.

[0101] refer to Figure 4In some optional implementations of this embodiment, step S20 includes the following steps:

[0102] Step S201, dividing the standard speech signal information into units to obtain language phoneme units;

[0103] In this embodiment, the language phoneme unit is the phoneme corresponding to the standard voice signal information. The phoneme is the smallest speech unit that can distinguish meaning in a language, such as a vowel, a consonant, etc.

[0104] Step S202, extracting features from the language phoneme unit to obtain a unit acoustic feature vector;

[0105] In this embodiment, the unit acoustic feature vector includes Mel frequency cepstral coefficients, fundamental frequency and energy, and the extraction of Mel frequency cepstral coefficients includes pre-emphasis: pre-emphasis processing is performed on the language phoneme unit to reduce the loss of the high frequency part and make the signal flatter in the frequency domain. Framing: the language phoneme unit is divided into multiple short-term frames, and each frame usually contains a certain number of sampling points. Windowing: each frame signal is windowed to reduce the discontinuity between frames. Among them, window functions such as rectangular windows and Hamming windows can be used for windowing. Fast Fourier transform (FFT): FFT is performed on each frame signal after windowing, and it is converted from the time domain to the frequency domain to obtain a spectrum. Mel filter bank: The spectrum is filtered through a group of Mel filter banks to obtain a Mel spectrum. Logarithm: The Mel spectrum is logarithmized to obtain a logarithmic Mel spectrum. Discrete cosine transform (DCT): DCT is performed on the logarithmic Mel spectrum to obtain Mel frequency cepstral coefficients (MFCC), and the first 13 coefficients after DCT can be taken as MFCC features. The fundamental frequency is extracted by calculating the autocorrelation function of the language phoneme unit and finding the position corresponding to the maximum value of the autocorrelation function. The frequency corresponding to this position is the fundamental frequency. Energy can be extracted by performing a square sum operation on each frame of speech signal to obtain the energy value of the frame.

[0106] Step S203: concatenate the unit acoustic feature vectors according to the corresponding order of the standard speech signal information to obtain the acoustic feature vector sequence.

[0107] In this embodiment, the corresponding order of the standard speech signal information is the same as the order of the language phoneme units in the standard speech signal information. The language phoneme units can be marked with serial numbers, and then the unit acoustic feature vectors are spliced ​​according to the marked serial numbers to obtain an acoustic feature vector sequence.

[0108] This embodiment obtains language phoneme units by dividing the standard speech signal information into units; extracts features from the language phoneme units to obtain unit acoustic feature vectors; and concatenates the unit acoustic feature vectors in the corresponding order of the standard speech signal information, thereby effectively obtaining an acoustic feature vector sequence that effectively represents the features of the standard speech signal information, so as to facilitate subsequent training of the voiceprint embedding model.

[0109] refer to Figure 5 In some optional implementations of this embodiment, step S30 includes the following steps:

[0110] Step S301, obtaining an initial model extraction identifier, and extracting an initial voiceprint embedding model from a database according to the initial model extraction identifier;

[0111] In this embodiment, the initial model extraction identifier is a unique identifier corresponding to the initial voiceprint embedding model, and a matching query is performed in the database through the initial model extraction identifier to extract the corresponding initial voiceprint embedding model.

[0112] Step S302, inputting the acoustic feature vector sequence into the initial voiceprint embedding model to obtain an initial voiceprint feature vector;

[0113] In this embodiment, the initial voiceprint embedding model includes a convolutional neural network layer, a recurrent neural network layer, and a fully connected layer. The local acoustic features are extracted by inputting the acoustic feature vector sequence into the convolutional neural network layer. The local acoustic features are time-series modeled by inputting the local acoustic features into the recurrent neural network layer to obtain a high-level feature representation of speech. The high-level feature representation of speech is input into the fully connected layer to map the high-level feature representation of speech to a voiceprint embedding space of a preset dimension to obtain an initial voiceprint feature vector.

[0114] Step S303, performing contrast loss calculation on the initial voiceprint feature vector, and optimizing the initial voiceprint embedding model according to the calculated loss value to obtain an optimized voiceprint embedding model.

[0115] In this embodiment, the loss value is calculated according to the contrast loss function preset by the initial voiceprint embedding model. The contrast loss function is usually used to measure the similarity (or difference) between the voiceprint feature vectors output by the model. In this embodiment, the model needs to make the voiceprint feature vectors of the same speaker as close as possible, and the voiceprint feature vectors of different speakers as far apart as possible. Therefore, the contrast loss function calculates the distance (or similarity) between these feature vectors and evaluates the performance of the model based on these distances (or similarities). After obtaining the loss value, an optimization algorithm (such as gradient descent, Adam, etc.) is used to update the parameters of the voiceprint embedding model based on the calculated loss value to reduce the loss value and improve the performance of the model. This process usually involves multiple iterations, each of which uses a batch of acoustic feature vector sequences to train the model and calculates a new loss value to guide the update of the parameters.

[0116] This embodiment obtains an initial model extraction identifier, extracts an initial voiceprint embedding model from a database according to the initial model extraction identifier; inputs the acoustic feature vector sequence into the initial voiceprint embedding model to obtain an initial voiceprint feature vector; performs comparative loss calculation on the initial voiceprint feature vector, and optimizes the initial voiceprint embedding model according to the calculated loss value, thereby effectively obtaining a trained and optimized voiceprint embedding model to facilitate subsequent feature extraction of the speech information to be recognized.

[0117] Continue to refer Figure 6 In some optional implementations of this embodiment, step S40 includes the following steps:

[0118] Step S401, obtaining a voice data extraction identifier, and extracting the multilingual voice data from a preset corpus according to the voice data extraction identifier;

[0119] In this embodiment, the voice data extraction identifier is unique identification information corresponding to the multi-language voice data, and a matching query is performed in a preset corpus according to the voice data extraction identifier to extract the corresponding multi-language voice data.

[0120] Step S402, preprocessing the multilingual voice data to obtain multilingual voice segments;

[0121] In this embodiment, the preprocessing of multilingual speech data includes denoising and segmentation, wherein denoising is to remove background noise in speech data to improve speech clarity, and segmentation is to divide continuous speech data into shorter speech segments for subsequent processing and analysis. The segmentation can be implemented according to the above-mentioned preset speech segmentation rules.

[0122] Step S403: input the multilingual speech segment into the voiceprint embedding model to obtain the voiceprint feature vector.

[0123] In this embodiment, by inputting multilingual speech segments into the trained voiceprint embedding model, the voiceprint feature vector corresponding to the multilingual speech segments is effectively obtained, where the voiceprint feature vector includes spectral features (Mel-frequency cepstral coefficients), fundamental frequency and its change rate, resonance peak features, time series features, etc.

[0124] This embodiment obtains a voice data extraction identifier, and extracts the multilingual voice data from a preset corpus according to the voice data extraction identifier; preprocesses the multilingual voice data to obtain multilingual voice segments; and inputs the multilingual voice segments into the voiceprint embedding model, thereby effectively obtaining voiceprint feature vectors corresponding to the multilingual voice data, so as to facilitate the subsequent construction of a language voiceprint database.

[0125] Continue to refer Figure 7 In some optional implementations of this embodiment, step S50 includes the following steps:

[0126] Step S501, obtaining language tags corresponding to the multilingual speech data, and classifying the voiceprint feature vectors according to the language tags to obtain a voiceprint feature vector library;

[0127] In this embodiment, the language tag is the label information corresponding to different languages ​​in the multilingual speech data. By matching the language tag with the voiceprint feature vector, the voiceprint feature vector is classified, and a voiceprint feature vector library of cross-language speakers is effectively constructed.

[0128] Step S502, clustering the voiceprint feature vectors in the voiceprint feature vector library according to a preset clustering algorithm to obtain voiceprint feature vector clusters;

[0129] In this embodiment, the preset clustering algorithm may adopt a K-means clustering algorithm, which clusters voiceprint features belonging to the same voice object (speaker) together to form voiceprint feature vector clusters corresponding to different voice objects.

[0130] Step S503, taking the cluster center of the voiceprint feature vector cluster as the voiceprint feature vector representative of the speech object, and constructing a voiceprint dictionary of the speech object;

[0131] In this embodiment, by taking the cluster center of the voiceprint feature vector cluster (the centroid in the K-means algorithm) as the voiceprint feature vector representative corresponding to the voice object, the voiceprint feature vector representatives of all voice objects (speakers) in each language are sorted into a dictionary. The key of the dictionary can be the unique identifier of the voice object (such as ID), and the value is the voiceprint feature vector representative of the voice object, thereby effectively constructing a voice object voiceprint dictionary.

[0132] Step S504: merging the voiceprint feature vector library and the speech object voiceprint dictionary to obtain the language voiceprint database.

[0133] In this embodiment, the voiceprint feature vector library classified by language and the voiceprint feature vector dictionary corresponding to the voice object (speaker) are merged into a unified cross-language speaker language voiceprint database, which contains voiceprint feature vectors of all languages, and each voice object is represented by a unique voiceprint feature vector.

[0134] This embodiment obtains the language tags corresponding to the multilingual speech data, classifies the voiceprint feature vectors according to the language tags, and obtains a voiceprint feature vector library; clusters the voiceprint feature vectors in the voiceprint feature vector library according to a preset clustering algorithm to obtain a voiceprint feature vector cluster; uses the cluster center of the voiceprint feature vector cluster as the voiceprint feature vector representative of the speech object to construct a speech object voiceprint dictionary; merges the voiceprint feature vector library and the speech object voiceprint dictionary, thereby effectively obtaining a language voiceprint database that integrates the voiceprint feature vector library and the speech object voiceprint dictionary, so as to provide reliable database support for subsequent speech recognition processing.

[0135] Continue to refer Figure 8 In some optional implementations of this embodiment, step S70 includes the following steps:

[0136] Step S701, calculating the similarity between the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database to obtain a similarity result;

[0137] In this embodiment, the cosine similarity algorithm is used to calculate the similarity between the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database. The calculation formula of cosine similarity is: cosine similarity = A*B / ||A|||B||, where A and B are the feature vectors of the voiceprint to be identified and the language voiceprint database, respectively, * represents the dot product, and ||A|| and ||B|| are the modulus (length) of vectors A and B, respectively. The cosine similarity value of the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database is calculated to obtain the corresponding similarity result.

[0138] Step S702, sorting the similarity results, and selecting the highest sorted similarity result as the comparison similarity value;

[0139] In this embodiment, the similarity results are sorted according to the cosine similarity values ​​calculated as above to obtain a corresponding similarity result sorting table, and the similarity result with the highest similarity is selected from the similarity result sorting table as the comparison similarity value that needs to be compared and judged. The comparison similarity value corresponds to the voiceprint feature vector of the target voice object with the highest similarity.

[0140] Step S703, determining whether the comparison similarity value exceeds a preset similarity threshold;

[0141] In this embodiment, the preset similarity threshold is a judgment value used to determine whether the voice object of the voiceprint feature vector to be identified is the same as the voice object corresponding to the feature vector in the language voiceprint database. In this embodiment, the comparison similarity value can be a percentage value. When the comparison similarity value is a percentage value, the preset similarity threshold can be set to 80%. The preset similarity threshold can be adjusted accordingly according to actual conditions.

[0142] Step S704: if the comparison similarity value exceeds the preset similarity threshold, the speech information to be recognized is determined as the speech object corresponding to the similarity result, and the speech object is output as the speech recognition result;

[0143] In this embodiment, the voice object corresponding to the similarity result refers to the voice object corresponding to the voiceprint feature vector in the language voiceprint database that is compared with the voiceprint feature vector to be identified in the similarity result. After the voice object is determined, a preset result display template can be obtained, and the field corresponding to the voice object can be filled in the result display template to obtain the voice recognition result and send it to the display interface for result output.

[0144] Step S705: if the comparison similarity value does not exceed the preset similarity threshold, outputting a result of no recognized object.

[0145] In this embodiment, by filling the field corresponding to the unrecognized object into the preset result display template, the corresponding unrecognized object result sending display interface is obtained to output the result.

[0146] This embodiment obtains a similarity result by calculating the similarity between the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database; sorts the similarity results and selects the similarity result with the highest sorting as the comparison similarity value; determines whether the comparison similarity value exceeds a preset similarity threshold; if the comparison similarity value exceeds the preset similarity threshold, determines the voice information to be identified as the voice object corresponding to the similarity result, and outputs the voice object as the voice recognition result; if the comparison similarity value does not exceed the preset similarity threshold, outputs a result of no recognition object. This effectively realizes accurate recognition of the voice object corresponding to the voice information to be identified.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0148] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0149] Further references Fig. 9 , as a response to the above Figure 1 The present application provides an embodiment of a speech recognition device, and the embodiment of the device is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0150] like Fig. 9 As shown, the speech recognition device 800 described in this embodiment includes: an information acquisition module 801, a sequence extraction module 802, a model training module 803, a model output module 804, a database construction module 805, a feature extraction module 806, and a result recognition module 807. Among them:

[0151] The information acquisition module 801 is used to acquire multilingual voice signal information and pre-process the multilingual voice signal information to obtain standard voice signal information;

[0152] A sequence extraction module 802 is used to extract features from the standard speech signal information to obtain an acoustic feature vector sequence;

[0153] A model training module 803, used for training a voiceprint embedding model based on the acoustic feature vector sequence;

[0154] The model output module 804 is used to obtain multi-language speech data and input the multi-language speech data into the voiceprint embedding model to obtain a voiceprint feature vector;

[0155] A database construction module 805 is used to construct a language voiceprint database based on the voiceprint feature vector;

[0156] A feature extraction module 806 is used to extract features of the speech information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized;

[0157] The result recognition module 807 is used to compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a speech recognition result.

[0158] By adopting the above-mentioned speech recognition device, this embodiment can obtain multi-language speech signal information, pre-process the multi-language speech signal information to obtain standard speech signal information; extract features from the standard speech signal information to obtain an acoustic feature vector sequence; train a voiceprint embedding model based on the acoustic feature vector sequence; obtain multi-language speech data, and input the multi-language speech data into the voiceprint embedding model to obtain a voiceprint feature vector; build a language voiceprint database based on the voiceprint feature vector; extract features from the speech information to be recognized based on the voiceprint embedding model to obtain a voiceprint feature vector to be recognized; compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a speech recognition result. This effectively realizes accurate recognition of the speech object of the speech information to be recognized, so as to improve the accuracy and security of the system speech recognition.

[0159] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0160] The computer device 9 includes a memory 91, a processor 92, and a network interface 93 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 9 with components 91-93, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0161] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0162] The memory 91 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 91 can be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 91 can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 91 can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 91 is generally used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions of the speech recognition method, etc. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or are to be output.

[0163] The processor 92 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 92 is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 92 is used to run computer-readable instructions or process data stored in the memory 91, such as computer-readable instructions for running the speech recognition method.

[0164] The network interface 93 may include a wireless network interface or a wired network interface. The network interface 93 is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0165] By using the above-mentioned computer device, this embodiment can obtain multilingual voice signal information, pre-process the multilingual voice signal information to obtain standard voice signal information; extract features from the standard voice signal information to obtain an acoustic feature vector sequence; train a voiceprint embedding model based on the acoustic feature vector sequence; obtain multilingual voice data, and input the multilingual voice data into the voiceprint embedding model to obtain a voiceprint feature vector; build a language voiceprint database based on the voiceprint feature vector; extract features from the voice information to be recognized based on the voiceprint embedding model to obtain a voiceprint feature vector to be recognized; compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a voice recognition result. This effectively realizes accurate recognition of the voice object of the voice information to be recognized, so as to improve the accuracy and security of the system voice recognition.

[0166] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the speech recognition method as described above.

[0167] By adopting the above-mentioned computer-readable storage medium, this embodiment can obtain multilingual voice signal information, preprocess the multilingual voice signal information, obtain standard voice signal information; extract features from the standard voice signal information to obtain an acoustic feature vector sequence; train a voiceprint embedding model based on the acoustic feature vector sequence; obtain multilingual voice data, and input the multilingual voice data into the voiceprint embedding model to obtain a voiceprint feature vector; build a language voiceprint database based on the voiceprint feature vector; extract features from the voice information to be recognized based on the voiceprint embedding model to obtain a voiceprint feature vector to be recognized; compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a voice recognition result. This effectively realizes accurate recognition of the voice object of the voice information to be recognized, so as to improve the accuracy and security of the system voice recognition.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0169] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0170] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A speech recognition method, characterized in that: The steps include: Acquiring multilingual voice signal information, and preprocessing the multilingual voice signal information to obtain standard voice signal information; Extracting features from the standard speech signal information to obtain an acoustic feature vector sequence; Training a voiceprint embedding model based on the acoustic feature vector sequence; Acquire multilingual speech data, and input the multilingual speech data into the voiceprint embedding model to obtain a voiceprint feature vector; Building a language voiceprint database based on the voiceprint feature vector; Extracting features of the voice information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized; The voiceprint feature vector to be recognized is compared and matched with the language voiceprint database to obtain a speech recognition result.

2. The speech recognition method according to claim 1, characterized in that: The step of acquiring multilingual voice signal information and preprocessing the multilingual voice signal information to obtain standard voice signal information specifically includes: Acquire a signal information extraction identifier, and extract the multilingual voice signal information from a database according to the signal information extraction identifier; Performing voice activity detection on the multilingual voice signal information to obtain valid voice signal information; Performing noise reduction processing on the effective voice signal information to obtain noise-reduced voice signal information; The noise reduction speech signal information is speech segmented according to a preset speech segmentation rule to obtain the standard speech signal information.

3. The speech recognition method according to claim 1, characterized in that: The step of extracting features from the standard speech signal information to obtain an acoustic feature vector sequence specifically includes: Dividing the standard voice signal information into units to obtain language phoneme units; Extracting features of the language phoneme unit to obtain a unit acoustic feature vector; The unit acoustic feature vectors are concatenated according to the corresponding order of the standard speech signal information to obtain the acoustic feature vector sequence.

4. The speech recognition method according to claim 1, characterized in that: The step of training the voiceprint embedding model based on the acoustic feature vector sequence specifically includes: Obtaining an initial model extraction identifier, and extracting an initial voiceprint embedding model from a database according to the initial model extraction identifier; Inputting the acoustic feature vector sequence into the initial voiceprint embedding model to obtain an initial voiceprint feature vector; A contrast loss calculation is performed on the initial voiceprint feature vector, and the initial voiceprint embedding model is optimized according to the calculated loss value to obtain an optimized voiceprint embedding model.

5. The speech recognition method according to claim 1, characterized in that: The step of acquiring multilingual speech data and inputting the multilingual speech data into the voiceprint embedding model to obtain a voiceprint feature vector specifically includes: Acquire a voice data extraction identifier, and extract the multilingual voice data from a preset corpus according to the voice data extraction identifier; Preprocessing the multilingual speech data to obtain multilingual speech segments; The multilingual speech segment is input into the voiceprint embedding model to obtain the voiceprint feature vector.

6. The speech recognition method according to claim 1, characterized in that: The step of constructing a language voiceprint database based on the voiceprint feature vector specifically includes: Acquire language tags corresponding to the multilingual speech data, classify the voiceprint feature vectors according to the language tags, and obtain a voiceprint feature vector library; Clustering the voiceprint feature vectors in the voiceprint feature vector library according to a preset clustering algorithm to obtain voiceprint feature vector clusters; Taking the cluster center of the voiceprint feature vector cluster as the voiceprint feature vector representative of the speech object, constructing a voiceprint dictionary of the speech object; The voiceprint feature vector library and the speech object voiceprint dictionary are merged to obtain the language voiceprint database.

7. The speech recognition method according to claim 1, characterized in that: The step of comparing and matching the voiceprint feature vector to be identified with the language voiceprint database to obtain a speech recognition result specifically includes: Calculating the similarity between the feature vector of the voiceprint to be identified and the feature vector in the language voiceprint database to obtain a similarity result; Sorting the similarity results, and selecting the highest ranked similarity result as the comparison similarity value; Determining whether the comparison similarity value exceeds a preset similarity threshold; If the comparison similarity value exceeds the preset similarity threshold, the speech information to be recognized is determined as the speech object corresponding to the similarity result, and the speech object is output as the speech recognition result.

8. A speech recognition device, characterized in that: include: An information acquisition module, used to acquire multilingual voice signal information and pre-process the multilingual voice signal information to obtain standard voice signal information; A sequence extraction module, used for extracting features from the standard speech signal information to obtain an acoustic feature vector sequence; A model training module, used for training a voiceprint embedding model based on the acoustic feature vector sequence; A model output module, used for acquiring multilingual speech data, and inputting the multilingual speech data into the voiceprint embedding model to obtain a voiceprint feature vector; A database construction module, used for constructing a language voiceprint database based on the voiceprint feature vector; A feature extraction module, used for extracting features of the speech information to be recognized based on the voiceprint embedding model to obtain a feature vector of the voiceprint to be recognized; The result recognition module is used to compare and match the voiceprint feature vector to be recognized with the language voiceprint database to obtain a speech recognition result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the speech recognition method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the speech recognition method according to any one of claims 1 to 7 are implemented.

Images