Biometric voice recognition method, device, computer equipment and storage medium

Abstract

The application relates to a voice recognition method and device based on biometric verification, computer equipment, a storage medium and a computer program product, and relates to the technical field of biometric recognition. The method comprises the following steps: after dividing the voice provided by a user during biometric verification into a plurality of voice segments to be recognized, a first voice recognition model in a voice recognition model library that matches the language of the first voice segment is determined; the first voice recognition model is used to sequentially recognize the plurality of voice segments to be recognized; when the first voice recognition model fails to recognize a voice segment, a second voice recognition model in the voice recognition model library that matches the language of the voice segment is determined; if the second voice recognition model is different from the first voice recognition model, all voice recognition models in the voice recognition model library are used to recognize the voice segment that has not been recognized. The method can meet the voice recognition requirement during multi-language biometric verification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biometric identification, and in particular to a voice recognition method and device for biometric verification, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] As a kind of biometric authentication technology, voice recognition plays an important auxiliary role in the prevention and control of resource risks in the financial field. Artificial intelligence voice recognition technology is widely used in scenarios such as login and payment of mobile banking. Text-related methods (text-related refers to focusing on the semantics and content of the speaker rather than the identity of the speaker) such as 8-digit random dynamic number string are used to collect complete voice and match each digit for biometric verification.

[0003] However, the voice recognition model based on 8-digit random dynamic numbers provided by the current technology usually only supports single-language voice recognition, which limits the use scenarios of voice recognition. SUMMARY

[0004] Therefore, it is necessary to provide a voice recognition method and device for biometric verification, a computer device, a computer readable storage medium, and a computer program product to solve the above technical problems.

[0005] In a first aspect, the present application provides a voice recognition method for biometric verification. The method comprises:

[0006] segmenting the voice to be recognized provided by a user during biometric verification into a plurality of voice segments to be recognized;

[0007] determining a first voice recognition model in a voice recognition model library that matches the language of a first voice segment in the plurality of voice segments to be recognized; the voice recognition model library comprises a plurality of voice recognition models respectively applicable to different languages;

[0008] performing voice recognition on the plurality of voice segments to be recognized in sequence using the first voice recognition model;

[0009] when the first voice recognition model fails to recognize a voice segment, determining a second voice recognition model in the voice recognition model library that matches the language of the voice segment;

[0010] if the second voice recognition model is the same as the first voice recognition model, using the first voice recognition model to continue voice recognition on the unrecognized voice segment;

[0011] if the second voice recognition model is different from the first voice recognition model, using all voice recognition models in the voice recognition model library to perform voice recognition on the unrecognized voice segment.

[0012] In one of the embodiments, the method further comprises:

[0013] When the first speech recognition model successfully recognizes the current speech segment, the first speech recognition model is continuously used to recognize the next speech segment.

[0014] In one of the embodiments, the method further comprises:

[0015] When the first speech recognition model fails to recognize a speech segment, if a speech recognition model matching the language of the speech segment is not successfully obtained from the speech recognition model library, the speech recognition of the speech to be recognized is ended.

[0016] In one of the embodiments, the method further comprises:

[0017] The biological verification text corresponding to the biological verification is obtained;

[0018] According to the biological verification text, the number of speech segment divisions acting on the speech to be recognized is determined;

[0019] According to the number of speech segment divisions, the speech to be recognized is divided into a plurality of speech segments to be recognized.

[0020] In one of the embodiments, the method further comprises:

[0021] The speech to be recognized is converted into a corresponding speech digital signal;

[0022] According to the number of speech segment divisions, the speech digital signal is divided to obtain a plurality of speech segments to be recognized.

[0023] In one of the embodiments, the method further comprises:

[0024] A speech training set corresponding to a target language is obtained from a pre-established speech library; the target language is a language matched by the speech recognition model to be trained; the speech library includes speech training sets corresponding to different languages respectively;

[0025] According to each text required for the biological verification, a speech training set corresponding to each text is extracted from the speech training set;

[0026] According to the speech training set corresponding to each text, the speech recognition model to be trained is trained.

[0027] In one of the embodiments, the method further comprises:

[0028] collecting speech produced when different speakers read the text required for the biometric verification using different languages respectively;

[0029] constructing the speech library according to the speech produced when different speakers read the text required for the biometric verification using different languages respectively.

[0030] In a second aspect, the present application further provides a speech recognition device for biometric verification. The device comprises:

[0031] a speech segmentation module configured to segment the speech to be recognized provided by the user during the biometric verification into a plurality of speech segments to be recognized;

[0032] a model matching module configured to determine a first speech recognition model in a speech recognition model library that matches the language of a first speech segment in the plurality of speech segments to be recognized; the speech recognition model library comprises a plurality of speech recognition models respectively applicable to different languages;

[0033] a speech recognition module configured to perform speech recognition on the plurality of speech segments to be recognized in sequence using the first speech recognition model;

[0034] a first recognition processing module configured to, when the first speech recognition model fails to recognize a speech segment, determine a second speech recognition model in the speech recognition model library that matches the language of the speech segment;

[0035] a second recognition processing module configured to, if the second speech recognition model is the same as the first speech recognition model, continue to perform speech recognition on the unrecognized speech segment using the first speech recognition model;

[0036] a third recognition processing module configured to, if the second speech recognition model is different from the first speech recognition model, perform speech recognition on the unrecognized speech segment using all the speech recognition models in the speech recognition model library.

[0037] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0038] segmenting the speech to be recognized provided by the user during the biometric verification into a plurality of speech segments to be recognized;

[0039] determining a first speech recognition model in a speech recognition model library that matches the language of a first speech segment in the plurality of speech segments to be recognized; the speech recognition model library comprises a plurality of speech recognition models respectively applicable to different languages;

[0040] performing speech recognition on the plurality of speech segments to be recognized in sequence using the first speech recognition model;

[0041] when the first speech recognition model fails to recognize a speech segment, determining a second speech recognition model in the speech recognition model library that matches the language of the speech segment;

[0042] if the second speech recognition model is the same as the first speech recognition model, continuing to use the first speech recognition model to recognize the unrecognized speech segment;

[0043] if the second speech recognition model is different from the first speech recognition model, using all speech recognition models in the speech recognition model library to recognize the unrecognized speech segment.

[0044] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the following steps:

[0045] segmenting the to-be-recognized speech provided by the user during the biometric verification into a plurality of to-be-recognized speech segments;

[0046] determining a first speech recognition model in the speech recognition model library that matches the language of a first speech segment in the plurality of to-be-recognized speech segments; the speech recognition model library includes a plurality of speech recognition models respectively suitable for different languages;

[0047] using the first speech recognition model to sequentially recognize the plurality of to-be-recognized speech segments;

[0048] when the first speech recognition model fails to recognize a speech segment, determining a second speech recognition model in the speech recognition model library that matches the language of the speech segment;

[0049] if the second speech recognition model is the same as the first speech recognition model, continuing to use the first speech recognition model to recognize the unrecognized speech segment;

[0050] if the second speech recognition model is different from the first speech recognition model, using all speech recognition models in the speech recognition model library to recognize the unrecognized speech segment.

[0051] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and the computer program, when executed by a processor, implements the following steps:

[0052] segmenting the to-be-recognized speech provided by the user during the biometric verification into a plurality of to-be-recognized speech segments;

[0053] determine a first speech recognition model in the speech recognition model library matching a language of a first speech segment in the plurality of speech segments to be recognized; the speech recognition model library comprises a plurality of speech recognition models respectively applicable to different languages;

[0054] perform speech recognition on the plurality of speech segments to be recognized in sequence using the first speech recognition model;

[0055] when the first speech recognition model fails to recognize a speech segment, determine a second speech recognition model in the speech recognition model library matching a language of the speech segment;

[0056] if the second speech recognition model is the same as the first speech recognition model, continue to perform speech recognition on the unrecognized speech segment using the first speech recognition model;

[0057] if the second speech recognition model is different from the first speech recognition model, perform speech recognition on the unrecognized speech segment using all speech recognition models in the speech recognition model library.

[0058] The above-mentioned biological verification speech recognition method, device, computer equipment, storage medium and computer program product cut the speech to be recognized into a plurality of speech segments, and the recognition result of each speech segment provides a reference for the recognition of subsequent speech segments; a first speech recognition model matching the language of the first speech segment is determined, and the first speech recognition model is used to perform speech recognition on the plurality of speech segments in sequence, thereby reducing the language matching cost of speech segments of the same language; when the first speech recognition model fails to recognize a speech segment, a second speech recognition model matching the language of the speech segment is determined; if the second speech recognition model is the same as the first speech recognition model, the first speech recognition model is continuously used to complete speech recognition. If the second speech recognition model is different from the first speech recognition model, speech recognition is performed on the unrecognized speech segment using speech recognition models of all languages. Through the above-mentioned biological verification speech recognition method, not only can the user use multiple languages for biological verification speech recognition, but also the speech recognition efficiency of the user using a single language for biological verification can be improved, thereby meeting the biological verification speech recognition in multiple scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 An application environment diagram of the biological verification speech recognition method in one embodiment;

[0060] Figure 2 A flowchart of the biological verification speech recognition method in one embodiment;

[0061] Figure 3 A flowchart of the speech recognition model training step in one embodiment;

[0062] Figure 4 This is a complete flowchart of a speech recognition method for biometric verification in another embodiment;

[0063] Figure 5 This is a structural block diagram of a biometric verification speech recognition device in one embodiment;

[0064] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] In financial scenarios, there are measures to prevent fraud risks. For example, when logging into or making payments on mobile banking, mobile banking can provide biometric verification. Currently, this is achieved using a speech recognition model, but it is usually a monolingual speech recognition model. Taking a speech recognition model that supports Mandarin as an example, users can only be correctly recognized when they speak Mandarin. This is inconvenient for users in non-Mandarin-speaking regions or foreign language users, which limits the technical scenarios in which this technology can be used.

[0067] To address this technical problem, this application provides a speech recognition method for biometric verification, which not only enables speech recognition for users performing biometric verification in multiple languages, but also improves the speech recognition efficiency when users perform biometric verification in a single language, thus meeting the needs of speech recognition for biometric verification in various scenarios. For example... Figure 1 As shown, this method can be applied to, for example Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 obtains the voice to be recognized provided by the user during biometric verification. Server 104 obtains the voice to be recognized from terminal 102, segments the voice, and then performs voice recognition processing on multiple voice segments based on the voice recognition model library in the data storage system. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0068] The following detailed description of the speech recognition method for biometric verification of this application is provided through various embodiments and corresponding figures.

[0069] In one embodiment, such as Figure 2 As shown, a biometric verification speech recognition method is provided, which can be applied to, for example... Figure 1 In server 104, the method may include the following steps:

[0070] Step S201: The voice to be recognized provided by the user during biometric verification is segmented into multiple voice segments to be recognized.

[0071] It should be noted that when a user performs biometric verification, randomly generated text is displayed. The user reads the content of the text aloud, and the user's voice is used as the voice to be recognized to complete the biometric verification.

[0072] For example, the speech to be recognized read by the user during biometric verification is obtained. Before speech recognition is performed on the speech to be recognized, preprocessing is performed: endpoint detection is performed on the speech to be recognized based on the short-time energy of the speech. Based on the detected endpoints, the speech to be recognized is segmented to obtain multiple speech segments to be recognized.

[0073] Step S202: Determine the first speech recognition model in the speech recognition model library that matches the language of the first speech segment among multiple speech segments to be recognized; the speech recognition model library includes multiple speech recognition models that are applicable to different languages.

[0074] The first speech segment is the speech segment that is the first in time among multiple speech segments to be recognized.

[0075] The languages ​​can include Mandarin, English, and local dialects (such as Cantonese).

[0076] For example, the first speech segment from multiple speech samples to be recognized is obtained, and speech feature extraction processing is performed on the first speech segment to obtain corresponding feature information. Based on the feature information, all speech recognition models in the speech recognition model library can be used to determine the recognition result of the first speech segment in different languages. The recognition result includes the text content of the speech and the corresponding recognition probability. The target recognition result with the highest recognition probability among the recognition results of the first speech segment is determined, and the language corresponding to the target recognition result is determined as the language of the first speech segment. At the same time, the speech recognition model corresponding to the target recognition result is determined as the first speech recognition model. Further, a probability threshold can be preset. If the recognition probability in the target recognition result is lower than the probability threshold, it may be because the language used by the user does not have a corresponding speech recognition model in the speech recognition library, and the speech recognition is considered to have failed.

[0077] Step S203: Use the first speech recognition model to perform speech recognition on multiple speech segments in sequence.

[0078] For example, the first speech recognition model is used to perform speech recognition on multiple speech segments to be recognized in a time sequence.

[0079] Step S204: When the first speech recognition model fails to recognize a speech segment, determine a second speech recognition model in the speech recognition model library that matches the language of the speech segment.

[0080] For example, when performing speech recognition on multiple speech segments to be recognized in sequence, if the first speech recognition model fails to recognize a certain speech segment, a second speech recognition model that matches the language of the speech segment to be recognized is determined from the speech recognition model library.

[0081] Step S205: If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments.

[0082] For example, when the second speech recognition model is the same as the first speech recognition model, the first speech recognition model is used to perform speech recognition on the unrecognized speech segments in turn; if the recognition fails again, step S204 is executed again.

[0083] Step S206: If the second speech recognition model is different from the first speech recognition model, then all speech recognition models in the speech recognition model library are used to perform speech recognition on the unrecognized speech segments.

[0084] For example, when the second speech recognition model is different from the first speech recognition model, it is assumed that the user has used multiple languages ​​for biometric verification. Then, all speech recognition models in the speech recognition model library are used simultaneously to perform speech recognition on the unrecognized speech segments, and the recognition results of each speech segment in multiple languages ​​are obtained. The speech recognition result of each speech segment is determined based on the recognition results in multiple languages.

[0085] In the aforementioned speech recognition method for biometric verification, the speech to be recognized is segmented into multiple speech segments, and the recognition result of each speech segment provides a reference for the recognition of subsequent speech segments. A first speech recognition model matching the language of the first speech segment is determined, and the first speech recognition model is used to sequentially perform speech recognition on multiple speech segments, reducing the language matching cost for speech segments of the same language. When the first speech recognition model fails to recognize a speech segment, a second speech recognition model matching the language of that speech segment is determined. If the second speech recognition model is the same as the first speech recognition model, the first speech recognition model is used to complete the speech recognition. If the second speech recognition model is different from the first speech recognition model, speech recognition models for all languages ​​are used to perform speech recognition on the unrecognized speech segments. Through the above speech recognition method for biometric verification, not only can users perform speech recognition for biometric verification using multiple languages, but the speech recognition efficiency for users performing biometric verification using a single language can also be improved, meeting the speech recognition needs of biometric verification in various scenarios.

[0086] In one embodiment, the above-described biometric verification speech recognition method further includes the following steps:

[0087] When the first speech recognition model successfully recognizes the current speech segment, it continues to use the first speech recognition model to recognize the next speech segment.

[0088] The current speech segment is the speech segment being recognized when the first speech recognition model sequentially performs speech recognition on multiple speech segments to be recognized.

[0089] For example, when the first speech recognition model performs speech recognition on multiple speech segments to be recognized in sequence, and the current speech segment can be recognized successfully, the first speech recognition model is still used when recognizing the next speech segment, and there is no need to reselect a speech recognition model.

[0090] In this embodiment, when a user provides the speech to be recognized in one language, after matching the language of the first speech segment, the same speech recognition model can be directly used for other speech segments, which effectively improves the efficiency of speech recognition.

[0091] In one embodiment, the above-described biometric verification speech recognition method further includes the following steps:

[0092] If the first speech recognition model fails to recognize a speech segment, and if it fails to obtain a speech recognition model that matches the language of the speech segment from the speech recognition model library, then the speech recognition of the speech to be recognized ends.

[0093] For example, when the first speech recognition model performs speech recognition on multiple speech segments to be recognized in sequence, fails to recognize the current speech segment, and cannot obtain a speech recognition model that matches the language of the current speech segment from the speech recognition model library based on the current speech segment, then the speech recognition of the speech provided by the user is terminated, and it is determined that the speech recognition and biometric verification have failed.

[0094] In one embodiment, step S101 above, which segments the voice to be recognized provided by the user during biometric verification into multiple voice segments, can also be achieved through the following steps:

[0095] Obtain the biometric verification text corresponding to the biometric verification; determine the number of speech segments to be segmented based on the biometric verification text; and segment the speech to be recognized into multiple speech segments based on the number of speech segments.

[0096] For example, during biometric verification, the computer displays randomly generated text, which serves as the biometric verification text. Based on the number of characters in the biometric verification text, the number of speech segments to be segmented for the speech to be recognized is determined. For instance, if the biometric verification text is "0154", the number of segments is 4; if the biometric verification text is "speech recognition method", the number of segments is 6. After determining the number of segments, the speech to be recognized is divided into the corresponding number of speech segments.

[0097] Based on the above embodiments, further, the above-mentioned segmentation of the speech to be recognized into multiple speech segments according to the number of speech segments can also be achieved through the following steps:

[0098] The speech to be recognized is converted into a corresponding digital speech signal; the digital speech signal is then segmented according to the number of speech segments to obtain multiple speech segments to be recognized.

[0099] For example, the speech to be recognized provided by the user may be an analog signal in the time domain. The analog signal needs to be converted to a digital signal. Based on the digital signal, feature information is extracted. Then, according to the feature information and the number of segments, the speech is segmented into multiple speech segments to be recognized.

[0100] In this embodiment, the speech to be recognized is segmented based on the biometric verification text speech digital signal to obtain multiple speech segments to be recognized. The user-provided speech to be recognized is transformed into isolated word speech recognition, eliminating the need to consider the contextual relationships within the speech segments and improving speech recognition efficiency. Simultaneously, the language recognition result of the previous speech segment can provide language references for subsequent speech segments, improving the recognition efficiency when using speech provided in the same language.

[0101] In one embodiment, such asFigure 3 As shown in Figure 3 , the above-mentioned voice recognition method for biometric verification further includes the following steps for training the model:

[0102] Step S301: Obtain the voice training set corresponding to the target language from a pre-established voice library; the target language is the language matched by the voice recognition model to be trained; the voice library includes voice training sets corresponding to different languages respectively;

[0103] Step S302: Extract the voice training set corresponding to each text from the voice training set according to each text required for biometric verification;

[0104] Step S303: Train the voice recognition model to be trained according to the voice training set corresponding to each text.

[0105] Exemplarily, determine the language of the voice recognition model to be trained, and screen out the voice training set of the target language from the pre-established voice library. The voice training set contains voice data and text data corresponding to the words expressed by the voice data. For example, the text corresponding to the voice of "one" read in English is "1", and the text corresponding to the voice of "一" read in Chinese is also "1". According to the requirements of biometric verification for biometric verification texts, and according to the text data in the voice training set, screen out the voice training set required for biometric verification texts from the voice training set. Use the required voice training set as the training set of the voice recognition model to be trained, and train the voice recognition model to be trained.

[0106] For example, the voice training sets of languages such as Mandarin, English, and Cantonese are stored in the voice library. The current biometric verification is the voice recognition of digital words, that is, the random text displayed by the computer during biometric verification is a digital string, such as "0146", "456413", and "554546", etc. According to the current Mandarin voice recognition model to be trained, screen out the voice training set of reading numbers in Mandarin from the voice library as the training set of the Mandarin voice recognition model to be trained; similarly, the voice recognition models of other languages are also trained through the above steps.

[0107] In one embodiment, the above-mentioned voice recognition method for biometric verification further includes the following steps:

[0108] Collect the voices generated when different speakers read the texts required for biometric verification in different languages respectively; construct a voice library according to the voices generated when different speakers read the texts required for biometric verification in different languages respectively.

[0109] For example, all possible biometric verification texts are identified during biometric verification, and the speech generated when different speakers read the biometric verification texts in different languages ​​is collected. A mapping relationship is established between the speech and the corresponding language and text data, and finally a speech library is constructed based on the speech and the mapping relationship.

[0110] In this embodiment, a speech database is constructed by collecting the voices of different speakers. This speech database serves as the training set for the subsequent speech recognition model, allowing the trained speech recognition model to ignore the speaker's voiceprint information in the speech and focus only on the text-related features of the speech, thus achieving text-related speech recognition.

[0111] In another embodiment, such as Figure 4 As shown, a biometric verification speech recognition method is provided, including the following steps:

[0112] Step S401: Obtain the biometric verification text corresponding to the biometric verification, and determine the number of speech segment segments to be applied to the speech to be recognized based on the biometric verification text.

[0113] Step S402: Convert the voice to be recognized provided by the user during biometric verification into a corresponding voice digital signal.

[0114] Step S403: The speech digital signal is segmented according to the number of speech segments to obtain multiple speech segments to be recognized.

[0115] Step S404: Determine the first speech recognition model in the speech recognition model library that matches the language of the first speech segment among multiple speech segments to be recognized.

[0116] Step S405: Use the first speech recognition model to sequentially perform speech recognition on multiple speech segments to be recognized.

[0117] Step S406: When the first speech recognition model successfully recognizes the current speech segment, continue to use the first speech recognition model to perform speech recognition on the next speech segment.

[0118] Step S407: When the first speech recognition model fails to recognize a speech segment, determine a second speech recognition model in the speech recognition model library that matches the language of the speech segment.

[0119] Step S408: If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments.

[0120] Step S409: If the second speech recognition model is different from the first speech recognition model, then use all speech recognition models in the speech recognition model library to perform speech recognition on the unrecognized speech segments.

[0121] Step S410: If a speech recognition model matching the language of the speech segment is not successfully obtained from the speech recognition model library, then the speech recognition of the speech to be recognized ends.

[0122] In this embodiment, by segmenting the user-provided speech to be recognized into multiple speech segments, the speech recognition of the speech to be recognized is converted into isolated word speech recognition, thereby improving recognition efficiency. Then, a first speech recognition model is determined based on the language matching result of the first speech segment to be recognized. If the languages ​​of the remaining speech segments to be recognized are all the same as those of the first speech segment, then the first speech recognition model is used to complete the speech recognition of all speech segments to be recognized, reducing the cost of language matching and improving the efficiency of speech recognition. If the speech segments to be recognized are in two or more languages, speech recognition can also be achieved using the above method, meeting the needs of biometric verification speech recognition in multilingual scenarios. At the same time, when the first speech recognition model fails, all speech recognition models are directly used for the speech recognition of subsequent speech segments to be recognized, which also reduces the cost of language matching to some extent.

[0123] To facilitate understanding of the embodiments of this application by those skilled in the art, the following will describe this application using a specific example of a biometric verification speech recognition method, first describing its training and then its application:

[0124] In the training process of the speech recognition model—a hybrid model of HMM (Hidden Markov Model) and GMM (Gaussian Mixture Model)—the mean and variance of the GMM model are initially adjusted based on the K-means algorithm. Then, the parameters of the entire speech recognition model are adjusted according to the EM algorithm (Expectation Maximization algorithm) to obtain the trained speech recognition model. When a user logs in or initiates a transaction in mobile banking, the mobile terminal displays randomly generated biometric verification text, such as "514623". The user provides the speech to be recognized based on the biometric verification text. After the mobile terminal acquires the continuous analog signal of the speech to be recognized in the time domain at a sampling rate of 8kHz, the backend server determines the discrete analog signal from the continuous analog signal based on Shannon sampling theorem and converts it into a digital signal in the frequency domain using 16-bit quantization. The energy of the start and end points of different digits of the speech is concentrated in a small range. The speech to be recognized is segmented using an endpoint detection method based on short-time energy, resulting in 6 segments of speech to be recognized. The speech signal is segmented into frames based on its short-term stationarity. Then, typical speech features MFCC (Mel Frequency Cepstrum Coefficient) are extracted as speech feature information. The speech feature information is input into the speech recognition model. The probability of the speech segment to be recognized belonging to a certain number is obtained according to the Viterbi algorithm. The number with the highest probability is taken as the recognition result of the speech segment to be recognized.

[0125] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0126] Based on the same inventive concept, this application also provides a biometric verification speech recognition device for implementing the biometric verification speech recognition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more biometric verification speech recognition device embodiments provided below can be found in the limitations of the biometric verification speech recognition method described above, and will not be repeated here.

[0127] In one embodiment, such as Figure 5 As shown, a biometric verification speech recognition device is provided, including: a speech segmentation module 501, a model matching module 502, a speech recognition module 503, a first recognition processing module 504, a second recognition processing module 505, and a third recognition processing module 506, wherein:

[0128] The speech segmentation module 501 is used to segment the speech to be recognized provided by the user during biometric verification into multiple speech segments to be recognized.

[0129] The model matching module 502 is used to determine the first speech recognition model in the speech recognition model library that matches the language of the first speech segment among multiple speech segments to be recognized; the speech recognition model library includes multiple speech recognition models that are applicable to different languages.

[0130] The speech recognition module 503 is used to sequentially perform speech recognition on multiple speech segments to be recognized using the first speech recognition model.

[0131] The first recognition processing module 504 is used to determine a second speech recognition model in the speech recognition model library that matches the language of the speech segment when the first speech recognition model fails to recognize a speech segment.

[0132] The second recognition processing module 505 is used to continue to perform speech recognition on unrecognized speech segments using the first speech recognition model when the second speech recognition model is the same as the first speech recognition model.

[0133] The third recognition processing module 506 is used to perform speech recognition on unrecognized speech segments by using all speech recognition models in the speech recognition model library when the second speech recognition model is different from the first speech recognition model.

[0134] In one embodiment, the speech recognition module 503 is further configured to continue using the first speech recognition model to perform speech recognition on the next speech segment if the first speech recognition model successfully recognizes the current speech segment.

[0135] In one embodiment, the first recognition processing module 504 is further configured to terminate the speech recognition of the speech to be recognized when the first speech recognition model fails to recognize a speech segment and fails to obtain a speech recognition model that matches the language of the speech segment from the speech recognition model library.

[0136] In one embodiment, the speech segmentation module 501 is further configured to acquire the biometric verification text corresponding to the biometric verification; determine the number of speech segments to be segmented for the speech to be recognized based on the biometric verification text; and segment the speech to be recognized into multiple speech segments to be recognized based on the number of speech segments.

[0137] In one embodiment, the speech segmentation module 501 is further configured to convert the speech to be recognized into a corresponding digital speech signal; and to segment the digital speech signal according to the number of speech segments to obtain multiple speech segments to be recognized.

[0138] In one embodiment, the aforementioned speech recognition device for biometric verification further includes a model training module, used to obtain a speech training set corresponding to a target language from a pre-established speech library; the target language is the language matched by the speech recognition model to be trained; the speech library includes speech training sets corresponding to different languages; according to each text required for biometric verification, the speech training set corresponding to each text is extracted from the speech training set; and the speech recognition model to be trained is trained according to the speech training set corresponding to each text.

[0139] In one embodiment, the speech recognition device for biometric verification further includes a speech library building module, which is used to collect the speech generated when different speakers read the text required for biometric verification in different languages; and to build a speech library based on the speech generated when different speakers read the text required for biometric verification in different languages.

[0140] The various modules in the aforementioned biometric verification voice recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0141] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores speech recognition model data in multiple languages. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a biometric-verified speech recognition method.

[0142] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0144] The voice to be recognized provided by the user during biometric verification is segmented into multiple voice segments to be recognized;

[0145] The first speech recognition model in the speech recognition model library is selected to match the language of the first speech segment among multiple speech segments to be recognized; the speech recognition model library includes multiple speech recognition models applicable to different languages.

[0146] The first speech recognition model is used to sequentially perform speech recognition on multiple speech segments to be recognized.

[0147] When the first speech recognition model fails to recognize a speech segment, a second speech recognition model in the speech recognition model library that matches the language of the speech segment is determined.

[0148] If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments;

[0149] If the second speech recognition model is different from the first speech recognition model, then all speech recognition models in the speech recognition model library are used to perform speech recognition on the unrecognized speech segments.

[0150] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the first speech recognition model successfully recognizes the current speech segment, it continues to use the first speech recognition model to perform speech recognition on the next speech segment.

[0151] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the first speech recognition model fails to recognize a speech segment, if a speech recognition model matching the language of the speech segment is not successfully obtained from the speech recognition model library, then the speech recognition of the speech to be recognized is terminated.

[0152] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining the biometric verification text corresponding to the biometric verification; determining the number of speech segment segments to be applied to the speech to be recognized based on the biometric verification text; and segmenting the speech to be recognized into multiple speech segments to be recognized based on the number of speech segment segments.

[0153] In one embodiment, when the processor executes the computer program, it further performs the following steps: converting the speech to be recognized into a corresponding digital speech signal; and segmenting the digital speech signal according to the number of speech segments to obtain multiple speech segments to be recognized.

[0154] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining a speech training set corresponding to the target language from a pre-established speech library; the target language is the language matched by the speech recognition model to be trained; the speech library includes speech training sets corresponding to different languages; extracting the speech training set corresponding to each text from the speech training set according to each text required for biometric verification; and training the speech recognition model to be trained according to the speech training set corresponding to each text.

[0155] In one embodiment, the processor, when executing the computer program, further performs the following steps: collecting the speech generated when different speakers read the text required for biometric verification in different languages; and constructing a speech database based on the speech generated when different speakers read the text required for biometric verification in different languages.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0157] The voice to be recognized provided by the user during biometric verification is segmented into multiple voice segments to be recognized;

[0158] The first speech recognition model in the speech recognition model library is selected to match the language of the first speech segment among multiple speech segments to be recognized; the speech recognition model library includes multiple speech recognition models applicable to different languages.

[0159] The first speech recognition model is used to sequentially perform speech recognition on multiple speech segments to be recognized.

[0160] When the first speech recognition model fails to recognize a speech segment, a second speech recognition model in the speech recognition model library that matches the language of the speech segment is determined.

[0161] If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments;

[0162] If the second speech recognition model is different from the first speech recognition model, then all speech recognition models in the speech recognition model library are used to perform speech recognition on the unrecognized speech segments.

[0163] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the first speech recognition model successfully recognizes the current speech segment, it continues to use the first speech recognition model to perform speech recognition on the next speech segment.

[0164] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the first speech recognition model fails to recognize a speech segment, if a speech recognition model matching the language of the speech segment is not successfully obtained from the speech recognition model library, then the speech recognition of the speech to be recognized is terminated.

[0165] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the biometric verification text corresponding to the biometric verification; determining the number of speech segment segments to be applied to the speech to be recognized based on the biometric verification text; and segmenting the speech to be recognized into multiple speech segments to be recognized based on the number of speech segment segments.

[0166] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: converting the speech to be recognized into a corresponding digital speech signal; and segmenting the digital speech signal according to the number of speech segments to obtain multiple speech segments to be recognized.

[0167] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a speech training set corresponding to the target language from a pre-established speech library; the target language is the language matched by the speech recognition model to be trained; the speech library includes speech training sets corresponding to different languages; extracting the speech training set corresponding to each text from the speech training set according to each text required for biometric verification; and training the speech recognition model to be trained according to the speech training set corresponding to each text.

[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting the speech generated when different speakers read the text required for biometric verification in different languages; and constructing a speech database based on the speech generated when different speakers read the text required for biometric verification in different languages.

[0169] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0170] The voice to be recognized provided by the user during biometric verification is segmented into multiple voice segments to be recognized;

[0171] The first speech recognition model in the speech recognition model library is selected to match the language of the first speech segment among multiple speech segments to be recognized; the speech recognition model library includes multiple speech recognition models applicable to different languages.

[0172] The first speech recognition model is used to sequentially perform speech recognition on multiple speech segments to be recognized.

[0173] When the first speech recognition model fails to recognize a speech segment, a second speech recognition model in the speech recognition model library that matches the language of the speech segment is determined.

[0174] If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments;

[0175] If the second speech recognition model is different from the first speech recognition model, then all speech recognition models in the speech recognition model library are used to perform speech recognition on the unrecognized speech segments.

[0176] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the first speech recognition model successfully recognizes the current speech segment, it continues to use the first speech recognition model to perform speech recognition on the next speech segment.

[0177] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the first speech recognition model fails to recognize a speech segment, if a speech recognition model matching the language of the speech segment is not successfully obtained from the speech recognition model library, then the speech recognition of the speech to be recognized is terminated.

[0178] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining the biometric verification text corresponding to the biometric verification; determining the number of speech segment segments to be applied to the speech to be recognized based on the biometric verification text; and segmenting the speech to be recognized into multiple speech segments to be recognized based on the number of speech segment segments.

[0179] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: converting the speech to be recognized into a corresponding digital speech signal; and segmenting the digital speech signal according to the number of speech segments to obtain multiple speech segments to be recognized.

[0180] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a speech training set corresponding to the target language from a pre-established speech library; the target language is the language matched by the speech recognition model to be trained; the speech library includes speech training sets corresponding to different languages; extracting the speech training set corresponding to each text from the speech training set according to each text required for biometric verification; and training the speech recognition model to be trained according to the speech training set corresponding to each text.

[0181] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting the speech generated when different speakers read the text required for biometric verification in different languages; and constructing a speech database based on the speech generated when different speakers read the text required for biometric verification in different languages.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0183] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0184] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A biometric verification speech recognition method, characterized in that, The method includes: The voice to be recognized provided by the user during biometric verification is segmented into multiple voice segments to be recognized; A first speech recognition model is determined from the speech recognition model library that matches the language of the first speech segment among the plurality of speech segments to be recognized; the speech recognition model library includes multiple speech recognition models applicable to different languages. The first speech recognition model is used to sequentially perform speech recognition on the plurality of speech segments to be recognized; When the first speech recognition model fails to recognize a speech segment, a second speech recognition model in the speech recognition model library that matches the language of the speech segment is determined. The recognition failure means that the recognition probability in the target recognition result is lower than the probability threshold. If the second speech recognition model is the same as the first speech recognition model, then the first speech recognition model is used to continue to perform speech recognition on the unrecognized speech segments; If the second speech recognition model is different from the first speech recognition model, then all speech recognition models in the speech recognition model library are used simultaneously to perform speech recognition on the unrecognized speech segments, and the recognition results of each speech segment in multiple languages ​​are obtained. The speech recognition result of each speech segment is determined based on the recognition results in multiple languages.

2. The method according to claim 1, characterized in that, The method further includes: When the first speech recognition model successfully recognizes the current speech segment, it continues to use the first speech recognition model to recognize the next speech segment.

3. The method according to claim 1, characterized in that, The method further includes: When the first speech recognition model fails to recognize a speech segment, if it fails to obtain a speech recognition model that matches the language of the speech segment from the speech recognition model library, then the speech recognition of the speech to be recognized ends.

4. The method according to claim 1, characterized in that, The process of segmenting the voice to be recognized provided by the user during biometric verification into multiple voice segments includes: Obtain the biometric verification text corresponding to the biometric verification; Based on the biometric verification text, determine the number of speech segment segments applied to the speech to be identified; Based on the number of speech segments, the speech to be recognized is divided into multiple speech segments to be recognized.

5. The method according to claim 4, characterized in that, The step of dividing the speech to be recognized into multiple speech segments according to the number of speech segments includes: The speech to be recognized is converted into a corresponding digital speech signal; The speech digital signal is segmented according to the number of speech segments to obtain multiple speech segments to be identified.

6. The method according to claim 1, characterized in that, The method further includes: Obtain a speech training set corresponding to the target language from a pre-established speech database; the target language is the language matched by the speech recognition model to be trained; the speech database includes speech training sets corresponding to different languages ​​respectively; Based on the texts required for the biometric verification, the corresponding speech training sets for each text are extracted from the speech training set. The speech recognition model to be trained is trained based on the speech training set corresponding to each text.

7. The method according to claim 6, characterized in that, The method further includes: Collect audio recordings generated when different speakers read the text required for biometric verification in different languages; The speech library is constructed based on the speech generated when different speakers read the text required for biometric verification in different languages.

8. A biometric verification voice recognition device, characterized in that, The device includes: The speech segmentation module is used to segment the speech to be recognized provided by the user during biometric verification into multiple speech segments to be recognized; The model matching module is used to determine the first speech recognition model in the speech recognition model library that matches the language of the first speech segment among the plurality of speech segments to be recognized; the speech recognition model library includes multiple speech recognition models that are applicable to different languages. The speech recognition module is used to sequentially perform speech recognition on the plurality of speech segments to be recognized using the first speech recognition model; The first recognition processing module is used to determine a second speech recognition model in the speech recognition model library that matches the language of the speech segment when the first speech recognition model fails to recognize a speech segment. The recognition failure means that the recognition probability in the target recognition result is lower than the probability threshold. The second recognition processing module is used to continue to perform speech recognition on the unrecognized speech segment using the first speech recognition model if the second speech recognition model is the same as the first speech recognition model. The third recognition processing module is used to simultaneously use all speech recognition models in the speech recognition model library to perform speech recognition on the unrecognized speech segments if the second speech recognition model is different from the first speech recognition model, so as to obtain the recognition results of each speech segment in multiple languages, and determine the speech recognition result of each speech segment based on the recognition results in multiple languages.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images