Voice verification method, device, electronic device and medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses a voice verification method, including: identifying original voice data to obtain multiple voice texts; standardizing comparison texts and multiple voice texts to obtain standard comparison texts and multiple standard voice texts; using a pre-trained entity recognition model to identify entities contained in the standard comparison text and multiple standard voice texts, and obtaining a standard text entity set and multiple voice text entity sets; calculating the similarity between the standard text entity set and each voice text entity set after pre-alignment, and taking the voice text entity set with the highest similarity as the standard corresponding entity set; calculating the total similarity score of the standard corresponding entity set and the standard text entity set based on a preset similarity formula, and comparing according to the similarity total score and a preset score threshold to realize voice verification. The present invention also relates to blockchain technology, and the comparison text can be stored in a blockchain node. The present invention can improve the accuracy of text entity recognition in speech.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a voice verification method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the further advancement of artificial intelligence, artificial intelligence is being used to replace it in the development of many online businesses, especially in the stage of calling back customers for review after the information review. Artificial intelligence is used to communicate with customers to reconfirm whether the person and the information are true and consistent, and finally determine whether the business approval is passed.

[0003] When existing voice verification technology converts customer oral content into text information, it is difficult to accurately identify text entities in the voice because the customer's oral review will contain short strings of sentences with poor semantics (such as text entities such as the customer's unit name and work address). Summary of the Invention

[0004] The present invention provides a speech verification method, device, electronic device and computer-readable storage medium, the main purpose of which is to improve the accuracy of text entity recognition in speech.

[0005] To achieve the above objectives, the present invention provides a voice verification method comprising:

[0006] Acquiring original voice data, and recognizing the original voice data using a preset voice recognition method to obtain multiple voice texts;

[0007] Obtaining a comparison text, and performing standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts;

[0008] Using a pre-trained entity recognition model, identifying entities contained in the standard comparison text and the plurality of standard voice texts, to obtain a standard text entity set and a plurality of voice text entity sets;

[0009] Aligning each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculating the similarity between the aligned standard text entity set and each speech text entity set, and taking the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set;

[0010] Calculating a total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula;

[0011] If the total similarity score is less than the preset score threshold, it is determined that the voice verification fails;

[0012] If the total similarity score is greater than or equal to the score threshold, it is determined that the voice verification has passed.

[0013] Optionally, the using a preset speech recognition method to recognize the original speech data to obtain a plurality of speech texts includes:

[0014] Denoising the original speech data to obtain denoised speech data;

[0015] Extracting speech features from the noise-reduced speech data using a speech feature extraction algorithm to obtain speech data features;

[0016] Build a speech recognition network using pre-trained acoustic models, pre-trained language models, and dictionaries;

[0017] The speech data features are recognized by the speech recognition network to obtain multiple speech texts.

[0018] Optionally, reducing noise from the original voice data to obtain noise-reduced voice data includes:

[0019] Decompressing the original voice data to obtain a waveform file of the original voice data;

[0020] Silence removal is performed on the waveform file of the original speech data to obtain noise-reduced speech data.

[0021] Optionally, the extracting speech features from the noise-reduced speech data using a speech feature extraction algorithm to obtain speech data features includes:

[0022] Pre-emphasize the noise reduction voice data to obtain emphasized voice data;

[0023] Framing and windowing the aggravated speech data to obtain a speech waveform;

[0024] The filter in the speech feature extraction algorithm is used to extract the features of the speech waveform to obtain speech data features.

[0025] Optionally, the standardizing the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts includes:

[0026] Standardizing the formats of the comparison text and the plurality of voice texts to obtain a standard format text and a plurality of standard format voice texts;

[0027] Performing word segmentation and part-of-speech tagging on the standard format text and each of the standard format voice texts, and removing non-keywords obtained by part-of-speech tagging to obtain a standard text word set and multiple voice text word sets;

[0028] Deduplication operations are performed on the words in the standard text word set and the words in each of the voice text word sets to obtain a deduplication standard word set and multiple deduplication voice word sets, and the words in the deduplication standard word set and the words in each of the deduplication voice word sets are combined to obtain the standard comparison text and multiple standard voice texts.

[0029] Optionally, aligning each standard text entity in the standard text entity set with each voice text entity in all the voice text entity sets, and calculating the similarity between the aligned standard text entity set and each voice text entity set, includes:

[0030] Inputting each standard text entity in the standard text entity set and each speech text entity in all the speech text entity sets into a pre-built multilingual BERT model to obtain a standard entity matrix and multiple semantic entity matrices;

[0031] Multiplying the standard entity matrix and each of the semantic entity matrices to obtain a multiplication matrix, and normalizing the multiplication matrix to obtain a plurality of aligned entity groups;

[0032] The similarity of each of the aligned entity groups is calculated using a cosine similarity formula to obtain the similarity between the standard text entity set and each of the voice text entity sets.

[0033] Optionally, the calculating the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula includes:

[0034] Dividing the plurality of standard corresponding entities in the standard corresponding entity set to obtain prefix entities, suffix entities, address entities, and name entities;

[0035] Searching for the name entity and the address entity in the standard text entity set;

[0036] If the same name entity and address entity exist in the standard text entity set, determining a similarity score between the name entity and the address entity;

[0037] If the same name entity and the address entity do not exist in the standard text entity set, obtaining the corresponding name entity and the corresponding address entity from the standard text entity set, converting the name entity, the address entity, the corresponding name entity, and the corresponding address entity into pinyin, and calculating the similarity between the name entity and the corresponding name entity, and between the address entity and the corresponding address entity using an edit distance formula to obtain a similarity score between the name entity and the address entity;

[0038] Acquire the corresponding prefix entity and the corresponding suffix entity from the standard text entity set based on the prefix entity and the suffix entity;

[0039] Converting the prefix entity and the corresponding prefix entity, as well as the suffix entity and the corresponding suffix entity, into word vectors, and calculating the similarity between the prefix entity and the corresponding prefix entity, as well as between the suffix entity and the corresponding suffix entity, using a cosine similarity formula to obtain a similarity score between the prefix entity and the suffix entity;

[0040] The total similarity score of the name entity, the address entity, the prefix entity and the suffix entity is calculated using the total similarity score formula, and the total similarity score is used as the total similarity score of the standard corresponding entity set and the standard text entity set.

[0041] In order to solve the above problems, the present invention further provides a voice verification device, comprising:

[0042] A speech recognition module is used to obtain original speech data, recognize the original speech data using a preset speech recognition method, and obtain multiple speech texts;

[0043] A text comparison module is used to obtain a comparison text and perform standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts;

[0044] An entity recognition module is used to use a pre-trained entity recognition model to identify entities contained in the standard comparison text and the plurality of standard voice texts, thereby obtaining a standard text entity set and a plurality of voice text entity sets;

[0045] An entity alignment module is configured to align each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculate the similarity between the aligned standard text entity set and each speech text entity set, and use the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set;

[0046] A total score acquisition module, configured to calculate a total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula;

[0047] The determination module is configured to determine that the voice verification fails if the total similarity score is less than a preset score threshold, and to determine that the voice verification passes if the total similarity score is greater than or equal to the score threshold.

[0048] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0049] at least one processor; and,

[0050] a memory communicatively connected to the at least one processor; wherein,

[0051] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the voice verification method described above.

[0052] In order to solve the above problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area, the data storage area stores created data, and the program storage area stores a computer program; wherein, when the computer program is executed by a processor, the voice verification method as described above is implemented.

[0053] In an embodiment of the present invention, original speech data is obtained, and a preset speech recognition method is used to recognize the original speech data to obtain multiple speech texts. The preset comparison text and the multiple speech texts are standardized to obtain multiple standard comparison texts and multiple standard speech texts, thereby removing noise interference. Then, a pre-trained entity recognition model is used to identify entities contained in the standard comparison text and the multiple standard speech texts to obtain a standard text entity set and multiple speech text entity sets. The similarity between the standard text entity set and each speech text entity set is calculated, and a standard corresponding entity set is screened out from the multiple speech text entity sets. The similarity between the standard corresponding entity set and the standard text entity set is calculated using a preset similarity total score formula, and whether the speech verification is passed is determined based on a preset score threshold and the similarity between the corresponding entity set and the standard text entity set. This avoids the problem of insufficient text comprehension and inaccurate entity recognition caused by directly using the speech text and the standard comparison text to calculate the similarity. Therefore, the embodiment of the present invention can achieve the purpose of improving the accuracy of text entity recognition in speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a voice verification method provided by one embodiment of the present invention;

[0055] Figure 2 A detailed flowchart of a step in a voice verification method is provided for one embodiment of the present invention;

[0056] Figure 3 A schematic diagram of the modules of a voice verification device provided by one embodiment of the present invention;

[0057] Figure 4 A schematic diagram of the internal structure of an electronic device for implementing a voice verification method according to an embodiment of the present invention;

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0060] The embodiment of the present application provides a voice verification method. The execution subject of the voice verification method includes but is not limited to at least one of the electronic devices such as the server and the terminal that can be configured to execute the method provided by the embodiment of the present application. Among them, the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. In other words, the voice verification method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0061] Reference Figure 1 FIG. 1 is a flow chart of a voice verification method according to an embodiment of the present invention. In this embodiment, the voice verification method includes:

[0062] S1. Acquire original voice data, and use a preset voice recognition method to recognize the original voice data to obtain multiple voice texts.

[0063] In the embodiment of the present invention, the speech recognition method, namely the automatic speech recognition method (ASR), is a method for converting speech data into text information. Due to the diversity and complexity of speech data, the automatic speech recognition method can only obtain satisfactory recognition results under certain restrictions. Among them, the recognition vocabulary, whether a single person speaks or multiple people speak, hardware, etc. will all affect the accuracy of the converted text.

[0064] The original voice data in this embodiment of the present invention may be voice data obtained by using an artificial intelligence customer service to communicate with a customer. The voice text is text obtained by voice recognition of the original voice data.

[0065] In detail, such as Figure 2 As shown, in the embodiment of the present invention, the original voice data is recognized by using a preset voice recognition method to obtain multiple voice texts, including:

[0066] S101, reducing noise from the original speech data to obtain noise-reduced speech data;

[0067] Furthermore, the denoising the original voice data to obtain the denoised voice data includes:

[0068] Decompressing the original voice data to obtain a waveform file of the original voice data;

[0069] Silence removal is performed on the waveform file of the original speech data to obtain noise-reduced speech data.

[0070] In the embodiment of the present invention, the format of the acquired original voice data is a compressed format such as mp3, wmv, etc., and the original voice data needs to be converted into an uncompressed pure waveform file, such as a wav file format, by decompression.

[0071] In an embodiment of the present invention, before extracting speech features using a speech feature extraction algorithm, it is necessary to perform noise reduction on the waveform file of the raw speech data by removing silence at the beginning and end to reduce the file size and prevent meaningless data from interfering with the extracted speech features. The pre-processed speech data is in the format of a waveform file.

[0072] S102, extracting speech features from the noise-reduced speech data using a speech feature extraction algorithm to obtain speech data features;

[0073] In an embodiment of the present invention, the speech feature extraction algorithm is a Mel-Frequency Cepstrum (MFCC) algorithm.

[0074] Furthermore, the extracting speech features from the noise-reduced speech data using a speech feature extraction algorithm to obtain speech data features includes:

[0075] Pre-emphasize the noise reduction voice data to obtain emphasized voice data;

[0076] Framing and windowing the aggravated speech data to obtain a speech waveform;

[0077] The filter in the speech feature extraction algorithm is used to extract the features of the speech waveform to obtain speech data features.

[0078] In an embodiment of the present invention, the noise reduction speech data is pre-emphasized through a high-pass filter to enhance the high-frequency data in the noise reduction speech data, so as to eliminate the vibration interference effect of the vocal cords and lips during the phonation process and compensate for the high-frequency part suppressed in the re-articulation system.

[0079] In the embodiment of the present invention, framing is achieved by dividing the emphasized speech data according to fixed time lengths, and windowing is achieved by using a preset window function Hamming.

[0080] In the embodiment of the present invention, the filter is a Mel filter (MelFilterbank) in the Mel frequency cepstrum algorithm.

[0081] S103, constructing a speech recognition network using a pre-trained acoustic model, a pre-trained language model, and a dictionary;

[0082] In an embodiment of the present invention, an acoustic model may be constructed using a Hidden Markov Model (HMM) to obtain an acoustic model, and then the acoustic model may be trained based on feature parameters in a pre-built training speech library to obtain a pre-trained acoustic model. The training speech library may be obtained by crawling publicly available speech data from the Internet using a crawler or other technology.

[0083] Furthermore, the pre-trained language model is a language model obtained by performing grammatical and semantic analysis on a pre-built training text library and training it based on a statistical model. The pre-trained language model is used to constrain the range of words searched based on voice. Similarly, the training text library can also be obtained by crawling publicly available data from the Internet using a web crawler.

[0084] The embodiment of the present invention utilizes a pre-trained language model to search for words in the dictionary based on voice.

[0085] S104: Identify the features of the voice data through the voice recognition network to obtain multiple voice texts.

[0086] In an embodiment of the present invention, identifying the voice data features according to the voice recognition network is specifically manifested as searching for the word range corresponding to each feature in the voice recognition network, where each feature may correspond to multiple words, that is, identifying the voice data features by the voice recognition network will obtain multiple voice texts.

[0087] S2. Obtain a comparison text, and perform standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts.

[0088] In an embodiment of the present invention, the comparison text is a preset text for comparison and verification. For example, when reviewing bank documents, the user is first asked to reserve a standard text, and then the review will compare the reserved standard text with the standard voice text converted from the voice data during the review.

[0089] The embodiments of the present application can acquire and process the comparative text based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0090] In the embodiment of the present invention, the standardization processing of the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts includes:

[0091] Standardizing the formats of the comparison text and the plurality of voice texts to obtain a standard format text and a plurality of standard format voice texts;

[0092] Performing word segmentation and part-of-speech tagging on the standard format text and each of the standard format voice texts, and removing non-keywords obtained by part-of-speech tagging to obtain a standard text word set and multiple voice text word sets;

[0093] Deduplication operations are performed on the words in the standard text word set and the words in each of the voice text word sets to obtain a deduplication standard word set and multiple deduplication voice word sets, and the words in the deduplication standard word set and the words in each of the deduplication voice word sets are combined to obtain the standard comparison text and multiple standard voice texts.

[0094] In an embodiment of the present invention, the formats of the comparison text and the multiple voice texts are standardized, including removing punctuation and other special symbols in the comparison text and the multiple voice texts, converting Chinese numerals into Arabic numerals, and uniformly converting uppercase and lowercase letters into lowercase letters.

[0095] The embodiment of the present invention performs word segmentation on the standard format text and each of the standard format voice texts by performing a mechanical word segmentation algorithm, and the mechanical word segmentation algorithm is also called a word segmentation algorithm based on string matching.

[0096] In the embodiment of the present invention, part-of-speech tagging is performed using a preset part-of-speech tagging table, wherein the part-of-speech tagging table contains all the words and the parts of speech of the words in the standard format text and the standard format voice text.

[0097] Specifically, the non-keywords are prepositions, conjunctions, auxiliary words, pronouns, adverbs, etc. By removing the non-keywords, the main content of the text can be highlighted.

[0098] In an embodiment of the present invention, the deduplication operation is to remove duplicate words in the standard text word set and to remove duplicate words in the speech text word set. If the standard text word set contains multiple first words, only one of the first words is retained and all others are removed. If the speech text word set contains multiple second words, only one of the second words is retained and all others are removed. If a third word exists in each of multiple speech text word sets, it does not meet the duplication condition and is not removed.

[0099] In the embodiment of the present invention, repeated words in the standard text word set and each of the sentence text word sets are found out by using a text comparison algorithm, wherein the text comparison algorithm may be a similarity-based text comparison algorithm.

[0100] S3. Utilize a pre-trained entity recognition model to identify entities contained in the standard comparison text and the plurality of standard voice texts, and obtain a standard text entity set and a plurality of voice text entity sets.

[0101] In this embodiment of the present invention, the entity recognition model is a sequence tagging (BILSTM+CRF) model. The BILSTM layer includes a bidirectional LSTM layer, which can more accurately find entities by combining the context of the standard comparison text and the standard speech text, and the CRF layer is used to filter the found entities.

[0102] In the embodiment of the present invention, before using the pre-trained entity recognition model to identify entities contained in the standard comparison text and the plurality of standard speech texts, the method further includes:

[0103] A training data set is obtained to train the pre-built entity recognition model to obtain a trained entity recognition model.

[0104] In detail, the training dataset is a text dataset with manually annotated entities.

[0105] In the embodiment of the present invention, the text entity is a noun with a clear meaning in a sentence.

[0106] Furthermore, the acquiring of a training data set to train a pre-built entity recognition model to obtain a trained entity recognition model includes:

[0107] Obtaining a training data set, and dividing the training data set into a training set and a test set;

[0108] Inputting the data in the training set into the neural network of the pre-built entity recognition model for forward propagation to obtain an output score;

[0109] Inputting the output score into an error function to obtain a function value, and comparing the function value with a preset expected value to obtain an error value;

[0110] Obtaining a gradient vector of the pre-built entity recognition model, and continuously adjusting parameters of the pre-built entity recognition model according to the gradient vector so that the error value approaches zero, and obtaining a preliminary entity recognition model when the error value no longer decreases;

[0111] The preliminary entity recognition model is tested using the test set, and when the test passes, the trained entity recognition model is obtained.

[0112] In another embodiment of the present invention, if the preliminary entity recognition model is tested using the test set, a training data set is re-acquired to train the pre-built entity recognition model if the test fails.

[0113] In the embodiment of the present invention, the error function (loss function) is a function for preventing overfitting of the model, and the gradient vector is a vector indicating how fast the model changes.

[0114] Forward propagation in the embodiment of the present invention refers to sequentially calculating and storing intermediate variables of the model from the input layer to the output layer of the neural network.

[0115] S4. Align each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculate the similarity between the aligned standard text entity set and each speech text entity set, and use the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set.

[0116] In an embodiment of the present invention, when using a pre-trained entity recognition model to identify entities of the standard text entity set and multiple voice text entities, since the original voice data received from the user may be unclear, the order of the entities obtained is not aligned, so it is necessary to align the entities in the standard text entity set with the entities in all the voice text entity sets in sequence to avoid the comparison between the text and the voice text not being in a corresponding relationship.

[0117] In an embodiment of the present invention, alignment is performed through the noun attributes between the standard text entity set and the speech text entity set. For example, the standard text entities included in the standard text entity set are "Ping An", "Bank", "Shares", and "Limited Liability Company", and the speech text entities included in the speech text entity set are "Ping An", "Insurance", and "Group". The alignment result obtained is "Ping An" = "Ping An", "Bank" = "Insurance", "Shares" = "Group", and "Limited Liability Company" = "Group".

[0118] In an embodiment of the present invention, aligning each standard text entity in the standard text entity set with each voice text entity in all the voice text entity sets, and calculating the similarity between the aligned standard text entity set and each voice text entity set includes:

[0119] Inputting each standard text entity in the standard text entity set and each speech text entity in the speech text entity set into a pre-built multilingual BERT model to obtain a standard entity matrix and multiple semantic entity matrices;

[0120] Multiplying the standard entity matrix and each of the semantic entity matrices to obtain a multiplication matrix, and normalizing the multiplication matrix to obtain a plurality of aligned entity groups;

[0121] The similarity of each of the aligned entity groups is calculated using a cosine similarity formula to obtain the similarity between the standard text entity set and each of the voice text entity sets.

[0122] In the embodiment of the present invention, the multilingual BERT model is an mBERT model used in the field of machine translation.

[0123] In the embodiment of the present invention, before calculating the similarity of each of the aligned entity groups using the cosine similarity formula, the method further includes converting two matrices in the aligned entity groups into vector forms.

[0124] S5. Calculate the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset total similarity score calculation formula.

[0125] In the embodiment of the present invention, there are multiple standard corresponding entities in the standard corresponding entity set.

[0126] In an embodiment of the present invention, the calculating of the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset total similarity score calculation formula includes:

[0127] Dividing the plurality of standard corresponding entities in the standard corresponding entity set to obtain prefix entities, suffix entities, address entities, and name entities;

[0128] Searching for the name entity and the address entity in the standard text entity set;

[0129] If the same name entity and address entity exist in the standard text entity set, determining a similarity score between the name entity and the address entity;

[0130] If the same name entity and the address entity do not exist in the standard text entity set, obtaining the corresponding name entity and the corresponding address entity from the standard text entity set, converting the name entity, the address entity, the corresponding name entity, and the corresponding address entity into pinyin, and calculating the similarity between the name entity and the corresponding name entity, and between the address entity and the corresponding address entity using an edit distance formula to obtain a similarity score between the name entity and the address entity;

[0131] Acquire the corresponding prefix entity and the corresponding suffix entity from the standard text entity set based on the prefix entity and the suffix entity;

[0132] Converting the prefix entity and the corresponding prefix entity, as well as the suffix entity and the corresponding suffix entity, into word vectors, and calculating the similarity between the prefix entity and the corresponding prefix entity, as well as between the suffix entity and the corresponding suffix entity, using a cosine similarity formula to obtain a similarity score between the prefix entity and the suffix entity;

[0133] The total similarity score of the name entity, the address entity, the prefix entity and the suffix entity is calculated using the total similarity score formula, and the total similarity score is used as the total similarity score of the standard corresponding entity set and the standard text entity set.

[0134] In this embodiment, if the same name entity and address entity exist in the standard text entity set, the similarity score of the name entity and the address entity is directly determined to be a preset score, for example, the similarity score of the name entity and the address entity is determined to be 1.

[0135] In the embodiment of the present invention, the similarity total score calculation formula is:

[0136] Total similarity score = w1*address entity similarity score + w2*prefix entity similarity score + w3*name entity similarity score + w4*suffix entity similarity score

[0137] Among them, wn is the weight coefficient.

[0138] S6. If the total similarity score is less than a preset score threshold, it is determined that the voice verification fails.

[0139] In an embodiment of the present invention, if the total similarity score is less than a preset score threshold, the voice verification fails. Previously, the method further includes: obtaining a preset score threshold, comparing the total similarity score with the score threshold, and determining whether the total similarity score is less than the preset score threshold.

[0140] In an embodiment of the present invention, the score threshold is a score used to determine whether the voice verification of the original voice data has passed.

[0141] Furthermore, the score threshold can be flexibly adjusted according to user needs.

[0142] S7. If the total similarity score is greater than or equal to the score threshold, it is determined that the voice verification is passed.

[0143] Furthermore, for example, when a user applies for a credit loan, if the voice verification fails, the application fails; if the voice verification passes, the application succeeds.

[0144] In an embodiment of the present invention, original speech data is obtained, and a preset speech recognition method is used to recognize the original speech data to obtain multiple speech texts. The preset comparison text and the multiple speech texts are standardized to obtain multiple standard comparison texts and multiple standard speech texts, thereby removing noise interference. Then, a pre-trained entity recognition model is used to identify entities contained in the standard comparison text and the multiple standard speech texts to obtain a standard text entity set and multiple speech text entity sets. The similarity between the standard text entity set and each speech text entity set is calculated, and a standard corresponding entity set is screened out from the multiple speech text entity sets. The similarity between the standard corresponding entity set and the standard text entity set is calculated using a preset similarity total score formula, and whether the speech verification is passed is determined based on a preset score threshold and the similarity between the corresponding entity set and the standard text entity set. This avoids the problem of insufficient text comprehension and inaccurate entity recognition caused by directly using the speech text and the standard comparison text to calculate the similarity. Therefore, the embodiment of the present invention can achieve the purpose of improving the accuracy of text entity recognition in speech.

[0145] like Figure 3 FIG. 1 is a schematic diagram of a module of a voice verification device according to the present invention.

[0146] The voice verification device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the voice verification device may include a voice recognition module 101, a text comparison module 102, an entity recognition module 103, an entity alignment module 104, a total score acquisition module 105, and a determination module 106. The module of the present invention, also known as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and can perform a fixed function, which is stored in the memory of the electronic device.

[0147] In this embodiment, the functions of each module / unit are as follows:

[0148] The speech recognition module 101 is used to obtain original speech data, recognize the original speech data using a preset speech recognition method, and obtain multiple speech texts.

[0149] In the embodiment of the present invention, the speech recognition method, namely the automatic speech recognition method (ASR), is a method for converting speech data into text information. Due to the diversity and complexity of speech data, the automatic speech recognition method can only obtain satisfactory recognition results under certain restrictions. Among them, the recognition vocabulary, whether a single person speaks or multiple people speak, hardware, etc. will all affect the accuracy of the converted text.

[0150] The original voice data in this embodiment of the present invention may be voice data obtained by using an artificial intelligence customer service to communicate with a customer. The voice text is text obtained by voice recognition of the original voice data.

[0151] Furthermore, in another optional embodiment of the present invention, the speech recognition module 101 specifically includes a noise reduction unit, a feature extraction unit, a recognition network construction unit and a speech recognition unit:

[0152] The noise reduction unit is used to reduce the noise of the original voice data to obtain noise-reduced voice data.

[0153] Furthermore, the noise reduction unit is specifically configured to:

[0154] Decompressing the original voice data to obtain a waveform file of the original voice data;

[0155] Silence removal is performed on the waveform file of the original speech data to obtain noise-reduced speech data.

[0156] In the embodiment of the present invention, the format of the acquired original voice data is a compressed format such as mp3, wmv, etc., and the original voice data needs to be converted into an uncompressed pure waveform file, such as a wav file format, by decompression.

[0157] In an embodiment of the present invention, before extracting speech features using a speech feature extraction algorithm, it is necessary to perform noise reduction on the waveform file of the raw speech data by removing silence at the beginning and end to reduce the file size and prevent meaningless data from interfering with the extracted speech features. The pre-processed speech data is in the format of a waveform file.

[0158] The feature extraction unit is used to extract the speech features in the noise reduction speech data using a speech feature extraction algorithm to obtain speech data features.

[0159] In an embodiment of the present invention, the speech feature extraction algorithm is a Mel-Frequency Cepstrum (MFCC) algorithm.

[0160] Furthermore, the feature extraction unit is specifically used to:

[0161] Pre-emphasize the noise reduction voice data to obtain emphasized voice data;

[0162] Framing and windowing the aggravated speech data to obtain a speech waveform;

[0163] The filter in the speech feature extraction algorithm is used to extract the features of the speech waveform to obtain speech data features.

[0164] In an embodiment of the present invention, the noise reduction speech data is pre-emphasized through a high-pass filter to enhance the high-frequency data in the noise reduction speech data, so as to eliminate the vibration interference effect of the vocal cords and lips during the phonation process and compensate for the high-frequency part suppressed in the re-articulation system.

[0165] In the embodiment of the present invention, framing is achieved by dividing the emphasized speech data according to fixed time lengths, and windowing is achieved by using a preset window function Hamming.

[0166] In the embodiment of the present invention, the filter is a Mel filter (MelFilterbank) in the Mel frequency cepstrum algorithm.

[0167] The recognition network construction unit is used to construct a speech recognition network using a pre-trained acoustic model, a pre-trained language model and a dictionary.

[0168] In an embodiment of the present invention, an acoustic model may be constructed using a Hidden Markov Model (HMM) to obtain an acoustic model, and then the acoustic model may be trained based on feature parameters in a pre-built training speech library to obtain a pre-trained acoustic model. The training speech library may be obtained by crawling publicly available speech data from the Internet using a crawler or other technology.

[0169] Furthermore, the pre-trained language model is a language model obtained by performing grammatical and semantic analysis on a pre-built training text library and training it based on a statistical model. The pre-trained language model is used to constrain the range of words searched based on voice. Similarly, the training text library can also be obtained by crawling publicly available data from the Internet using a web crawler.

[0170] The embodiment of the present invention utilizes a pre-trained language model to search for words in the dictionary based on voice.

[0171] The speech recognition unit is used to recognize the speech data features through the speech recognition network to obtain multiple speech texts.

[0172] In an embodiment of the present invention, identifying the voice data features according to the voice recognition network is specifically manifested as searching for the word range corresponding to each feature in the voice recognition network, where each feature may correspond to multiple words, that is, identifying the voice data features by the voice recognition network will obtain multiple voice texts.

[0173] The text comparison module 102 is configured to obtain a comparison text and perform standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts.

[0174] In an embodiment of the present invention, the comparison text is a preset text for comparison and verification. For example, when reviewing bank documents, the user is first asked to reserve a standard text, and then the review will compare the reserved standard text with the standard voice text converted from the voice data during the review.

[0175] The embodiments of the present application can acquire and process the compared text based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0176] In the embodiment of the present invention, the text comparison module 102 is specifically configured to:

[0177] Standardizing the formats of the comparison text and the plurality of voice texts to obtain a standard format text and a plurality of standard format voice texts;

[0178] Performing word segmentation and part-of-speech tagging on the standard format text and each of the standard format voice texts, and removing non-keywords obtained by part-of-speech tagging to obtain a standard text word set and multiple voice text word sets;

[0179] Deduplication operations are performed on the words in the standard text word set and the words in each of the voice text word sets to obtain a deduplication standard word set and multiple deduplication voice word sets, and the words in the deduplication standard word set and the words in each of the deduplication voice word sets are combined to obtain the standard comparison text and multiple standard voice texts.

[0180] In an embodiment of the present invention, the formats of the comparison text and the multiple voice texts are standardized, including removing punctuation and other special symbols in the comparison text and the multiple voice texts, converting Chinese numerals into Arabic numerals, and uniformly converting uppercase and lowercase letters into lowercase letters.

[0181] The embodiment of the present invention performs word segmentation on the standard format text and each of the standard format voice texts by performing a mechanical word segmentation algorithm, and the mechanical word segmentation algorithm is also called a word segmentation algorithm based on string matching.

[0182] In the embodiment of the present invention, part-of-speech tagging is performed using a preset part-of-speech tagging table, wherein the part-of-speech tagging table contains all the words and the parts of speech of the words in the standard format text and the standard format voice text.

[0183] Specifically, the non-keywords are prepositions, conjunctions, auxiliary words, pronouns, adverbs, etc. By removing the non-keywords, the main content of the text can be highlighted.

[0184] In an embodiment of the present invention, the deduplication operation is to remove duplicate words in the standard text word set and to remove duplicate words in the speech text word set. If the standard text word set contains multiple first words, only one of the first words is retained and all others are removed. If the speech text word set contains multiple second words, only one of the second words is retained and all others are removed. If a third word exists in each of multiple speech text word sets, it does not meet the duplication condition and is not removed.

[0185] In the embodiment of the present invention, repeated words in the standard text word set and each of the sentence text word sets are found out by using a text comparison algorithm, wherein the text comparison algorithm may be a similarity-based text comparison algorithm.

[0186] The entity recognition module 103 is used to use a pre-trained entity recognition model to identify entities contained in the standard comparison text and the multiple standard voice texts, and obtain a standard text entity set and multiple voice text entity sets.

[0187] In this embodiment of the present invention, the entity recognition model is a sequence tagging (BILSTM+CRF) model. The BILSTM layer includes a bidirectional LSTM layer, which can more accurately find entities by combining the context of the standard comparison text and the standard speech text, and the CRF layer is used to filter the found entities.

[0188] In the embodiment of the present invention, before using the pre-trained entity recognition model to identify entities contained in the standard comparison text and the plurality of standard speech texts, the method further includes:

[0189] A training data set is obtained to train the pre-built entity recognition model to obtain a trained entity recognition model.

[0190] In detail, the training dataset is a text dataset with manually annotated entities.

[0191] In the embodiment of the present invention, the text entity is a noun with a clear meaning in a sentence.

[0192] Furthermore, the acquiring of a training data set to train a pre-built entity recognition model to obtain a trained entity recognition model includes:

[0193] Obtaining a training data set, and dividing the training data set into a training set and a test set;

[0194] Inputting the data in the training set into the neural network of the pre-built entity recognition model for forward propagation to obtain an output score;

[0195] Inputting the output score into an error function to obtain a function value, and comparing the function value with a preset expected value to obtain an error value;

[0196] Obtaining a gradient vector of the pre-built entity recognition model, and continuously adjusting parameters of the pre-built entity recognition model according to the gradient vector so that the error value approaches zero, and obtaining a preliminary entity recognition model when the error value no longer decreases;

[0197] The preliminary entity recognition model is tested using the test set, and when the test passes, the trained entity recognition model is obtained.

[0198] In another embodiment of the present invention, if the preliminary entity recognition model is tested using the test set, a training data set is re-acquired to train the pre-built entity recognition model if the test fails.

[0199] In the embodiment of the present invention, the error function (loss function) is a function for preventing overfitting of the model, and the gradient vector is a vector indicating how fast the model changes.

[0200] Forward propagation in the embodiment of the present invention refers to sequentially calculating and storing intermediate variables of the model from the input layer to the output layer of the neural network.

[0201] The entity alignment module 104 is used to align each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculate the similarity between the aligned standard text entity set and each speech text entity set, and use the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set.

[0202] In an embodiment of the present invention, when using a pre-trained entity recognition model to identify entities of the standard text entity set and multiple voice text entities, since the original voice data received from the user may be unclear, the order of the entities obtained is not aligned, so it is necessary to align the entities in the standard text entity set with the entities in all the voice text entity sets in sequence to avoid the comparison between the text and the voice text not being in a corresponding relationship.

[0203] In an embodiment of the present invention, alignment is performed through the noun attributes between the standard text entity set and the speech text entity set. For example, the standard text entities included in the standard text entity set are "Ping An", "Bank", "Shares", and "Limited Liability Company", and the speech text entities included in the speech text entity set are "Ping An", "Insurance", and "Group". The alignment result obtained is "Ping An" = "Ping An", "Bank" = "Insurance", "Shares" = "Group", and "Limited Liability Company" = "Group".

[0204] In an embodiment of the present invention, the following operations can be performed to align each standard text entity in the standard text entity set with each voice text entity in all the voice text entity sets, and calculate the similarity between the aligned standard text entity set and each voice text entity set:

[0205] Inputting each standard text entity in the standard text entity set and each speech text entity in the speech text entity set into a pre-built multilingual BERT model to obtain a standard entity matrix and multiple semantic entity matrices;

[0206] Multiplying the standard entity matrix and each of the semantic entity matrices to obtain a multiplication matrix, and normalizing the multiplication matrix to obtain a plurality of aligned entity groups;

[0207] The similarity of each of the aligned entity groups is calculated using a cosine similarity formula to obtain the similarity between the standard text entity set and each of the voice text entity sets.

[0208] In the embodiment of the present invention, the multilingual BERT model is an mBERT model used in the field of machine translation.

[0209] In the embodiment of the present invention, before calculating the similarity of each of the aligned entity groups using the cosine similarity formula, the method further includes converting two matrices in the aligned entity groups into vector forms.

[0210] The total score acquisition module 105 is configured to calculate the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula.

[0211] In the embodiment of the present invention, there are multiple standard corresponding entities in the standard corresponding entity set.

[0212] In the embodiment of the present invention, the total score acquisition module 105 is specifically configured to:

[0213] Dividing the plurality of standard corresponding entities in the standard corresponding entity set to obtain prefix entities, suffix entities, address entities, and name entities;

[0214] Searching for the name entity and the address entity in the standard text entity set;

[0215] If the same name entity and address entity exist in the standard text entity set, determining a similarity score between the name entity and the address entity;

[0216] If the same name entity and the address entity do not exist in the standard text entity set, obtaining the corresponding name entity and the corresponding address entity from the standard text entity set, converting the name entity, the address entity, the corresponding name entity, and the corresponding address entity into pinyin, and calculating the similarity between the name entity and the corresponding name entity, and between the address entity and the corresponding address entity using an edit distance formula to obtain a similarity score between the name entity and the address entity;

[0217] Acquire the corresponding prefix entity and the corresponding suffix entity from the standard text entity set based on the prefix entity and the suffix entity;

[0218] Converting the prefix entity and the corresponding prefix entity, as well as the suffix entity and the corresponding suffix entity, into word vectors, and calculating the similarity between the prefix entity and the corresponding prefix entity, as well as between the suffix entity and the corresponding suffix entity, using a cosine similarity formula to obtain a similarity score between the prefix entity and the suffix entity;

[0219] The total similarity score of the name entity, the address entity, the prefix entity and the suffix entity is calculated using the total similarity score formula, and the total similarity score is used as the total similarity score of the standard corresponding entity set and the standard text entity set.

[0220] In this embodiment, if the same name entity and address entity exist in the standard text entity set, the similarity score of the name entity and the address entity is directly determined to be a preset score, for example, the similarity score of the name entity and the address entity is determined to be 1.

[0221] In the embodiment of the present invention, the similarity total score calculation formula is:

[0222] Total similarity score = w1*address entity similarity score + w2*prefix entity similarity score + w3*name entity similarity score + w4*suffix entity similarity score

[0223] Among them, wn is the weight coefficient.

[0224] The voice verification module 106 is configured to determine that the voice verification fails if the total similarity score is less than a preset score threshold, and to determine that the voice verification passes if the total similarity score is greater than or equal to the score threshold.

[0225] In an embodiment of the present invention, if the total similarity score is less than a preset score threshold, the voice verification fails. Previously, the method further includes: obtaining a preset score threshold, comparing the total similarity score with the score threshold, and determining whether the total similarity score is less than the preset score threshold.

[0226] In an embodiment of the present invention, the score threshold is a score used to determine whether the voice verification of the original voice data has passed.

[0227] Furthermore, the score threshold can be flexibly adjusted according to user needs.

[0228] Furthermore, for example, when a user applies for a credit loan, if the voice verification fails, the application fails; if the voice verification passes, the application succeeds.

[0229] like Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the voice verification method of the present invention.

[0230] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a voice verification program.

[0231] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the memory 11 (for example, executing a voice verification program, etc.), as well as calling data stored in the memory 11, to execute various functions of the electronic device and process data.

[0232] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can be used not only to store application software and various types of data installed in the electronic device, such as the code of the voice verification program, etc., but also to temporarily store data that has been output or is to be output.

[0233] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0234] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0235] Figure 4 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 4 The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0236] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0237] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0238] The voice verification program stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve the following:

[0239] Acquiring original voice data, and recognizing the original voice data using a preset voice recognition method to obtain multiple voice texts;

[0240] Obtaining a comparison text, and performing standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts;

[0241] Using a pre-trained entity recognition model, identifying entities contained in the standard comparison text and the plurality of standard voice texts, to obtain a standard text entity set and a plurality of voice text entity sets;

[0242] Aligning each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculating the similarity between the aligned standard text entity set and each speech text entity set, and taking the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set;

[0243] Calculating a total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula;

[0244] If the total similarity score is less than the preset score threshold, it is determined that the voice verification fails;

[0245] If the total similarity score is greater than or equal to the score threshold, it is determined that the voice verification has passed.

[0246] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0247] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0248] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0249] Acquiring original voice data, and recognizing the original voice data using a preset voice recognition method to obtain multiple voice texts;

[0250] Obtaining a comparison text, and performing standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts;

[0251] Using a pre-trained entity recognition model, identifying entities contained in the standard comparison text and the plurality of standard voice texts, to obtain a standard text entity set and a plurality of voice text entity sets;

[0252] Aligning each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculating the similarity between the aligned standard text entity set and each speech text entity set, and taking the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set;

[0253] Calculating a total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula;

[0254] If the total similarity score is less than the preset score threshold, it is determined that the voice verification fails;

[0255] If the total similarity score is greater than or equal to the score threshold, it is determined that the voice verification has passed.

[0256] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0257] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0258] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0259] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0260] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0261] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0262] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0263] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A voice verification method, characterized in that: The method comprises: Acquire original voice data, and recognize the original voice data using a preset voice recognition method to obtain multiple voice texts; Acquire a comparison text, and perform standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts; Using a pre-trained entity recognition model, identifying entities contained in the standard comparison text and the plurality of standard voice texts, to obtain a standard text entity set and a plurality of voice text entity sets; Aligning each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculating the similarity between the aligned standard text entity set and each speech text entity set, and taking the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set; Calculating the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula; If the total similarity score is less than the preset score threshold, it is determined that the voice verification fails; If the total similarity score is greater than or equal to the score threshold, it is determined that the voice verification has passed.

2. The voice verification method according to claim 1, characterized in that: The method of recognizing the original voice data by using a preset voice recognition method to obtain a plurality of voice texts includes: Denoising the original voice data to obtain denoised voice data; Extracting speech features from the noise-reduced speech data using a speech feature extraction algorithm to obtain speech data features; Build a speech recognition network using pre-trained acoustic models, pre-trained language models, and dictionaries; The speech data features are recognized by the speech recognition network to obtain a plurality of speech texts.

3. The voice verification method according to claim 2, characterized in that: The step of reducing the noise of the original voice data to obtain the reduced noise voice data comprises: Decompressing the original voice data to obtain a waveform file of the original voice data; The waveform file of the original speech data is subjected to silence removal to obtain noise-reduced speech data.

4. The voice verification method according to claim 2, characterized in that: The method of extracting speech features from the noise reduction speech data using a speech feature extraction algorithm to obtain speech data features includes: Pre-emphasize the noise reduction speech data to obtain emphasized speech data; Framing and windowing the aggravated speech data to obtain a speech waveform diagram; The filter in the speech feature extraction algorithm is used to extract the features of the speech waveform to obtain speech data features.

5. The voice verification method according to claim 1, characterized in that: The step of performing standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts includes: Standardizing the formats of the comparison text and the plurality of voice texts to obtain a standard format text and a plurality of standard format voice texts; Performing word segmentation and part-of-speech tagging on the standard format text and each of the standard format voice texts, and removing non-keywords obtained through part-of-speech tagging to obtain a standard text word set and multiple voice text word sets; Deduplication operations are performed on the words in the standard text word set and the words in each of the voice text word sets to obtain a deduplicated standard word set and multiple deduplicated voice word sets, and the words in the deduplicated standard word set and the words in each of the deduplicated voice word sets are combined to obtain the standard comparison text and multiple standard voice texts.

6. The voice verification method according to claim 1, characterized in that: The step of aligning each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, and calculating the similarity between the aligned standard text entity set and each speech text entity set, comprises: Inputting each standard text entity in the standard text entity set and each speech text entity in all the speech text entity sets into a pre-built multilingual BERT model to obtain a standard entity matrix and multiple semantic entity matrices; Multiplying the standard entity matrix and each of the semantic entity matrices to obtain a multiplication matrix, and normalizing the multiplication matrix to obtain a plurality of aligned entity groups; The similarity of each of the aligned entity groups is calculated using the cosine similarity formula to obtain the similarity between the standard text entity set and each of the voice text entity sets.

7. The voice verification method according to claim 1, characterized in that: The calculating the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula includes: Dividing the multiple standard corresponding entities in the standard corresponding entity set to obtain prefix entities, suffix entities, address entities, and name entities; Searching for the name entity and the address entity in the standard text entity set; If the same name entity and address entity exist in the standard text entity set, determining a similarity score between the name entity and the address entity; If the same name entity and the address entity do not exist in the standard text entity set, obtain the corresponding name entity and the corresponding address entity from the standard text entity set, convert the name entity, the address entity, the corresponding name entity and the corresponding address entity into pinyin, and calculate the similarity between the name entity and the corresponding name entity and between the address entity and the corresponding address entity using the edit distance formula to obtain the similarity score between the name entity and the address entity; Based on the prefix entity and the suffix entity, obtain a corresponding prefix entity and a corresponding suffix entity from the standard text entity set; Convert the prefix entity and the corresponding prefix entity and the suffix entity and the corresponding suffix entity into word vectors, and calculate the similarity between the prefix entity and the corresponding prefix entity and the suffix entity and the corresponding suffix entity using a cosine similarity formula to obtain a similarity score between the prefix entity and the suffix entity; The preset similarity formula is used to calculate the total similarity score of the name entity, the address entity, the prefix entity and the suffix entity, and the total similarity score is used as the total similarity score of the standard corresponding entity set and the standard text entity set.

8. A voice verification device, characterized in that: The device comprises: A speech recognition module is used to obtain original speech data, recognize the original speech data using a preset speech recognition method, and obtain multiple speech texts; A text comparison module, used for obtaining a comparison text, and performing standardization processing on the comparison text and the plurality of voice texts to obtain a standard comparison text and a plurality of standard voice texts; An entity recognition module, used for recognizing entities contained in the standard comparison text and the plurality of standard voice texts by using a pre-trained entity recognition model, and obtaining a standard text entity set and a plurality of voice text entity sets; An entity alignment module is used to align each standard text entity in the standard text entity set with each speech text entity in all the speech text entity sets, calculate the similarity between the standard text entity set and each speech text entity set after alignment, and use the speech text entity set with the highest similarity as the standard corresponding entity set of the standard text entity set; A total score acquisition module, used to calculate the total similarity score between the standard corresponding entity set and the standard text entity set based on a preset similarity formula; The determination module is used to determine that the voice verification fails if the total similarity score is less than a preset score threshold, and to determine that the voice verification passes if the total similarity score is greater than or equal to the score threshold.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the voice verification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, comprising a data storage area and a program storage area, wherein the data storage area stores created data and the program storage area stores a computer program; wherein: When the computer program is executed by a processor, the voice verification method according to any one of claims 1 to 7 is implemented.

Images