Anomaly detection methods, apparatus, electronic devices and computer-readable storage media

Abstract

This disclosure provides an anomaly detection method, apparatus, electronic device, and computer-readable storage medium, relating to the field of speech recognition technology. The anomaly detection method includes: acquiring speech recognition text, which is obtained by performing speech recognition on speech data using a speech recognition model, wherein the speech recognition model outputs speech recognition model information during the speech recognition process; extracting semantic features from the speech recognition text to obtain semantic features of the recognized text; extracting features from the speech recognition model information to obtain speech recognition model features; fusing the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features; and processing the anomaly measurement features using an anomaly detection model to determine the recognition effect of recognizing speech data as speech recognition text. Embodiments of this disclosure can accurately determine whether the speech recognition text transcribed from speech data is correct.

Description

Technical Field

[0001] This disclosure relates to the field of speech recognition technology, and in particular to an anomaly detection method and apparatus, electronic device and computer-readable storage medium. Background Technology

[0002] Because speech data is often highly colloquial or contains multiple dialects, specialized vocabulary, and new words, text transcription using speech recognition models may result in errors. Detecting these transcription errors allows for timely correction of speech-recognition text errors, preventing impact on downstream text transcription tasks, and also enables optimization of the speech recognition model. Therefore, anomaly identification in speech data text transcription is of great significance for speech data recognition.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0004] The purpose of this disclosure is to provide an anomaly detection method, apparatus, electronic device, and computer-readable storage medium, which can process the anomaly measurement features corresponding to speech recognition text through an anomaly detection model in order to determine whether the text transcription of speech data by the speech recognition model is correct.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] This disclosure provides an anomaly detection method, comprising: acquiring speech recognition text, wherein the speech recognition text is obtained by performing speech recognition on speech data using a speech recognition model, and the speech recognition model outputs speech recognition model information during the speech recognition process on the speech data; extracting semantic features from the speech recognition text to obtain semantic features of the recognized text; extracting features from the speech recognition model information to obtain speech recognition model features; fusing the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features; and processing the anomaly measurement features using an anomaly detection model to determine the recognition effect of recognizing the speech data as the speech recognition text.

[0007] In some embodiments, semantic feature extraction is performed on the speech recognition text to obtain semantic features of the recognized text, including: determining the state information of each character or word in the speech recognition text, the state information including current state information, previous state information, and subsequent state information; determining character feature information of the speech recognition text based on the state information of each character or word in the speech recognition text; performing feature extraction on the speech recognition text to determine sentence feature information of the speech recognition text; and generating the semantic features of the recognized text based on the character feature information and the sentence feature information.

[0008] In some embodiments, the speech recognition text includes target characters or target words; wherein determining the state information of each character or word in the speech recognition text, the state information including current state information, prior state information, and subsequent state information includes: determining the prior state information of the target character or target word based on the preceding n-1 characters or words in the speech recognition text; determining the current state information of the target character or target word based on the preceding (n-1) / 2 characters or words and the following (n-1) / 2 characters or words in the speech recognition text; and determining the subsequent state information of the target character or target word based on the following n-1 characters or words in the speech recognition text, where n is an integer greater than or equal to 1.

[0009] In some embodiments, before obtaining the speech recognition text, the method further includes: performing speech recognition processing on the speech data using the speech recognition model to determine a plurality of candidate texts, wherein each candidate text corresponds to a posterior probability; and determining the candidate text with the highest posterior probability as the speech recognition text.

[0010] In some embodiments, the speech recognition model information includes the maximum posterior probability among the plurality of candidate texts, the second largest posterior probability among the candidate texts, the duration of each character or word, the frame interval between each character or word and the previous character or word, and the frame interval between each character or word and the next character or word; wherein, feature extraction of the speech recognition model information to obtain speech recognition model features includes: generating the speech recognition model features based on the maximum posterior probability, the second largest posterior probability, the duration of each character or word, the frame interval between each character or word and the previous character or word, and the frame interval between each character or word and the next character or word.

[0011] In some embodiments, the method further includes: processing the speech data and the speech-recognized text to determine the speech audio length, speech-recognized text length, speech energy, and speech speed of the speech data; generating speech quality features based on the speech audio length, speech-recognized text length, speech energy, and speech speed of the speech data; wherein fusing the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features includes: fusing the semantic features of the recognized text, the speech recognition model features, and the speech quality features to obtain the anomaly measurement features.

[0012] In some embodiments, the anomaly measurement features are processed by an anomaly detection model to determine the recognition effect of the speech recognition model in transcribing the speech data into speech recognition text. This includes: processing the anomaly measurement features by the anomaly detection model to determine an anomaly prediction sequence, the anomaly prediction sequence being used to indicate whether a character or word in the speech recognition text is transcribed correctly; obtaining an anomaly annotation sequence, the anomaly annotation sequence being used to annotate whether a character or word in the speech recognition text is transcribed correctly; and training the anomaly detection model using the anomaly annotation sequence and the anomaly prediction sequence.

[0013] In some embodiments, obtaining an anomaly annotation sequence includes: obtaining annotation text corresponding to the speech data; aligning the annotation text with the speech recognition text; and generating the anomaly annotation sequence based on the aligned annotation text and the aligned speech recognition text.

[0014] In some embodiments, determining the recognition effect of recognizing the speech data as speech recognition text includes: determining the proportion of incorrectly transcribed characters and words in the speech recognition text based on the abnormal prediction sequence; if the proportion exceeds a target threshold, then determining that the speech recognition text is transcribed abnormally.

[0015] This disclosure provides an anomaly detection device, including: a speech recognition text acquisition module, a semantic feature extraction module, a speech recognition model feature acquisition module, a fusion module, and an anomaly judgment module.

[0016] The speech recognition text acquisition module is used to acquire speech recognition text, which is obtained by performing speech recognition on speech data through a speech recognition model. The speech recognition model outputs speech recognition model information during the speech recognition process. The semantic feature extraction module is used to extract semantic features from the speech recognition text to obtain semantic features of the recognized text. The speech recognition model feature acquisition module is used to extract features from the speech recognition model information to obtain speech recognition model features. The fusion module is used to fuse the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features. The anomaly judgment module is used to process the anomaly measurement features through an anomaly detection model to determine the recognition effect of recognizing the speech data as the speech recognition text.

[0017] This disclosure provides an electronic device comprising: a memory and a processor; the memory for storing program instructions; and the processor for calling the program instructions stored in the memory to implement the anomaly detection method described above.

[0018] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the anomaly detection method as described in any of the preceding embodiments.

[0019] This disclosure provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned anomaly detection method.

[0020] The anomaly detection method, apparatus, electronic device, and computer-readable storage medium provided in this disclosure can, on the one hand, process the anomaly measurement features corresponding to speech recognition text through an anomaly detection model to determine whether the text transcription of speech data by the speech recognition model is normal; on the other hand, the aforementioned anomaly measurement features integrate both the semantic features of the speech recognition text and the speech recognition model features generated when the speech recognition model transcribes the semantic data into text, making the text transcription anomaly judgment result more accurate.

[0021] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 A schematic diagram of an exemplary system architecture that can be applied to the anomaly detection method or anomaly detection apparatus in the embodiments of this disclosure is shown.

[0024] Figure 2 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment.

[0025] Figure 3 This is a flowchart illustrating a method for identifying semantic features of text according to an exemplary embodiment.

[0026] Figure 4 This is a schematic diagram illustrating a state information determination method according to an exemplary embodiment.

[0027] Figure 5 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment.

[0028] Figure 6 This is a flowchart illustrating an anomaly detection model training method according to an exemplary embodiment.

[0029] Figure 7 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment.

[0030] Figure 8 This is a block diagram illustrating an anomaly detection device according to an exemplary embodiment.

[0031] Figure 9 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences; the terms "contains," "includes," and "has" are used to indicate an open-ended inclusion and mean that additional elements / components / etc. may exist besides the listed elements / components / etc.

[0037] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0038] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0039] Figure 1A schematic diagram of an exemplary system architecture that can be applied to the anomaly detection method or anomaly detection apparatus in the embodiments of this disclosure is shown.

[0040] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0041] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart home devices, etc.

[0042] Server 105 can be a server that provides various services, such as a backend management server that supports the devices operated by users using terminal devices 101, 102, and 103. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal devices.

[0043] A server can be a standalone physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This disclosure does not impose any restrictions on this.

[0044] Server 105 may, for example, acquire speech-recognized text, which is obtained by a speech recognition model after performing speech recognition on speech data. During the speech recognition process, the speech recognition model outputs speech recognition model information. Server 105 may, for example, extract semantic features from the speech-recognized text to obtain semantic features of the recognized text. Server 105 may, for example, extract features from the speech recognition model information to obtain speech recognition model features. Server 105 may, for example, fuse the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features. Server 105 may, for example, process the anomaly measurement features through an anomaly detection model to determine the recognition effect of recognizing speech data as speech-recognized text.

[0045] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Server 105 can be a single physical server or a combination of multiple servers. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0046] Figure 2 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment. The method provided in this disclosure can be executed by any electronic device with computing power; for example, the method can be implemented by the aforementioned... Figure 1 The execution can be performed by a server or terminal device in the embodiments, or it can be performed by both a server and a terminal device. In the following embodiments, the server is used as the execution subject for illustration, but this disclosure is not limited to this.

[0047] Reference Figure 2 The anomaly detection method provided in this disclosure may include the following steps.

[0048] Step S202: Obtain speech recognition text. Speech recognition text is obtained by a speech recognition model after performing speech recognition on speech data. The speech recognition model outputs speech recognition model information during the speech recognition process on the speech data.

[0049] The speech recognition model can be an end-to-end speech recognition model, such as a model built with CTC (Connection-Temporal Classification, a method to solve the problem of input and output sequences having different lengths and being unable to align) algorithm and deep neural network. This application does not limit the specific architecture of the speech recognition model, and those skilled in the art can set it according to actual needs.

[0050] The aforementioned speech recognition model information can be the information output by the speech recognition model during the process of performing speech recognition on the speech data. For example, the speech recognition model information may include the posterior probability of the speech recognition text relative to the speech data, the posterior probability of each word or character in the speech recognition text, the frame spacing of each word or character, and the frame spacing (or time spacing) between words (or characters), etc. This application does not impose any limitations on these aspects.

[0051] In some embodiments, speech data can be processed by a speech recognition model to determine multiple candidate texts, each candidate text corresponding to a posterior probability; then the candidate text with the highest posterior probability is determined as the speech recognition text.

[0052] In some embodiments, the speech recognition model may transcribe speech word by word. During transcription, the speech recognition model may provide multiple candidate characters (or words) for each position of the text, where each character (or word) may correspond to a posterior probability. In some embodiments, the character (or word) with the highest posterior probability among the multiple candidate characters (or words) corresponding to the target position may be used as the transcribed text at that position.

[0053] In some embodiments, the speech recognition model can transcribe speech word by word and sentence by sentence. During transcription, the corresponding frame interval, frame duration, etc., can be provided for each transcribed word or character at each position.

[0054] Step S204: Extract semantic features from the speech recognition text to obtain the semantic features of the recognized text.

[0055] In some embodiments, the aforementioned semantic features of the identified text may refer to feature information extracted from the semantically identified text that can represent relevant information in the semantically identified text, such as the state information between the current word (or character) and the preceding word (or character), or the state information between the current word (or character) and the following word (or character), etc. This disclosure does not limit this.

[0056] Step S206: Extract features from the speech recognition model information to obtain speech recognition model features.

[0057] Among them, the semantic recognition model features can include relevant information in the process of speech recognition model performing speech recognition on speech data, such as the posterior probability of each word or character in the speech recognition text, or the continuous frame length of each word or character, or the frame spacing between words (or characters).

[0058] In some embodiments, the speech recognition model information may include the maximum posterior probability among multiple candidate texts, the second maximum posterior probability among candidate texts, the maximum posterior probability and the second maximum posterior probability corresponding to each character or word in the candidate texts, the continuous frames of each character or word, the frame interval between each character or word and the previous character or word, and the frame interval between each character or word and the next character or word.

[0059] In some embodiments, speech recognition model features can be generated based on the maximum posterior probability, the second largest posterior probability, the continuous frame length of each character or word, the frame spacing between each character or word and the previous character or word, and the frame spacing between each character or word and the next character or word.

[0060] In some embodiments, speech recognition model features can be generated based on the maximum posterior probability, the second largest posterior probability corresponding to each character or word, the continuous frame length of each character or word, the frame spacing between each character or word and the previous character or word, and the frame spacing between each character or word and the next character or word.

[0061] Step S208: The semantic features of the identified text and the features of the speech recognition model are fused to obtain anomaly measurement features.

[0062] Step S210: The anomaly measurement features are processed by the anomaly detection model to determine the recognition effect of recognizing speech data as speech recognition text.

[0063] In some embodiments, an anomaly measurement feature can be processed by an anomaly detection model to obtain an anomaly prediction sequence, wherein the anomaly prediction sequence may include multiple values, each value being used to predict whether a word or character at a corresponding position is transcribed abnormally.

[0064] In some embodiments, the proportion of incorrectly transcribed words and phrases in the speech recognition text can be determined based on the abnormal prediction sequence; if the proportion exceeds a target threshold, the speech recognition text is determined to be transcribed abnormally.

[0065] In the above embodiments, on the one hand, the anomaly detection model can be used to process the anomaly measurement features corresponding to the speech recognition text in order to determine whether the text transcription of speech data by the speech recognition model is normal; on the other hand, the above anomaly measurement features integrate both the semantic features of the speech recognition text and the speech recognition model features in the process of obtaining speech recognition text by transcribing semantic data, making the text transcription anomaly judgment results more accurate.

[0066] Figure 3 This is a flowchart illustrating a method for identifying semantic features of text according to an exemplary embodiment.

[0067] refer to Figure 3 The above-mentioned method for identifying semantic features of text may include the following steps.

[0068] Step S302: Determine the state information of each character or word in the speech recognition text. The state information includes the current state information, the previous state information, and the subsequent state information.

[0069] In some embodiments, the speech recognition text may include target characters or target words. The following will explain how to determine the state information of each word (or character) using target words or characters as an example.

[0070] like Figure 4 As shown, you can base your search on the target character or target word (e.g., ...). Figure 4The first n - 1 characters or words in the speech recognition text (such as Figure 4 "said" and "today" in) are used to determine the previous state information (Previous state) of the target character or target word.

[0071] For example, an n - gram language model can be used to jointly calculate the previous state information (Previous state) of the target word (or target character) with the previous n - 1 words (or characters).

[0072] Such as Figure 4 As shown, according to the first (n - 1) / 2 characters or words in the speech recognition text of the target character or target word (such as Figure 4 "day" in) and the last (n - 1) / 2 characters or words (such as Figure 4 "today" in) and the second "day" in Figure 4 the current state information of the target character or target word is determined.

[0073] For example, an n - gram language model can be used to jointly calculate the current state information (Current state) of the target word (or target character) with the first n - 1 / 2 and the last n - 1 / 2 words (or characters).

[0074] Such as Figure 4 , the subsequent state information of the target character or target word can be determined according to the last n - 1 characters or words in the speech recognition text of the target character or target word (such as Figure 4 "day" in) (such as Figure 4 "weather" in), where n is an integer greater than or equal to 1.

[0075] For example, an n - gram language model can be used to jointly calculate the subsequent state information (next state) of the target word (or target character) with the last n - 1 words (or characters).

[0076] The above n can be odd numbers such as 3, 5, 7...

[0077] Among them, the above n - gram can be obtained by training with text data in the data field where the voice data is located. For example, if the above voice data is voice data in the operator field, then text data in the operator field can be used to model the n - gram language model.

[0078] Step S304, according to the state information of each character or word in the speech recognition text, determine the character feature information of the speech recognition text.

[0079] In some embodiments, the current state information, prior state information, and subsequent state information of each character or word in the speech recognition text can be concatenated to serve as the character feature information of that character or word.

[0080] Step S306: Extract features from the speech recognition text to determine the sentence feature information of the speech recognition text.

[0081] In some embodiments, text features in speech recognition text can be extracted using a language model.

[0082] In some embodiments, a general domain language model can be trained to extract text features from speech recognition text.

[0083] Step S308: Generate semantic features of the recognized text based on character feature information and sentence feature information.

[0084] The above method can extract the state information of each word (or character) in the speech recognition text from the n-gram of the domain where the speech data is located, and can also extract the overall text features of the speech recognition text by training a language model in a general domain. This makes the semantic features of the recognition text generated by the character feature information and sentence feature information express the feature information of each character at the micro level and the text feature information at the macro level, so that the semantic feature information of the speech recognition text contains richer information, thereby making the anomaly recognition effect more accurate.

[0085] Figure 5 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment.

[0086] refer to Figure 5 The above-mentioned anomaly detection method may include the following steps.

[0087] Step S502: Obtain speech recognition text. Speech recognition text is obtained by a speech recognition model after performing speech recognition on speech data. The speech recognition model outputs speech recognition model information during the speech recognition process on the speech data.

[0088] Step S504: Extract semantic features from the speech recognition text to obtain the semantic features of the recognized text.

[0089] Step S506: Extract features from the speech recognition model information to obtain speech recognition model features.

[0090] Step S508: Process the speech data and speech recognition text to determine the speech audio length, speech recognition text length, speech energy, and speech speed of the speech data.

[0091] Step S510: Generate speech quality features based on the speech audio length, speech recognition text length, speech energy, and speech speed of the speech data.

[0092] Step S512 involves fusing the semantic features of the identified text, the features of the speech recognition model, and the speech quality features to obtain anomaly measurement features.

[0093] Step S514: The anomaly measurement features are processed by the anomaly detection model to determine the recognition effect of recognizing speech data as speech recognition text.

[0094] In the above embodiments, on the one hand, the anomaly detection model can process the anomaly measurement features corresponding to the speech recognition text in order to determine whether the text transcription of speech data by the speech recognition model is normal; on the other hand, the above anomaly measurement features not only integrate the semantic features of the speech recognition text, but also the speech recognition model features in the process of obtaining speech recognition text by transcribing semantic data, and also integrate the audio length, recognized text length, speech energy and speech speed of the speech data, making the text transcription anomaly judgment result more accurate.

[0095] Figure 6 This is a flowchart illustrating an anomaly detection model training method according to an exemplary embodiment.

[0096] refer to Figure 6 The above-mentioned anomaly detection model training method may include the following steps.

[0097] Step S602: The anomaly measurement features are processed by the anomaly detection model to determine the anomaly prediction sequence. The anomaly prediction sequence is used to indicate whether the characters or words in the speech recognition text are transcribed correctly.

[0098] Step S604: Obtain the anomaly annotation sequence. The anomaly annotation sequence is used to annotate whether the characters or words in the speech recognition text are transcribed correctly.

[0099] The process of obtaining an anomaly annotation sequence may include the following steps: obtaining the annotation text corresponding to the speech data; aligning the annotation text with the speech recognition text; and generating an anomaly annotation sequence based on the aligned annotation text and the aligned speech recognition text.

[0100] Step S606: Train the anomaly detection model using anomaly labeling sequences and anomaly prediction sequences.

[0101] Through the above embodiments, the pre-annotated text anomaly annotation sequence and the anomaly prediction sequence of speech recognition files predicted using the anomaly prediction model can be compared to train the anomaly prediction model and thus improve the prediction accuracy of the anomaly prediction model.

[0102] In related technologies, deep neural network models (such as end-to-end deep neural network models) have achieved excellent results in speech recognition tasks and have empowered various applications in the industry. One application scenario is text transcription in customer service to assist customer service personnel in providing intelligent services. Although current cutting-edge speech recognition models have achieved good accuracy, compared with ordinary text transcription tasks, text transcription tasks in customer service scenarios, which are more complex and involve a more diverse customer base, encounter an unavoidable difficulty: the recognition effect of massive and diverse unlabeled speech data online is unpredictable, and anomalies in online speech recognition tasks may cause irreparable losses to downstream tasks. Text transcription tasks in customer service scenarios may encounter various unexpected anomalies, such as multiple dialects, highly colloquial language, domain-specific vocabulary, and new vocabulary, which may lead to a deterioration in the model's recognition performance. Detecting transcription errors caused by these anomalies is of great significance for both timely correction of text errors to avoid impacting downstream tasks and for optimizing speech recognition models.

[0103] For current speech recognition models, anomaly detection methods in text transcription can be divided into the following two categories:

[0104] 1) Regarding speech recognition models: The posterior probability of the speech recognition output is directly used as the confidence score. Transcription results with too low a score are judged as recognition errors. This method relies too much on the performance of the speech recognition model, resulting in poor detection of the results.

[0105] 2) Regarding text recognition: The speech recognition model outputs sentences and uses a language model to estimate the semantic level, obtains sentences with perplexity scores, and then uses a threshold to determine whether there are any abnormalities. This method only uses semantic information, which leads to the inability to guarantee the evaluation effect.

[0106] To address the aforementioned technical problems, this application proposes the following technical solutions.

[0107] like Figure 7 As shown, the technical solution proposed in this application is mainly divided into three stages: speech recognition, abnormal situation fusion feature extraction, and abnormal detection module.

[0108] The overall flowchart is as follows: Figure 7 As shown, firstly, the end-to-end speech recognition model recognizes the speech data. Then, for the speech recognition process and results, the three feature extractors proposed in this application extract speech quality features, semantic features of the recognized text, and features of the speech recognition model, respectively. The results of the above feature extractors are fused to form anomaly measurement features. Finally, an anomaly detection model based on a bidirectional GRU neural network is established in the anomaly detection module to predict anomalies in speech recognition.

[0109] Phase 1: Speech Recognition

[0110] (1) Select the current mainstream end-to-end speech recognition model, recognize the speech data to obtain the recognized text. For each audio segment with a duration of T [t1,t2,…tT], the corresponding recognized text [c1,c2,…cn] with a length of n can be obtained. Among them, the end-to-end speech recognition model is a model built based on the CTC algorithm and deep neural network. Here, the relevant internal structure of the speech recognition model will not be described in detail. This application directly selects the frozen model after training for speech recognition. For each frame of audio, the character c with the highest probability in the dictionary Y is obtained based on the following recognition formula (1), and the final text sequence [c1,c2,…cn] is obtained based on the CTC algorithm.

[0111] P(c)=max(P(y|x))) (1)

[0112] (2) If the length of the labeled text for each audio segment is L, let it be [c1',c2',…cL'], then aligning it with the above speech recognition text will yield an anomaly labeling sequence [x1,x2,…xd] of length d. For each labeled text and recognized text, if they are the same, x is 0; otherwise, it is 1. Insertion and replacement character errors in the text are replaced with spaces, so d is the larger value between n and L.

[0113] Phase Two: Feature Extraction for Anomaly Fusion

[0114] The abnormal situation fusion feature extraction is divided into three parts: speech quality feature extraction, recognition text semantic feature extraction, and speech recognition model feature extraction.

[0115] (1) Speech quality feature extraction: For each speech segment, calculate the speech audio length, speech recognition text length, speech energy, and speech velocity. Merge the speech quality features of each speech segment to obtain the feature matrix a. i .

[0116] (2) Extraction of semantic features of the identified text: The n-gram language model is modeled using the text data of the operator domain. Then, for each identified text of the operator domain speech, the current state, previous state and next state are calculated based on the n-gram language model.

[0117] The previous state is calculated using an n-gram language model, combining the currently recognized text with the preceding n-1 texts. The current state is calculated based on the current text, the preceding n-1 / 2 texts, and the following n-1 / 2 texts. The next state is calculated based on the current text and the following n-1 texts, where n is typically 3 or 5. A general-domain pre-trained language model is used to extract text features, which are then merged with the semantic features to obtain the semantic feature matrix e. i .

[0118] (3) End-to-end speech recognition model related feature extraction: For the recognized text of each speech audio, the following speech recognition model related information is stored:

[0119] 1. Maximum a posteriori probability: represents the degree of confidence the model has in selecting the result of each audio frame in the candidate set.

[0120] 2. Second largest posterior probability: The second largest candidate in the candidate set, which can reflect the degree of confusion of the model about the result.

[0121] 3. Continuous frame length: The number of frames contained in each recognized text (character).

[0122] 4. Frame spacing between the previous recognized character or word: The CTC algorithm merges identical text frames during decoding, and this metric can be used to measure the effectiveness of the CTC algorithm.

[0123] 5. Frame spacing between the next character or word.

[0124] 6. The final model feature matrix m is obtained for each speech segment. i .

[0125] (4) The above feature matrices can be further fused by referring to formula (2). For different speech, a more reasonable anomaly measurement fusion feature expression x' can be obtained by setting hyperparameter weights for different types of features. The fusion feature calculation method can be obtained by setting hyperparameter weights for different types of features:

[0126] x'=αa i +βe i +γm i (2)

[0127] Phase 3: Anomaly Detection Module

[0128] In this stage, based on the extracted speech quality features, semantic features of the recognized text, and speech recognition model features, an anomaly detection encoder-decoder based on a bidirectional GRU neural network is established to predict anomalies in the speech recognition text.

[0129] To further integrate the multi-source fusion features obtained in Phase 2, an encoder based on a bidirectional GRU neural network was established. Subsequently, a fully connected neural network was used as the decoder to predict the anomaly probability.

[0130] The bidirectional GRU neural network is described by formulas (3) to (4):

[0131]

[0132]

[0133]

[0134] Wherein, ei represents the multi-source fusion feature obtained in Phase 2. or These are the parameters in the bidirectional GRU neural network. The decoder predicts anomalies based on the following formula:

[0135] Pi=P(yi=1|X)=σ(Whi+b) (6)

[0136] The closer the probability of recognizing text is to 1, the greater the probability of an anomaly.

[0137] The model is trained based on the labeled data of the anomaly detection model obtained in Phase 1. An anomaly detection model is obtained by setting an appropriate threshold. Online data with a probability greater than the threshold output by the anomaly detection model are judged to contain anomalies.

[0138] This application proposes an automatic anomaly detection algorithm for speech recognition based on bidirectional GRU neural networks and multi-source feature fusion technology, targeting complex operator scenarios. It innovatively proposes a multi-level feature construction method for speech quality, speech recognition model, and recognized text throughout the entire speech recognition pipeline process, forming multi-source fusion features and establishing an anomaly detection module to automatically detect erroneous text in speech recognition. This solves the problem that current online speech recognition models cannot evaluate the recognition effect of unlabeled speech data, avoids the negative impact of anomaly detection on downstream tasks, and can further optimize the effect and shortcomings of the speech recognition model based on the detection results.

[0139] The method described in this application can be used to evaluate the recognition performance of a speech recognition model and to detect recognition anomalies in a speech recognition model.

[0140] This application addresses the anomaly detection problem in speech recognition under complex operator scenarios involving multiple dialects and user groups. It proposes an automatic anomaly detection algorithm based on bidirectional GRU and multi-source feature fusion. The algorithm innovatively utilizes multi-level information such as speech quality and environmental influences contained in the speech, semantic information contained in the recognized text, and recognition performance information contained in the speech recognition model to generate anomaly fusion features. It then constructs an anomaly detection encoder-decoder based on a bidirectional GRU neural network, thereby automatically predicting anomalies in the speech recognition results.

[0141] Based on the same inventive concept, this disclosure also provides an anomaly detection device, as shown in the following embodiment. Since the principle by which this device solves the problem is similar to that of the method embodiment described above, the implementation of this device embodiment can refer to the implementation of the method embodiment described above, and repeated details will not be elaborated further.

[0142] Figure 8 This is a block diagram illustrating an anomaly detection device according to an exemplary embodiment. (Refer to...) Figure 8 The anomaly detection device 800 provided in this embodiment may include: a speech recognition text acquisition module 801, a semantic feature extraction module 802, a speech recognition model feature acquisition module 803, a fusion module 804, and an anomaly judgment module 805.

[0143] The speech recognition text acquisition module 801 is used to acquire speech recognition text, which is obtained by a speech recognition model after performing speech recognition on speech data. The speech recognition model outputs speech recognition model information during the speech recognition process. The semantic feature extraction module 802 is used to extract semantic features from the speech recognition text to obtain semantic features of the recognized text. The speech recognition model feature acquisition module 803 is used to extract features from the speech recognition model information to obtain speech recognition model features. The fusion module 804 is used to fuse the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features. The anomaly judgment module 805 is used to process the anomaly measurement features through an anomaly detection model to determine the recognition effect of recognizing speech data as speech recognition text.

[0144] It should be noted that the aforementioned speech recognition text acquisition module 801, semantic feature extraction module 802, speech recognition model feature acquisition module 803, fusion module 804, and anomaly detection module 805 correspond to S202 to S210 in the method embodiment. The examples and application scenarios implemented by these modules and their corresponding steps are the same, but they are not limited to the content disclosed in the above method embodiment. It should also be noted that these modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0145] In some embodiments, the semantic feature extraction module 802 may include: a state information determination unit, a character feature information determination unit, a sentence feature information determination unit, and a text semantic feature determination unit.

[0146] The state information determination unit can be used to determine the state information of each character or word in the speech recognition text, including the current state information, the previous state information, and the subsequent state information; the character feature information determination unit can be used to determine the character feature information of the speech recognition text based on the state information of each character or word in the speech recognition text; the sentence feature information determination unit can be used to extract features from the speech recognition text and determine the sentence feature information of the speech recognition text; and the semantic feature determination unit can be used to generate semantic features of the recognition text based on the character feature information and the sentence feature information.

[0147] In some embodiments, the speech recognition text includes target characters or target words; wherein, the state information determination unit may include: a prior state information determination character unit, a current state information determination subunit, and a subsequent state information determination subunit.

[0148] The prior state information determination unit can be used to determine the prior state information of the target character or word based on the first n-1 characters or words in the speech recognition text; the current state information determination subunit can be used to determine the current state information of the target character or word based on the first (n-1) / 2 characters or words and the last (n-1) / 2 characters or words in the speech recognition text; and the subsequent state information determination subunit can be used to determine the subsequent state information of the target character or word based on the last n-1 characters or words in the speech recognition text, where n is an integer greater than or equal to 1.

[0149] In some embodiments, the anomaly detection device may include a posterior probability determination module and a speech recognition text determination module.

[0150] The posterior probability determination module can be used to perform speech recognition processing on the speech data through a speech recognition model before obtaining the speech recognition text, so as to determine multiple candidate texts, each of which corresponds to a posterior probability; the speech recognition text determination module can be used to determine the candidate text with the highest posterior probability as the speech recognition text.

[0151] In some embodiments, the speech recognition model information includes the maximum posterior probability among multiple candidate texts, the second largest posterior probability among candidate texts, the continuous frames of each character or word, the frame interval between each character or word and the previous character or word, and the frame interval between each character or word and the next character or word; wherein, the speech recognition model feature acquisition module 803 may include: speech recognition model features.

[0152] Among them, speech recognition model features can be used to generate speech recognition model features based on the maximum posterior probability, the second largest posterior probability, the continuous frame length of each character or word, the frame spacing between each character or word and the previous character or word, and the frame spacing between each character or word and the next character or word.

[0153] In some embodiments, the anomaly detection device may further include: a voice quality determination module and a feature fusion module.

[0154] The speech quality determination module can be used to process speech data and speech recognition text to determine the speech audio length, speech recognition text length, speech energy, and speech speed of the speech data; the feature fusion module can be used to generate speech quality features based on the speech audio length, speech recognition text length, speech energy, and speech speed of the speech data; the fusion module 804 can also be used to fuse the semantic features of the recognition text, the speech recognition model features, and the speech quality features to obtain anomaly measurement features.

[0155] In some embodiments, the anomaly detection module 805 may include: an anomaly prediction sequence determination unit, an anomaly annotation sequence acquisition unit, and a training unit.

[0156] The anomaly prediction sequence determination unit can be used to process the anomaly measurement features through the anomaly detection model to determine the anomaly prediction sequence, which is used to indicate whether the characters or words in the speech recognition text are transcribed correctly; the anomaly standard sequence acquisition unit can be used to acquire the anomaly annotation sequence, which is used to annotate whether the characters or words in the speech recognition text are transcribed correctly; and the training unit can be used to train the anomaly detection model through the anomaly annotation sequence and the anomaly prediction sequence.

[0157] In some embodiments, the anomaly standard sequence acquisition unit may include: a standard text acquisition subunit, an alignment subunit, and an anomaly annotation sequence acquisition subunit.

[0158] Among them, the standard text acquisition subunit can be used to acquire the labeled text corresponding to the speech data; the alignment subunit can be used to align the labeled text with the speech recognition text; and the abnormal label sequence acquisition subunit can be used to generate an abnormal label sequence based on the aligned labeled text and the aligned speech recognition text.

[0159] In some embodiments, the anomaly detection module 805 may include a ratio determination unit and a transcription anomaly detection unit.

[0160] The proportion determination unit can be used to determine the proportion of incorrectly transcribed words and phrases in the speech recognition text based on the abnormal prediction sequence; the transcription anomaly judgment unit can be used to determine that the speech recognition text is transcribed abnormal if the proportion exceeds the target threshold.

[0161] Since the functions of the device 800 have been described in detail in their respective method embodiments, they will not be repeated here.

[0162] The modules and / or units and / or sub-units described in the embodiments of this application can be implemented in software or hardware. The described modules and / or units and / or sub-units can also be located in a processor. The names of these modules and / or units and / or sub-units do not, in some cases, constitute a limitation on the module and / or unit and / or sub-unit itself.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0164] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0165] Figure 9 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. It should be noted that... Figure 9 The electronic device 900 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0166] like Figure 9 As shown, the electronic device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0167] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0168] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of this application.

[0169] It should be noted that the computer-readable storage medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0170] In another aspect, this application also provides a computer-readable storage medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the device, enable the device to perform the following functions: acquiring speech-recognized text, wherein the speech-recognized text is obtained by performing speech recognition on speech data using a speech recognition model, and the speech recognition model outputs speech recognition model information during the speech recognition process; extracting semantic features from the speech-recognized text to obtain semantic features of the recognized text; extracting features from the speech recognition model information to obtain speech recognition model features; fusing the semantic features of the recognized text and the speech recognition model features to obtain anomaly measurement features; and processing the anomaly measurement features using an anomaly detection model to determine the recognition effect of recognizing speech data as speech-recognized text.

[0171] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0172] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or smart device, etc.) to execute the method according to the embodiments of this disclosure, for example... Figure 2 , Figure 3 , Figures 5-7 One or more of the steps shown in the diagram.

[0173] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0174] It should be understood that this disclosure is not limited to the detailed structures, drawing arrangements or implementations shown herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. An anomaly detection method characterized by, The method comprises: obtaining speech recognition text, wherein the speech recognition text is obtained by performing speech recognition on speech data by using a speech recognition model, and the speech recognition model outputs speech recognition model information in the process of performing speech recognition on the speech data; the speech recognition model information comprises maximum posterior probability in multiple candidate texts, second maximum posterior probability in the candidate texts, duration of each word or character, frame interval between each word or character and a previous word or character, and frame interval between each word or character and a next word or character; performing semantic feature extraction on the speech recognition text to obtain recognition text semantic features; performing feature extraction on the speech recognition model information to obtain speech recognition model features; the speech recognition model features comprise posterior probability of each word or character in the speech recognition text, or duration of each word or character, or frame interval between words or characters; fusing the recognition text semantic features and the speech recognition model features to obtain abnormality measurement features; processing the abnormality measurement features by using an abnormality detection model to determine recognition effect of the speech recognition model in transcribing the speech data into the speech recognition text; wherein processing the abnormality measurement features by using the abnormality detection model to determine the recognition effect of the speech recognition model in transcribing the speech data into the speech recognition text comprises: processing the abnormality measurement features by using the abnormality detection model to determine an abnormality prediction sequence, wherein the abnormality prediction sequence is used to indicate whether a word or character in the speech recognition text is transcribed correctly; obtaining an abnormality annotation sequence, wherein the abnormality annotation sequence is used to annotate whether a word or character in the speech recognition text is transcribed correctly; training the abnormality detection model by using the abnormality annotation sequence and the abnormality prediction sequence.

2. The method of claim 1, wherein, performing semantic feature extraction on the speech recognition text to obtain recognition text semantic features comprises: determining state information of each word or character in the speech recognition text, wherein the state information comprises current state information, previous state information and subsequent state information; determining word feature information of the speech recognition text according to the state information of each word or character in the speech recognition text; performing feature extraction on the speech recognition text to determine sentence feature information of the speech recognition text; generating the recognition text semantic features according to the word feature information and the sentence feature information.

3. The method of claim 2, wherein, the speech recognition text comprises a target word or a target character; wherein determining state information of each word or character in the speech recognition text, wherein the state information comprises current state information, previous state information and subsequent state information, comprises: determining previous state information of the target word or the target character according to (n-1) previous words or characters of the target word or the target character in the speech recognition text; determining current state information of the target word or the target character according to (n-1) / 2 previous words or characters and (n-1) / 2 subsequent words or characters of the target word or the target character in the speech recognition text; Determine the post-state information of the target word or character according to n-1 words or characters after the target word or character in the speech recognition text, n being an integer greater than or equal to 1.

4. The method of claim 1, wherein, Before obtaining the speech recognition text, the method further comprises: performing speech recognition processing on the speech data by the speech recognition model to determine a plurality of candidate texts, wherein each candidate text corresponds to a posterior probability respectively; determine the candidate text with the maximum posterior probability as the speech recognition text.

5. The method of claim 4, wherein, perform feature extraction on the speech recognition model information to obtain speech recognition model features, comprising: generate the speech recognition model features according to the maximum posterior probability, the second maximum posterior probability, the duration frame length of each word or character, the frame interval between each word or character and the previous word or character, and the frame interval between each word or character and the next word or character.

6. The method of claim 1, wherein, The method further comprises: processing the speech data and the speech recognition text to determine the speech audio length, the speech recognition text length, the speech energy, and the speech speed of the speech data; generate speech quality features according to the speech audio length, the speech recognition text length, the speech energy, and the speech speed of the speech data; wherein the recognition text semantic features and the speech recognition model features are fused to obtain the abnormality measurement features, comprising: fuse the recognition text semantic features, the speech recognition model features, and the speech quality features to obtain the abnormality measurement features.

7. The method of claim 1, wherein, Obtain an abnormality annotation sequence, comprising: obtain an annotation text corresponding to the speech data; align the annotation text with the speech recognition text; generate the abnormality annotation sequence according to the aligned annotation text and the aligned speech recognition text.

8. The method of claim 1, wherein, Determine the recognition effect of recognizing the speech data as the speech recognition text, comprising: determine the proportion of transcribed incorrect words and characters in the speech recognition text according to the abnormality prediction sequence; if the proportion exceeds a target threshold, determine that the speech recognition text transcribes abnormally.

9. An abnormality detection device characterized by comprising: comprising: a speech recognition text acquisition module, configured to acquire a speech recognition text, the speech recognition text being obtained by performing speech recognition on speech data by a speech recognition model, the speech recognition model outputting speech recognition model information in the process of performing speech recognition on the speech data; the speech recognition model information comprising a maximum posterior probability in a plurality of candidate texts, a second maximum posterior probability in the candidate texts, a duration frame of each word or character, a frame interval between each word or character and the previous word or character, and a frame interval between each word or character and the next word or character; a semantic feature extraction module, configured to perform semantic feature extraction on the speech recognition text to obtain recognition text semantic features; a speech recognition model feature acquisition module, configured to perform feature extraction on the speech recognition model information to obtain speech recognition model features; the speech recognition model features comprising a posterior probability of each word or character in the speech recognition text, or a duration frame length of each word or character, or a frame interval between words or characters. a fusion module configured to fuse the identified text semantic features and the speech recognition model features to obtain abnormality measurement features; an abnormality judgment module configured to process the abnormality measurement features by an abnormality detection model to determine a recognition effect of recognizing the speech data as the speech recognition text; wherein processing the abnormality measurement features by the abnormality detection model to determine the recognition effect of the speech recognition model in transcribing the speech data into the speech recognition text comprises: processing the abnormality measurement features by the abnormality detection model to determine an abnormality prediction sequence, the abnormality prediction sequence being configured to indicate whether a word or character in the speech recognition text is transcribed correctly; obtaining an abnormality annotation sequence, the abnormality annotation sequence being configured to annotate whether a word or character in the speech recognition text is transcribed correctly; training the abnormality detection model by the abnormality annotation sequence and the abnormality prediction sequence.

10. An electronic device, comprising: comprise: a memory; and a processor coupled to the memory, the processor being configured to execute instructions stored in the memory to perform the abnormality detection method according to any one of claims 1-8. 11.A machine readable storage medium having stored thereon program instructions that, when executed by a processor, implement the abnormality detection method according to any one of claims 1-8. ​

Images