Dialect data automatic screening and identification method, device and equipment and storage medium thereof

Abstract

The embodiment of the application belongs to the technical field of identification process optimization, and relates to a dialect data automatic screening and identification method, device and equipment and a storage medium thereof, which are used for assisting an electric salesperson in prior identification of dialect data, and include the following steps: inputting a to-be-tested voice sequence into a pre-trained mixed structure type ASR voice recognition model to obtain a first text sequence; inputting the to-be-tested voice sequence into a pre-trained end-to-end type ASR voice recognition model to obtain a second text sequence; calculating an edit distance of the first text sequence and the second text sequence; and identifying whether the to-be-tested voice sequence is a dialect sequence through the edit distance. Two different ASR models of common speech are used to identify the same voice sequence. If the voice sequence is common speech, the edit distance is small. If the voice sequence is a dialect, the difference between the two ASR model recognition results is large, and the edit distance is far. The method solves the problem that it is difficult to collect a large amount of dialect corpus, and overcomes the technical prejudice that a recognition model must be trained by using dialect corpus.

Description

Technical Field

[0001] This application relates to the field of recognition process optimization technology, and is used to assist telemarketers in performing prior recognition of dialect data. In particular, it relates to a method, device, equipment and storage medium for automatic screening and recognition of dialect data. Background Technology

[0002] With the rise of the fintech industry, telemarketing customer service has long been the mainstream method for direct voice communication with customers in the financial sector. However, due to language family differences, dialect recognition has always been a major challenge in speech recognition. Speech recognition systems that support dialects require collecting and labeling dialect speech data before using this labeled data to train a speech recognition model. The main problem with dialect recognition is the scarcity and difficulty in obtaining dialect data. In real-world production scenarios, dialect data may only account for 1% of the total. This means that manually listening to 10,000 recordings might only yield one dialect data point, resulting in extremely low efficiency and high cost in dialect data collection. Traditional dialect data filtering often requires first training a dialect recognition model to filter the data, but this model also requires prior training with dialect data. Then, a corresponding dialect recognition model is trained to filter the dialect data.

[0003] Therefore, existing technologies for dialect data recognition suffer from a technical bias of always trying to train recognition models with dialect data, and there is also the problem of data scarcity in obtaining dialect data. Summary of the Invention

[0004] The purpose of this application is to propose an automatic dialect data screening and recognition method, apparatus, device and storage medium, in order to solve the technical bias of the prior art in dialect data recognition, which always wants to train the recognition model with dialect data, and the problem of scarce dialect data.

[0005] To address the aforementioned technical problems, this application provides an automatic dialect data filtering and recognition method, employing the following technical solution:

[0006] An automatic dialect data filtering and recognition method includes the following steps:

[0007] Obtain the speech sequence to be tested and the source region identifier of the speech sequence to be tested;

[0008] The speech sequence to be tested is input into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture.

[0009] The speech sequence to be tested is input into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture mode.

[0010] The edit distance between the first text sequence and the second text sequence is obtained by comparison calculation.

[0011] Based on a preset difference threshold and the edit distance, it is determined whether the speech sequence to be tested is a dialect sequence;

[0012] If the speech sequence to be tested is a dialect sequence, the dialect category to which the dialect sequence belongs is determined by the source region identifier of the speech sequence to be tested.

[0013] Furthermore, after performing the step of obtaining the speech sequence to be tested and the source region identifier of the speech sequence to be tested, the method further includes:

[0014] Acquire pre-collected Mandarin speech recognition data;

[0015] According to a preset semantic annotation method, each Mandarin sequence in the Mandarin speech recognition corpus is semantically annotated to obtain an annotated Mandarin speech recognition corpus. The semantic annotation method includes using a trained Chinese natural language processing model for semantic annotation.

[0016] According to a preset ratio, the annotated Mandarin speech recognition corpus is divided into a training set and a test set.

[0017] Furthermore, before executing the step of inputting the test speech sequence into the pre-trained hybrid structured ASR speech recognition model to obtain the first text sequence, the method further includes:

[0018] The training set is input into the initialized hybrid structured ASR speech recognition model to perform model pre-training, thereby obtaining a hybrid structured ASR speech recognition model that has been initially trained.

[0019] The test set is input into the pre-trained hybrid structured ASR speech recognition model for recognition verification.

[0020] If the recognition verification fails, the training parameters of the initialized hybrid structured ASR speech recognition model are adjusted, and the hybrid structured ASR speech recognition model is re-pre-trained.

[0021] If the recognition verification is successful, a pre-trained hybrid structured ASR speech recognition model is obtained.

[0022] Furthermore, before executing the step of inputting the test speech sequence into the pre-trained end-to-end ASR speech recognition model to obtain the second text sequence, the method further includes:

[0023] The training set is input into the initialized end-to-end ASR speech recognition model to perform model pre-training and obtain a pre-trained end-to-end ASR speech recognition model.

[0024] The test set is input into the pre-trained end-to-end ASR speech recognition model for recognition verification.

[0025] If the recognition verification fails, the training parameters of the initialized end-to-end ASR speech recognition model are adjusted, and the end-to-end ASR speech recognition model is re-pre-trained.

[0026] If the recognition and verification are successful, a pre-trained end-to-end ASR speech recognition model is obtained.

[0027] Furthermore, the step of obtaining the edit distance between the first text sequence and the second text sequence using a comparative calculation method specifically includes:

[0028] Obtain the text content and number of characters in the first text sequence;

[0029] Obtain the text content and number of characters in the second text sequence;

[0030] By comparing the two text sequences, the different text contents in the first text sequence and the second text sequence are obtained.

[0031] Count the total number of characters in the first text sequence and the second text sequence;

[0032] Calculate the percentage of the different text content in the total number of texts;

[0033] The percentage value is set as the edit distance between the first text sequence and the second text sequence.

[0034] Furthermore, the step of identifying whether the speech sequence to be tested is a dialect sequence based on a preset difference threshold and the edit distance specifically includes:

[0035] Determine whether the edit distance exceeds a preset difference threshold;

[0036] If the limit is not exceeded, then the speech sequence to be tested is a Mandarin sequence;

[0037] If the number of cases exceeds a certain threshold, then the speech sequence to be tested is a dialect sequence.

[0038] Furthermore, before performing the step of determining the dialect category of the dialect sequence by using the source region identifier of the speech sequence to be tested, the method further includes:

[0039] Obtain a dialect category table pre-defined based on regional information, wherein the regional information includes national administrative division information;

[0040] The step of determining the dialect category of the dialect sequence by using the source region identifier of the speech sequence to be tested specifically includes:

[0041] Based on the national administrative division information, the region information corresponding to the source region identifier is identified;

[0042] Based on the regional information corresponding to the source region identifier and the dialect category table, the dialect category to which the dialect sequence belongs is determined.

[0043] To address the aforementioned technical problems, this application also provides an automatic dialect data filtering and recognition device, which employs the following technical solution:

[0044] An automatic dialect data filtering and recognition device, comprising:

[0045] The test data preparation module is used to obtain the test speech sequence and the source region identifier of the test speech sequence;

[0046] The first text sequence acquisition module is used to input the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to acquire the first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture.

[0047] The second text sequence acquisition module is used to input the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to acquire the second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture mode.

[0048] The edit distance calculation module is used to obtain the edit distance between the first text sequence and the second text sequence by comparison calculation.

[0049] The dialect sequence recognition module is used to identify whether the speech sequence to be tested is a dialect sequence based on a preset difference threshold and the edit distance;

[0050] The dialect category determination module is used to determine the dialect category to which the speech sequence to be tested belongs by means of the source region identifier of the speech sequence to be tested if the speech sequence to be tested is a dialect sequence.

[0051] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0052] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the dialect data automatic screening and recognition method described above.

[0053] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0054] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the dialect data automatic filtering and recognition method described above.

[0055] Compared with the prior art, the embodiments of this application have the following main advantages:

[0056] The dialect data automatic filtering and recognition method described in this application involves: acquiring a speech sequence to be tested and its source region identifier; inputting the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture; inputting the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture; obtaining the edit distance between the first text sequence and the second text sequence using a comparative calculation method; identifying whether the speech sequence to be tested is a dialect sequence based on a preset difference threshold and the edit distance; if the speech sequence to be tested is a dialect sequence, determining the dialect category to which the dialect sequence belongs based on the source region identifier of the speech sequence to be tested. Using two different ASR models for Mandarin, the same speech sequence can be identified. If it is Mandarin, both can be detected with a small edit distance. However, if it is a dialect, the two ASR models show a large difference in recognition results with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias that recognition models must be trained with dialect data. Attached Figure Description

[0057] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0059] Figure 2 A flowchart of an embodiment of the dialect data automatic screening and recognition method according to this application;

[0060] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 204 shown;

[0061] Figure 4 A schematic diagram of the structure of an embodiment of the dialect data automatic filtering and recognition device according to this application;

[0062] Figure 5 yes Figure 4 A schematic diagram of a specific embodiment of module 404 is shown;

[0063] Figure 6 A schematic diagram of the structure of an embodiment of the computer device according to this application. Detailed Implementation

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0065] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0066] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0067] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0068] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0069] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0070] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0071] It should be noted that the dialect data automatic filtering and recognition method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the dialect data automatic filtering and recognition device is generally set in the server / terminal device.

[0072] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0073] To facilitate understanding, let's first introduce the development process of ASR speech recognition models. Initially, ASR speech recognition models were built based on GMM (Gaussian Mixture Model)-HMM (Hidden Markov Model). With the rapid development of deep learning neural networks, DNN (Deep Neural Network)-HMM (Hidden Markov Model) gradually replaced GMM-HMM in building ASR speech recognition models. With the introduction of recurrent neural networks and end-to-end models, the CTC (Connectionist Temporal Classification) criterion + LSTM (Long Short-Term Memory) model was gradually adopted to build end-to-end ASR speech recognition models.

[0074] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the dialect data automatic screening and recognition method according to this application. The dialect data automatic screening and recognition method includes the following steps:

[0075] Step 201: Obtain the speech sequence to be tested and the source region identifier of the speech sequence to be tested.

[0076] In this embodiment, the speech sequence to be tested can be a speech sequence collected through a telemarketing agent terminal, and the speech sequence includes a Mandarin speech sequence or a dialect speech sequence.

[0077] In this embodiment, after performing the steps of obtaining the speech sequence to be tested and the source region identifier of the speech sequence to be tested, the method further includes: obtaining pre-collected Mandarin speech recognition corpus; performing semantic annotation on each Mandarin sequence in the Mandarin speech recognition corpus according to a preset semantic annotation method to obtain annotated Mandarin speech recognition corpus, wherein the semantic annotation method includes using a trained Chinese natural language processing model for semantic annotation; and dividing the annotated Mandarin speech recognition corpus into a training set and a test set according to a preset ratio.

[0078] By pre-collecting Mandarin speech recognition corpora and semantically annotating them, this approach avoids the previous problem of needing to collect dialect data beforehand, which is often scarce. A reverse testing method is employed, using more readily available Mandarin speech recognition corpora as the model training data. This solves the problem of needing to collect forward corpora for dialect data recognition, which is difficult to achieve. Here, forward corpora refer to dialect data collected directly for dialect recognition as training data; the dialect data itself constitutes the forward corpora.

[0079] Step 202: Input the speech sequence to be tested into the pre-trained hybrid structured ASR speech recognition model to obtain the first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture.

[0080] In this embodiment, before executing the step of inputting the speech sequence to be tested into the pre-trained hybrid structured ASR speech recognition model to obtain the first text sequence, the method further includes: inputting the training set into the initialized hybrid structured ASR speech recognition model to perform model pre-training, thereby obtaining a pre-trained hybrid structured ASR speech recognition model; inputting the test set into the pre-trained hybrid structured ASR speech recognition model for recognition verification; if the recognition verification fails, adjusting the training parameters of the initialized hybrid structured ASR speech recognition model and re-pre-training the hybrid structured ASR speech recognition model; if the recognition verification succeeds, obtaining the pre-trained hybrid structured ASR speech recognition model.

[0081] A hybrid structured ASR speech recognition model is trained using Mandarin speech recognition corpus. Then, the pre-trained hybrid structured ASR speech recognition model is used to recognize unknown test speech sequences. If the sequence is Mandarin, the corresponding output result is obtained; if it is not Mandarin, the corresponding output sequence can also be output.

[0082] Step 203: Input the speech sequence to be tested into the pre-trained end-to-end ASR speech recognition model to obtain the second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture mode.

[0083] In this embodiment, the construction of the end-to-end ASR speech recognition model is based on the original CTC criterion + LSTM to build an end-to-end ASR speech recognition model. A neural network attention mechanism is introduced, and the Conformer model is used to replace the LSTM model. The Conformer model is a speech recognition model proposed by Google in 2020, which is based on the Transformer. The main improvement is that the Transformer is more effective in extracting long sequence dependencies, while convolution is good at extracting local features. Therefore, convolution is applied to the Encoder layer of the Transformer to improve the model's performance on both long sequence and local features.

[0084] In this embodiment, before executing the step of inputting the speech sequence to be tested into the pre-trained end-to-end ASR speech recognition model to obtain the second text sequence, the method further includes: inputting the training set into the initialized end-to-end ASR speech recognition model for model pre-training to obtain a pre-trained end-to-end ASR speech recognition model; inputting the test set into the pre-trained end-to-end ASR speech recognition model for recognition verification; if the recognition verification fails, adjusting the training parameters of the initialized end-to-end ASR speech recognition model and re-pre-training the end-to-end ASR speech recognition model; if the recognition verification succeeds, obtaining the pre-trained end-to-end ASR speech recognition model.

[0085] An end-to-end ASR speech recognition model is trained using Mandarin speech recognition corpus. Then, the pre-trained end-to-end ASR speech recognition model is used to recognize unknown test speech sequences. If the sequence is Mandarin, the corresponding output result is obtained; if it is not Mandarin, the corresponding output sequence can also be output.

[0086] Step 204: Use a comparison calculation method to obtain the edit distance between the first text sequence and the second text sequence.

[0087] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0088] Step 301: Obtain the text content and number of characters in the first text sequence;

[0089] Step 302: Obtain the text content and number of characters in the second text sequence;

[0090] Step 303: Using a comparison method, obtain the different text content in the first text sequence and the second text sequence;

[0091] Step 304: Count the total number of characters in the first text sequence and the second text sequence;

[0092] Step 305: Calculate the percentage of the different text contents in the total number of texts;

[0093] Step 306: Set the percentage value as the edit distance between the first text sequence and the second text sequence.

[0094] The edit distance between the first text sequence output by the hybrid structured ASR speech recognition model and the second text sequence output by the end-to-end ASR speech recognition model is calculated. Clearly, if the unknown speech sequence is a Mandarin sequence, the edit distance between the first and second text sequences is smaller; conversely, if the unknown speech sequence is a dialect sequence, the edit distance is larger. Based on this characteristic, determining whether the speech sequence to be tested is a dialect sequence using text sequences output by two different models is more scientific and reasonable.

[0095] Step 205: Based on the preset difference threshold and the edit distance, identify whether the speech sequence to be tested is a dialect sequence.

[0096] In this embodiment, the step of identifying whether the speech sequence to be tested is a dialect sequence based on a preset difference threshold and the edit distance specifically includes: determining whether the edit distance exceeds a preset difference threshold; if it does not exceed the threshold, the speech sequence to be tested is a Mandarin sequence; if it exceeds the threshold, the speech sequence to be tested is a dialect sequence.

[0097] Step 206: If the speech sequence to be tested is a dialect sequence, then the dialect category to which the dialect sequence belongs is determined by the source region identifier of the speech sequence to be tested.

[0098] In this embodiment, before performing the step of determining the dialect category of the dialect sequence by means of the source region identifier of the speech sequence to be tested, the method further includes: obtaining a dialect category table pre-divided according to regional information, wherein the regional information includes national administrative division information;

[0099] In this embodiment, the step of determining the dialect category of the dialect sequence by means of the source region identifier of the speech sequence to be tested specifically includes: identifying the region information corresponding to the source region identifier based on the national administrative division information; and determining the dialect category of the dialect sequence based on the region information corresponding to the source region identifier and the dialect category table.

[0100] This application obtains a speech sequence to be tested and its source region identifier; inputs the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture; inputs the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture; uses a comparative calculation method to obtain the edit distance between the first text sequence and the second text sequence; based on a preset difference threshold and the edit distance, identifies whether the speech sequence to be tested is a dialect sequence; if the speech sequence to be tested is a dialect sequence, then determines the dialect category to which the dialect sequence belongs through the source region identifier of the speech sequence to be tested. Using two different ASR models for Mandarin, the same speech sequence can be identified. If it is Mandarin, both can be detected with a small edit distance. However, if it is a dialect, the two ASR models show a large difference in recognition results with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias that the recognition model must be trained with dialect data. Using Mandarin data as training data is easier to collect and more scientific and reasonable.

[0101] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0102] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0103] In this embodiment, a first text sequence is obtained by inputting the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture. A second text sequence is obtained by inputting the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture. The edit distance between the first and second text sequences is calculated. The edit distance is used to identify whether the speech sequence to be tested is a dialect sequence. By using two different ASR models for Mandarin to recognize the same speech sequence, if it is Mandarin, both can be detected with a small edit distance; if it is a dialect, the recognition results of the two ASR models differ significantly with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias of requiring dialect data to train the recognition model, making it more scientific and intelligent.

[0104] Further reference Figure 4 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of an automatic dialect data filtering and recognition device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0105] like Figure 4 As shown, the dialect data automatic screening and recognition device 400 described in this embodiment includes: a test data preparation module 401, a first text sequence acquisition module 402, a second text sequence acquisition module 403, an edit distance calculation module 404, a dialect sequence recognition module 405, and a dialect category determination module 406. Wherein:

[0106] The test data preparation module 401 is used to obtain the test speech sequence and the source region identifier of the test speech sequence;

[0107] The first text sequence acquisition module 402 is used to input the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to acquire the first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture.

[0108] The second text sequence acquisition module 403 is used to input the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to acquire the second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture mode.

[0109] The edit distance calculation module 404 is used to obtain the edit distance between the first text sequence and the second text sequence by means of comparison calculation;

[0110] The dialect sequence recognition module 405 is used to identify whether the speech sequence to be tested is a dialect sequence based on a preset difference threshold and the edit distance;

[0111] The dialect category determination module 406 is used to determine the dialect category to which the dialect sequence belongs by means of the source region identifier of the speech sequence to be tested if the speech sequence to be tested is a dialect sequence.

[0112] In some specific embodiments of this application, the dialect data automatic screening and recognition device 400 further includes a training data acquisition module, which is used to acquire pre-collected Mandarin speech recognition corpus; it is also used to perform semantic annotation on each Mandarin sequence in the Mandarin speech recognition corpus according to a preset semantic annotation method, and acquire annotated Mandarin speech recognition corpus, wherein the semantic annotation method includes semantic annotation using a trained Chinese natural language processing model; and it is also used to divide the annotated Mandarin speech recognition corpus into a training set and a test set according to a preset ratio.

[0113] In some specific embodiments of this application, the dialect data automatic screening and recognition device 400 further includes a first training module. The first training module is used to input the training set into the initialized hybrid structured ASR speech recognition model for model pre-training to obtain a pre-trained hybrid structured ASR speech recognition model; it is also used to input the test set into the pre-trained hybrid structured ASR speech recognition model for recognition verification; if the recognition verification fails, it is also used to adjust the training parameters of the initialized hybrid structured ASR speech recognition model and re-pre-train the hybrid structured ASR speech recognition model; and if the recognition verification succeeds, it is also used to obtain the pre-trained hybrid structured ASR speech recognition model.

[0114] In some specific embodiments of this application, the dialect data automatic screening and recognition device 400 further includes a second training module. The second training module is used to input the training set into the initialized end-to-end ASR speech recognition model for model pre-training to obtain a pre-trained end-to-end ASR speech recognition model; it is also used to input the test set into the pre-trained end-to-end ASR speech recognition model for recognition verification; if the recognition verification fails, it is also used to adjust the training parameters of the initialized end-to-end ASR speech recognition model and re-perform end-to-end ASR speech recognition model pre-training; and if the recognition verification succeeds, it is also used to obtain a pre-trained end-to-end ASR speech recognition model.

[0115] Continue to refer to Figure 5 , Figure 5 yes Figure 4 The schematic diagram shows a specific embodiment of module 404. The editing distance calculation module 404 includes a first acquisition submodule 501, a second acquisition submodule 502, a comparison submodule 503, a total text quantity statistics submodule 504, a percentage calculation submodule 505, and an editing distance determination submodule 506, wherein:

[0116] The first acquisition submodule 501 is used to acquire the text content and the number of characters in the first text sequence;

[0117] The second acquisition submodule 502 is used to acquire the text content and the number of characters in the second text sequence;

[0118] The comparison submodule 503 is used to obtain the different text content in the first text sequence and the second text sequence by using a comparison method;

[0119] The text total count submodule 504 is used to count the total number of characters in the first text sequence and the second text sequence;

[0120] The percentage calculation submodule 505 is used to calculate the percentage of the different text contents in the total number of texts;

[0121] The edit distance determination submodule 506 is used to set the percentage value as the edit distance between the first text sequence and the second text sequence.

[0122] This application obtains a speech sequence to be tested and its source region identifier; inputs the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture; inputs the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture; uses a comparative calculation method to obtain the edit distance between the first text sequence and the second text sequence; based on a preset difference threshold and the edit distance, identifies whether the speech sequence to be tested is a dialect sequence; if the speech sequence to be tested is a dialect sequence, then determines the dialect category to which the dialect sequence belongs through the source region identifier of the speech sequence to be tested. Using two different ASR models for Mandarin, the same speech sequence can be identified. If it is Mandarin, both can be detected with a small edit distance. However, if it is a dialect, the two ASR models show a large difference in recognition results with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias that recognition models must be trained with dialect data.

[0123] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0124] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0125] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0126] The computer device 6 includes a memory 6a, a processor 6b, and a network interface 6c that are interconnected via a system bus. It should be noted that only the computer device 6 with components 6a-6c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0127] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0128] The memory 6a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 6a may be an internal storage unit of the computer device 6, such as the hard disk or memory of the computer device 6. In other embodiments, the memory 6a may also be an external storage device of the computer device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 6. Of course, the memory 6a may include both the internal storage unit and its external storage device of the computer device 6. In this embodiment, the memory 6a is typically used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions for an automatic dialect data filtering and recognition method. In addition, the memory 6a can also be used to temporarily store various types of data that have been output or will be output.

[0129] In some embodiments, the processor 6b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 6b is typically used to control the overall operation of the computer device 6. In this embodiment, the processor 6b is used to execute computer-readable instructions stored in the memory 6a or to process data, such as executing computer-readable instructions for the automatic dialect data filtering and recognition method.

[0130] The network interface 6c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 6 and other electronic devices.

[0131] The computer device proposed in this embodiment belongs to the field of recognition process optimization technology and is used to assist telemarketers in performing prior recognition of dialect data when communicating with customers via voice. This application obtains a speech sequence to be tested and its source region identifier; inputs the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture; inputs the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture; uses a comparative calculation method to obtain the edit distance between the first text sequence and the second text sequence; based on a preset difference threshold and the edit distance, identifies whether the speech sequence to be tested is a dialect sequence; if the speech sequence to be tested is a dialect sequence, the dialect category to which the dialect sequence belongs is determined by the source region identifier of the speech sequence to be tested. Using two different ASR models for Mandarin, the same speech sequence can be identified. If it is Mandarin, both can be detected with a small edit distance. However, if it is a dialect, the two ASR models show a large difference in recognition results with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias that recognition models must be trained with dialect data.

[0132] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the dialect data automatic screening and recognition method described above.

[0133] The computer-readable storage medium proposed in this embodiment belongs to the field of recognition process optimization technology and is used to assist telemarketers in performing prior recognition of dialect data during voice communication with customers. This application obtains a speech sequence to be tested and its source region identifier; inputs the speech sequence to be tested into a pre-trained hybrid structured ASR speech recognition model to obtain a first text sequence, wherein the hybrid structured ASR speech recognition model is a DNN-HMM architecture; inputs the speech sequence to be tested into a pre-trained end-to-end ASR speech recognition model to obtain a second text sequence, wherein the end-to-end ASR speech recognition model is a Conformer+CTC architecture; uses a comparative calculation method to obtain the edit distance between the first text sequence and the second text sequence; based on a preset difference threshold and the edit distance, identifies whether the speech sequence to be tested is a dialect sequence; if the speech sequence to be tested is a dialect sequence, the dialect category to which the dialect sequence belongs is determined by the source region identifier of the speech sequence to be tested. Using two different ASR models for Mandarin, the same speech sequence can be identified. If it is Mandarin, both can be detected with a small edit distance. However, if it is a dialect, the two ASR models show a large difference in recognition results with a large edit distance. This solves the problem of the difficulty in collecting large amounts of dialect data and overcomes the technical bias that recognition models must be trained with dialect data.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0135] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for automatically filtering and identifying dialect data, characterized by, The method comprises the following steps: obtaining a to-be-tested voice sequence and a source region identifier of the to-be-tested voice sequence; inputting the to-be-tested voice sequence into a pre-trained hybrid structure type ASR voice recognition model to obtain a first text sequence, wherein the hybrid structure type ASR voice recognition model is in a DNN-HMM architecture mode; inputting the to-be-tested voice sequence into a pre-trained end-to-end type ASR voice recognition model to obtain a second text sequence, wherein the end-to-end type ASR voice recognition model is in a Conformer+CTC architecture mode; obtaining an edit distance between the first text sequence and the second text sequence by using a comparison calculation mode; identifying whether the to-be-tested voice sequence is a dialect sequence based on a preset difference threshold and the edit distance; if the to-be-tested voice sequence is a dialect sequence, determining a dialect category to which the dialect sequence belongs through the source region identifier of the to-be-tested voice sequence.

2. The method of claim 1, wherein, After the steps of obtaining the to-be-tested voice sequence and the source region identifier of the to-be-tested voice sequence, the method further comprises: obtaining a pre-collected Mandarin voice recognition corpus; performing semantic annotation on each Mandarin sequence in the Mandarin voice recognition corpus according to a preset semantic annotation mode to obtain an annotated Mandarin voice recognition corpus, wherein the semantic annotation mode comprises performing semantic annotation by using a trained Chinese natural language processing model; dividing the annotated Mandarin voice recognition corpus into a training set and a test set according to a preset proportion relationship.

3. The method of claim 2, wherein the dialect data is automatically filtered and identified by, Before the step of inputting the to-be-tested voice sequence into the pre-trained hybrid structure type ASR voice recognition model to obtain the first text sequence, the method further comprises: inputting the training set into an initialized hybrid structure type ASR voice recognition model to perform model pre-training to obtain a preliminarily trained hybrid structure type ASR voice recognition model; inputting the test set into the preliminarily trained hybrid structure type ASR voice recognition model to perform identification verification; if the identification verification fails, adjusting training parameters of the initialized hybrid structure type ASR voice recognition model to re-perform hybrid structure type ASR voice recognition model pre-training; if the identification verification succeeds, obtaining the pre-trained hybrid structure type ASR voice recognition model.

4. The method of claim 2, wherein the dialect data is automatically filtered and identified by, Before the step of inputting the to-be-tested voice sequence into the pre-trained end-to-end type ASR voice recognition model to obtain the second text sequence, the method further comprises: inputting the training set into an initialized end-to-end type ASR voice recognition model to perform model pre-training to obtain a preliminarily trained end-to-end type ASR voice recognition model; inputting the test set into the preliminarily trained end-to-end type ASR voice recognition model to perform identification verification; if the identification verification fails, adjusting training parameters of the initialized end-to-end type ASR voice recognition model to re-perform end-to-end type ASR voice recognition model pre-training; if the identification verification succeeds, obtaining the pre-trained end-to-end type ASR voice recognition model.

5. The method of claim 1, wherein, The step of obtaining the edit distance between the first text sequence and the second text sequence by comparison calculation specifically comprises: obtaining the literal content and the number of literals in the first text sequence; obtaining the literal content and the number of literals in the second text sequence; obtaining the different literal content in the first text sequence and the second text sequence by comparison; counting the total number of literals in the first text sequence and the second text sequence; calculating the proportion of the different literal content in the total number of literals; setting the proportion as the edit distance between the first text sequence and the second text sequence.

6. The method of automatic dialect data screening identification according to claim 1 or 5, characterized in that, The step of identifying whether the to-be-tested voice sequence is a dialect sequence based on the preset difference threshold and the edit distance specifically comprises: determining whether the edit distance exceeds the preset difference threshold; if not, the to-be-tested voice sequence is a Mandarin sequence; if so, the to-be-tested voice sequence is a dialect sequence.

7. The method of dialect data automatic screening identification according to claim 1, characterized in that, Before the step of determining the dialect category to which the dialect sequence belongs through the source area identification of the to-be-tested voice sequence, the method further comprises: obtaining a dialect category table divided according to area information in advance, wherein the area information comprises national administrative division information; The step of determining the dialect category to which the dialect sequence belongs through the source area identification of the to-be-tested voice sequence specifically comprises: identifying the area information corresponding to the source area identification according to the national administrative division information; determining the dialect category to which the dialect sequence belongs according to the area information corresponding to the source area identification and the dialect category table.

8. A device for automatically screening and identifying dialect data, characterized by comprising: Comprise: a to-be-tested data preparation module for obtaining a to-be-tested voice sequence and a source area identification of the to-be-tested voice sequence; a first text sequence obtaining module for inputting the to-be-tested voice sequence into a pre-trained hybrid structure type ASR voice recognition model to obtain a first text sequence, wherein the hybrid structure type ASR voice recognition model is a DNN-HMM architecture mode; a second text sequence obtaining module for inputting the to-be-tested voice sequence into a pre-trained end-to-end type ASR voice recognition model to obtain a second text sequence, wherein the end-to-end type ASR voice recognition model is a Conformer+CTC architecture mode; an edit distance calculation module for obtaining the edit distance between the first text sequence and the second text sequence by comparison calculation; a dialect sequence identification module for identifying whether the to-be-tested voice sequence is a dialect sequence based on a preset difference threshold and the edit distance; a dialect category determination module for determining the dialect category to which the dialect sequence belongs through the source area identification of the to-be-tested voice sequence if the to-be-tested voice sequence is a dialect sequence.

9. A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions, the processor implementing the steps of the dialect data automatic screening and identification method of any one of claims 1 to 7 when executing the computer readable instructions.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the dialect data automatic screening and identifying method in any one of claims 1 to 7.

Images