Accent recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an accent recognition method and device, electronic equipment and a storage medium. The method comprises the steps that to-be-detected voice is acquired; inputting to-be-detected voice into the voice recognition model, performing framing processing and feature extraction processing on the to-be-detected voice to obtain a plurality of first voice features, and processing the first voice features to determine text probability distribution; determining target characters according to the recognition probability, eliminating blank characters in all the target characters, eliminating first voice features corresponding to the blank characters, and performing alignment processing on the target characters and the to-be-detected voice according to the text occurrence time corresponding to each target character to obtain voice text alignment information; inputting the voice text alignment information and the first voice feature into an accent recognition model, and determining a target accent type of the to-be-detected voice; according to the embodiment of the invention, the complexity of an accent recognition process can be reduced, and the accuracy and robustness of an accent recognition model are improved.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of speech recognition technology, and in particular to an accent recognition method, device, electronic device and storage medium. Background Art

[0002] Speech recognition technology is a technology that uses computer algorithms to identify different accents. This method collects and analyzes a large number of speech samples and trains the model to recognize different accents. The accent recognition problem in speech recognition technology can be improved by adjusting the model and parameter settings to improve the recognition accuracy. This method requires professional technology and a large amount of data support, and is an important research direction in the field of modern speech recognition.

[0003] In the prior art, an accent recognition model usually needs to perform text recognition on the test speech to obtain the recognition text corresponding to the test speech, and query the standard accent dictionary based on the recognition text to obtain the accent classification; however, in the process of performing text recognition on the test speech, since there are usually segments of silence or environmental noise in the test speech, these segments are not helpful for the accent recognition model to recognize the accent of the test speech, and will also increase the complexity of the calculation process. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide an accent recognition method, device, electronic device and storage medium, which can reduce the redundancy of the accent recognition process and improve the accuracy and robustness of the accent recognition model.

[0006] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application proposes 1. An accent recognition method, characterized in that it includes: obtaining a speech to be tested; inputting the speech to be tested into a speech recognition model, performing frame processing and feature extraction processing on the speech to be tested to obtain multiple first speech features, processing the first speech features, and determining a text probability distribution, wherein the text probability distribution includes recognition probabilities of multiple recognized characters and text occurrence times corresponding to each of the recognized characters; determining a target character according to the recognition probability, eliminating blank characters from all the target characters, eliminating the first speech features corresponding to the blank characters, and aligning the target character with the speech to be tested according to the text occurrence times corresponding to each of the target characters to obtain speech-text alignment information; inputting the speech-text alignment information and the first speech features into the accent recognition model to determine the target accent type of the speech to be tested.

[0007] In one embodiment, the speech recognition model includes a first encoding module, a linear mapping module and an activation module, and the speech to be tested is input into the speech recognition model, the speech to be tested is framed and feature extracted to obtain a first speech feature, the first speech feature is processed, and a text probability distribution is determined, including: inputting the speech to be tested into the speech recognition model, frame-processing the speech to be tested to obtain multiple speech frames; performing feature extraction processing on each of the speech frames through the first encoding module to obtain a first speech feature; recognizing each of the first speech features through the linear mapping module to obtain a recognized character; and activating each of the recognized characters through the activation module to obtain a text probability distribution.

[0008] In one embodiment, the target characters are aligned with the speech to be tested according to the text appearance time corresponding to each of the target characters, and before the speech-text alignment information is obtained, it also includes: when the text appearance times corresponding to the target characters are continuous and the target characters are the same, the highest probability character is determined according to the recognition probabilities corresponding to each of the target characters; other target characters that are the same and continuous with the highest probability character are eliminated, and the first speech features corresponding to the eliminated target characters are eliminated.

[0009] In one embodiment, the step of inputting the speech-text alignment information and the first speech feature into an accent recognition model to determine a target accent type of the speech to be tested includes: calling an accent recognition model to transform the first speech feature to obtain a target accent feature; and determining the target accent type of the speech to be tested based on the target accent feature.

[0010] In one embodiment, the accent recognition model includes a second encoding module, a quantization module and an attention module. The calling of the accent recognition model to transform the first speech feature to obtain a target accent feature includes: performing convolution processing on the first speech feature again through the second encoding module to obtain a second speech feature; performing discrete processing on the second speech feature through the quantization module to obtain a plurality of third speech features; adjusting each of the third speech features through the attention module to obtain a plurality of fourth speech features; and determining the target accent feature based on the fourth speech features.

[0011] In one embodiment, determining the target accent feature based on the fourth speech feature includes: determining an initial accent feature based on the fourth speech feature; determining a mean accent feature and a variance accent feature based on the initial accent feature, and determining the target accent feature based on the mean accent feature and the variance accent feature.

[0012] In one embodiment, determining the target accent type of the speech to be tested based on the target accent features includes: acquiring standard accent features of the speech-text alignment information; aligning the target accent features with the speech-text alignment information, and determining the target accent type of the speech to be tested based on the aligned standard accent features and the target accent features.

[0013] To achieve the above-mentioned purpose, the second aspect of the present application proposes an accent recognition device, comprising: an acquisition module, used to acquire a speech to be tested; a speech recognition module, used to input the speech to be tested into a speech recognition model, perform frame processing and feature extraction processing on the speech to be tested to obtain multiple first speech features, process the first speech features, and determine a text probability distribution, wherein the text probability distribution includes recognition probabilities of multiple recognized characters and text occurrence times corresponding to each of the recognized characters; a processing module, used to determine a target character according to the recognition probability, remove blank characters from all the target characters, remove the first speech features corresponding to the blank characters, align the target character with the speech to be tested according to the text occurrence times corresponding to each of the target characters, and obtain speech-text alignment information; an accent recognition module, used to input the speech-text alignment information and the first speech features into the accent recognition model to determine the target accent type of the speech to be tested.

[0014] To achieve the above-mentioned purpose, the third aspect of the present application further proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the accent recognition method described in any one of claims 1 to 7.

[0015] To achieve the above-mentioned purpose, the fourth aspect of the present application further proposes a storage medium, which stores a computer program, characterized in that when the computer program is executed by a processor, the accent recognition method described in any one of claims 1 to 7 is implemented.

[0016] The embodiments of the present application include at least the following beneficial effects: by inputting the speech to be tested into the speech recognition model, performing frame processing and feature extraction processing on the speech to be tested, obtaining multiple first speech features, and then recognizing the first speech features through the speech recognition model to obtain speech-text alignment information. In this process, since character recognition needs to be performed based on each first speech feature, there are usually speech frames with continuous silence or full of environmental noise in the speech to be tested, and these speech frames often do not include human voice. Therefore, these speech frames will be recognized as blank characters, and there is no feature data conducive to accent recognition in the blank characters and the first speech features corresponding to the blank characters. Therefore, by removing the blank characters from the speech-text alignment information and removing the first speech features corresponding to the blank characters, the first speech features without redundant speech data can be obtained, and each first speech feature includes human voice, that is, the redundancy of the accent recognition model processing process is reduced, so that the accent recognition model can focus on the human voice data in the speech to be tested for accent recognition, thereby reducing the calculation amount of the accent recognition model, reducing the interference of redundant sounds such as environmental noise on accent recognition, and improving the accuracy and robustness of the accent recognition model.

[0017] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0019] Figure 1 A schematic diagram of an optional flow chart of an accent recognition method provided in an embodiment of the present application;

[0020] Figure 2 An optional flowchart of text recognition provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an optional specific process for removing identical target characters provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of another optional flow chart of the accent recognition method provided in the embodiment of the present application;

[0023] Figure 5 A schematic diagram of another optional flow chart of the accent recognition method provided in the embodiment of the present application;

[0024] Figure 6 A schematic diagram of an optional process for determining target accent features provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of an optional process for determining a target accent type provided in an embodiment of the present application;

[0026] Figure 8 An optional schematic diagram of selecting a target character provided in an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of an optional architecture of an accent recognition model provided in an embodiment of the present application

[0029] Fig.10 A schematic diagram of an optional structure of the accent recognition device provided in this embodiment;

[0030] Fig.11 A schematic diagram of an optional hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In the description of this application, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood to exclude the number, and “above”, “below”, “within”, etc. are understood to include the number.

[0033] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0034] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain the target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page, etc., and after the separate permission or separate consent of the target object is clearly obtained, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0035] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0036] Filter Bank (FBank) is a commonly used technique in audio signal processing. Filter Bank Analysis simulates the nonlinear response of the human ear to the sound spectrum and improves the performance of speech recognition by designing a front-end processing algorithm. The main purpose of filter bank analysis is to process speech signals in a way similar to the human ear, and to transform the speech signal in the frequency domain through a series of Mel-scale-based filters, so as to better simulate the processing of the auditory process.

[0037] Mel-frequency Cepstral Coefficients (MFCC) are a feature representation method widely used in the field of speech signal processing. Mel-frequency Cepstral Coefficients provide an effective representation of speech signal features by simulating the nonlinear perception of the human ear to sound frequency. Mel-frequency Cepstral Coefficients can capture the key features of speech signals and reduce the dimensionality of data. Therefore, they are widely used in speech processing tasks such as automatic speech recognition, speaker recognition, and emotion recognition. The design of Mel-frequency Cepstral Coefficients takes into account the perceptual characteristics of the human auditory system, so that they can better reflect the important features of speech signals, and the calculation is relatively efficient, which can be quickly implemented through modern signal processing libraries.

[0038] One-hot encoding is a commonly used data preprocessing method, which is mainly used to convert categorical features into numerical features so that machine learning models can process them. In one-hot encoding, for a feature with N different values, N binary variables, also called "bits", are created. Each binary variable represents a potential value of the feature, where only the binary variable corresponding to the original feature value is set to 1, and the remaining N-1 variables are set to 0.

[0039] A codebook is a set of discrete vectors used to represent possible output states or categories. A codebook consists of a finite number of codewords. Each codeword is a vector that represents a specific point in the latent space. The codebook provides a method for mapping a continuous latent space to a discrete representation.

[0040] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction system, mechatronics, etc. Among them, the pre-trained model is also called the large model or basic model. After fine-tuning, it can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0041] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0042] In the related art, the accent recognition model usually needs to perform text recognition on the test speech to obtain the recognition text corresponding to the test speech, and query the standard accent dictionary based on the recognition text to obtain the accent classification; however, in the process of text recognition of the test speech, since there are usually fragments of silence or environmental noise in the test speech, these fragments are not helpful for the accent recognition model to recognize the accent of the test speech, and will also increase the complexity of the calculation process.

[0043] Based on this, the embodiments of the present application provide an accent recognition method, device, electronic device and storage medium, which can reduce the redundancy of the accent recognition process and improve the accuracy and robustness of the accent recognition model.

[0044] The accent recognition method, device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the accent recognition method in the embodiments of the present application is described.

[0045] The accent recognition method provided in the embodiment of the present application relates to the field of computer technology. The accent recognition method provided in the embodiment of the present application can be applied in a terminal, can be applied in a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the accent recognition method, etc., but is not limited to the above forms.

[0046] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0047] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0048] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, Figure 1 An optional flow chart of an accent recognition method provided in an embodiment of the present application is provided. The accent recognition method may be executed by a server, or may be executed by a terminal, or may be executed by a server in cooperation with a terminal. The accent recognition method includes but is not limited to the following steps S110 to S140:

[0050] Step S110, obtaining the voice to be tested;

[0051] Step S120, inputting the speech to be tested into the speech recognition model, performing frame processing and feature extraction processing on the speech to be tested to obtain a plurality of first speech features, processing the first speech features, and determining a text probability distribution, wherein the text probability distribution includes recognition probabilities of a plurality of recognized characters and a text appearance time corresponding to each recognized character;

[0052] Step S130, determining the target characters according to the recognition probability, removing the blank characters from all the target characters, removing the first speech features corresponding to the blank characters, and aligning the target characters with the speech to be tested according to the text appearance time corresponding to each target character to obtain speech-text alignment information;

[0053] Step S140: input the speech-text alignment information and the first speech feature into an accent recognition model to determine a target accent type of the speech to be tested.

[0054] Based on this, by inputting the test speech into the speech recognition model, the test speech is subjected to frame processing and feature extraction processing to obtain multiple first speech features, and then the first speech features are recognized by the speech recognition model to obtain speech-text alignment information. In this process, since character recognition needs to be performed based on each first speech feature, there are usually speech frames with continuous silence or full of environmental noise in the test speech, and these speech frames often do not include human voices. Therefore, these speech frames will be recognized as blank characters, and there is no feature data conducive to accent recognition in the blank characters and the first speech features corresponding to the blank characters. Therefore, by removing the blank characters from the speech-text alignment information and removing the first speech features corresponding to the blank characters, the first speech features without redundant speech data can be obtained, and each first speech feature includes human voices, that is, the redundancy of the accent recognition model processing process is reduced, so that the accent recognition model can focus on the human voice data in the test speech for accent recognition, thereby reducing the amount of calculation of the accent recognition model, reducing the interference of redundant sounds such as environmental noise on accent recognition, and improving the accuracy and robustness of the accent recognition model.

[0055] In a specific implementation, assuming that the speech to be tested is recognized, the speech-text alignment information obtained is "h", "-", "e", "l", "-", "l", "o", among which, "h", "-", "e", "l", and "o" are all target characters, and "-" represents a blank character. In the speech-text alignment information, the "-" character is eliminated, and the target recognized characters obtained are "h", "e", "l", "l", and "o", and the first speech feature corresponding to the "-" character is eliminated.

[0056] In a specific implementation, taking one of the speech frames of the speech to be tested as an example, after extracting features from the speech frame to obtain the first speech feature, a sound-character comparison table is obtained, and the probability distribution of the recognized characters in the sound-character comparison table can be determined according to one or more first speech features. According to the recognition probability, the maximum value of the recognition probability is taken to determine the target character. For example, for the first speech feature, the probability distribution of each character is obtained in the pre-acquired sound-character comparison table. Among the recognized characters, it is assumed that the recognized characters with higher probabilities include "c", "d" and " e" and the blank character "-", the recognition probabilities of "c", "d", "e" and "-" are approximately 0.1, 0.3, 0.5 and 0.1, respectively. Therefore, the maximum value of 0.5 in the recognition probability is taken to determine that the target character is "e"; in other examples, if the first speech feature in the current recognition is the coda feature, it may be judged as a blank character. For example, the recognized characters may also be "a", "b", "c", "d" and "-", and the corresponding recognition probabilities are 0.1, 0.1, 0.1, 0.1 and 0.6, respectively, then the target character is a blank character.

[0057] In a specific embodiment, referring to Figure 8 As shown, after feature extraction of the speech frame is performed to obtain the first speech feature, the first recognized character obtained and the independent recognition probability of each recognized character, the recognized characters obtained later and the conditional recognition probability of the recognized characters based on the previous recognized characters, in the previous time step, according to the preset selection width M, select M character combinations with the largest probability in the previous time step, on the basis of the previous time step, according to the recognized characters and recognition probabilities corresponding to the current time step, calculate the product of the recognition probabilities in the traversed time steps, and select M character combinations with the largest probability in the current time step to enter the next time step, and so on, determine the speech-text alignment information of the speech to be tested; exemplarily, the preset selection width is 2, in the first time step, a part of the first speech feature is analyzed, and the recognized characters obtained include "a", "b", "c" and "-", with recognition probabilities of P(a)=0.5, P(b)=0.3, P(c)=0.1 and P(-)=0.1 respectively, select the "a" character and the "b" character, in the second time step, analyze the second part of the first speech feature, and obtain the recognized characters Including "a", "b", "c" and "-", based on the selection of the first time step, the conditional recognition probabilities of the recognized characters at the second time step are P(a|a)=0.5, P(b|a)=0.3, P(c|a)=0.1, P(-|a)=0.1, P(a|b)=0.1, P(b|b)=0.2, P(c|b)=0.6, b(-|b)=0.1, where P(b|a) means that when the target character of the first part of the speech feature is "a", the target character corresponding to the first speech feature of the second part The probability of "b" is calculated, and the product of the conditional recognition probability and the corresponding recognition probability is calculated one by one. According to the selection width, the two combinations with the largest product are selected, specifically P(a,a)=P(a)×P(a|a)=0.25, P(b,c)=P(b)×P(c|b)=0.18; at the next time step, based on the selected (a,a) and (b,c) combinations, the conditional recognition probability of recognizing the character on the first speech feature of the third part is calculated, and so on. The combination with the largest recognition probability product is used as the speech-text alignment information of the speech to be tested.

[0058] In addition, the speech recognition model includes a first encoding module, a linear mapping module and an activation module, referring to Figure 2 As shown, one embodiment of the present application, Figure 1 Step S120 in the embodiment includes but is not limited to the following steps:

[0059] Step S210, inputting the speech to be tested into the speech recognition model, performing frame processing on the speech to be tested, and obtaining a plurality of speech frames;

[0060] Step S220, performing feature extraction processing on each speech frame through a first encoding module to obtain a first speech feature;

[0061] Step S230, recognizing each first voice feature through a linear mapping module to obtain a recognized character;

[0062] Step S240: Activate each recognized character through an activation module to obtain a text probability distribution.

[0063] In a specific implementation, before extracting features from a speech frame, the speech segment needs to be Fourier processed to convert the speech segment into a spectrum. The spectrum may be in the form of FBank coefficients or MFCC coefficients of the speech frame, which will not be limited here.

[0064] In a specific implementation, the speech recognition model is trained as follows:

[0065] First, the training speech is input into the speech recognition model, and the training speech is framed to obtain multiple speech frames, wherein the training speech has real text labels;

[0066] Then, each speech frame is subjected to feature extraction processing by a first encoding module to obtain training speech features;

[0067] Then, each training speech feature is recognized through a linear mapping module to obtain training characters;

[0068] Then, each training character is activated through the activation module to obtain the training probability distribution;

[0069] Then, the training loss is constructed by training the probability distribution and the true text label, and the parameters of the speech recognition model are adjusted based on the training loss.

[0070] In addition, refer to Figure 3 As shown, one embodiment of the present application, Figure 1 Before step S130 in the above method, the following steps are also included but not limited to:

[0071] Step S310, when the texts corresponding to the target characters appear continuously and the target characters are the same, the character with the highest probability is determined according to the recognition probabilities corresponding to the target characters;

[0072] Step S320, remove other target characters that are identical to and continuous with the character with the highest probability, and remove the first speech features corresponding to the removed target characters.

[0073] In a specific implementation, when the text corresponding to the target character appears continuously and the target characters are the same, the recognition probability of the target character is obtained, the maximum value of the above recognition probabilities is taken as the maximum recognition probability, and the target character corresponding to the maximum recognition probability is taken as the maximum probability character; exemplarily, in the steps of the previous embodiment, assuming that among three consecutive target characters, the first recognized character includes "a", "b", "c" and "-", and the corresponding recognition probabilities are 0.6, 0.3, 0.05 and 0.05, then the first target character is "a", and the recognition probability is 0.6, the second recognized character includes "a", "b", "c" and "-", and the corresponding recognition probabilities are 0.8, 0.1, 0.05 and 0.05, then the first target character is "a", and the recognition probability is 0.8, the third recognized character includes "a", "b", "c" and "-", and the corresponding recognition probabilities are 0.5, 0.3, 0.1 and 0.1, then the third target character is "a", and the recognition probability is 0.5, therefore, the second target character is determined to be the maximum probability character.

[0074] In a specific embodiment, when the target character is determined and the speech-text alignment information is combined by using the conditional recognition probability between each target character in different time steps through progressive time steps, the recognition probability of selecting the target character at the current time step is determined by the conditional recognition probability of selecting the target character at the current time step under the premise that the previous target character was selected at the previous time step, and the conditional recognition probability of selecting the target character at the current time step under the premise that the previous target character was not selected at the previous time step.

[0075] It can be understood that different recognized characters are relatively independent, and the sum of all recognition probabilities is equal to 1. Therefore, at the current time step, the conditional recognition probability of selecting the target character at the current time step under the premise that the target character of the previous speech segment was selected at the previous time step, and the conditional recognition probability of selecting the target character at the current time step under the premise that the previous target character was selected at the previous time step, are equal to the recognition probability of selecting the target character at the current time step.

[0076] Exemplarily, at the previous time step, the recognized characters include "a", "b", "c" and "-", the target character is "a", and the corresponding recognition probability is 0.4. At the current time step, the target character includes "a". At the current time step, the conditional recognition probabilities are P(a|a)=0.3, P(a|b)=0.1, P(a|c)=0.05, and P(a|-)=0.05, respectively. Then, at the current time step, the recognition probability of the target character "a" is P(a|a)+P(a|b)+P(a|c)+P(a|-)=0.6. Therefore, the target character at the current time step is the character with the highest probability.

[0077] In some specific embodiments, among multiple consecutive first speech features with the same corresponding target characters, when the recognition probabilities of at least two target characters are the same, the target characters with the same recognition probabilities and their corresponding first speech features are retained, and the remaining consecutive and identical target characters and their corresponding first speech features are eliminated; exemplarily, for the consecutive first part features, second part features, third part features and fourth part features, the target character is "a", and the recognition probabilities are 0.4, 0.6, 0.5 and 0.6 respectively, so the first part features and the second part features and their target characters are eliminated, and the second part features and the fourth part features and their target characters are retained, and the first speech features and the target characters are recombined and aligned based on the text appearance time.

[0078] In addition, refer to Figure 4 As shown, one embodiment of the present application, Figure 1 Step S140 in the embodiment further includes but is not limited to the following steps:

[0079] Step S410, calling an accent recognition model to transform the first speech feature to obtain a target accent feature;

[0080] Step S420: determining the target accent type of the speech to be tested according to the target accent features.

[0081] In a specific embodiment, the accent recognition model is trained using standard accent segments, which may be one or more combinations of Northeastern accent, Sichuan accent, Tianjin accent, Henan accent, Taiwanese accent or Cantonese accent. By selecting standard accent segments of different accent types to train the accent recognition model, the accent recognition model obtains a first speech feature capable of identifying a specific type of accent.

[0082] In another specific implementation, the accent recognition model transforms the first speech feature again to obtain the target accent feature of the new speech-text alignment information, and obtains the target accent type of the speech to be tested by comparing and analyzing the standard accent feature and the first speech feature.

[0083] It can be understood that the recognition methods of the above two accent recognition models belong to the prior art and can be applied to the accent recognition model in this application without restriction and are not limited here.

[0084] In a specific implementation, the accent recognition model is specifically a wav2vec2.0 model, and the accent recognition model is trained by constructing a cross entropy loss function.

[0085] In addition, refer to Figure 5 and Fig. 9As shown, in one embodiment of the present application, the accent recognition model includes a second encoding module, a quantization module and an attention module. Figure 4 Step S410 in the embodiment includes but is not limited to the following steps:

[0086] Step S510, performing convolution processing on the first speech feature again through a second encoding module to obtain a second speech feature;

[0087] Step S520, performing discrete processing on the second speech feature through a quantization module to obtain a plurality of third speech features;

[0088] Step S530, adjusting each third speech feature through an attention module to obtain a plurality of fourth speech features;

[0089] Step S540: determining a target accent feature according to the fourth speech feature.

[0090] In a specific embodiment, the attention module selects one of the third speech features, assigns attention weights to all the third speech features based on the correlation between the selected third speech feature and other third speech features, sums the third speech features based on the attention weights, and thus determines the fourth speech feature.

[0091] It can be understood that the closer the third speech feature is on the time axis, the greater the allocated attention weight is. For example, for the continuous first part features, second part features, third part features and fourth part features, the third part features are selected for attention weight allocation, and the attention weights are the first weight, the second weight, the third weight and the fourth weight respectively. The relationship between the attention weights is: the first weight> the second weight> the third weight> the fourth weight. The first part features, the second part features, the third part features and the fourth part features are weighted, and the target part feature corresponding to the third part feature is equal to the first part feature×the first weight+the second part feature×the second weight+the third part feature×the third weight+the fourth part feature×the fourth weight.

[0092] In a specific implementation, the accent recognition model masks some elements on the fourth speech feature to obtain masked features, the accent recognition model predicts the masked features to obtain predicted features, a cross entropy loss function is constructed based on the predicted features and the fourth speech feature, and the accent recognition model is trained based on the loss function.

[0093] In a specific implementation, after the second speech feature is discretized by the quantization module, the third speech feature obtained is of equal length.

[0094] In a specific implementation, the accent recognition model further includes an accent classification module, and the accent classification module is used to execute step S540. The detailed process of the accent classification module will be described in detail in the following embodiments.

[0095] In addition, refer to Figure 6 As shown, one embodiment of the present application, Figure 5 Step S540 in the embodiment includes but is not limited to the following steps:

[0096] Step S610, determining an initial accent feature based on the fourth speech feature;

[0097] Step S620: determining a mean accent feature and a variance accent feature based on the initial accent feature, and determining a target accent feature according to the mean accent feature and the variance accent feature.

[0098] In a specific implementation, the accent classification module analyzes the fourth speech feature and extracts an initial accent feature from the fourth speech feature, wherein each initial accent feature represents the accent of a corresponding portion of the speech frame when reciting the target character; the mean and variance of all the initial accent features are calculated to obtain a mean accent feature and a variance accent feature, wherein the elements in the mean accent feature are equal to the mean of the elements at the corresponding positions of all the initial accent features, and the elements in the variance accent feature are equal to the variance of the elements at the corresponding positions of all the initial accent features; the mean accent feature and the variance accent feature are summed, differentiated, or weightedly summed to obtain the target accent feature.

[0099] It can be understood that by analyzing the fourth voice feature and extracting the initial accent feature, the computational difficulty of the accent classification module for accent classification can be reduced, and redundant and irrelevant feature data can be reduced, thereby improving the accuracy of accent classification; by calculating the mean accent feature and variance accent feature of the initial accent feature, the mean accent feature can identify the approximate accent direction of the target, and the variance accent feature can determine the differences in accent expression in different voice segments; by determining the target accent feature based on the mean accent feature and the variance accent feature, the accuracy of the accent classification module can be ensured, thereby improving the accuracy of accent classification.

[0100] In addition, refer to Figure 7 As shown, one embodiment of the present application, Figure 4 Step S420 in the embodiment includes but is not limited to the following steps:

[0101] Step S710, obtaining standard accent features of speech-text alignment information;

[0102] Step S720: align the target accent feature with the speech-text alignment information, and determine the target accent type of the speech to be tested based on the aligned standard accent feature and the target accent feature.

[0103] In a specific implementation, the accent classification module is trained using an accent segment dataset, where the accent segment dataset includes standard accent segments and non-standard accent segments.

[0104] In a specific implementation, the standard accent features are obtained by extracting features from a standard accent segment of a certain type of accent through a coding module, a quantization module, and an attention module. The standard accent segment may be one or more combinations of the Northeastern accent, Sichuan accent, Tianjin accent, Henan accent, Taiwanese accent, or Cantonese accent. By selecting standard accent segments of different accent types to compare and analyze the target accent features, the target accent type for identifying the target speech can be obtained.

[0105] In addition, refer to Fig.10 The present application also provides an accent recognition device 1100, comprising:

[0106] An acquisition module 1001 is used to acquire the speech to be tested;

[0107] The speech recognition module 1002 is used to input the speech to be tested into the speech recognition model, perform frame processing and feature extraction processing on the speech to be tested, obtain multiple first speech features, process the first speech features, and determine the text probability distribution, wherein the text probability distribution includes the recognition probabilities of multiple recognized characters and the text appearance time corresponding to each recognized character;

[0108] The processing module 1003 is used to determine the target characters according to the recognition probability, remove the blank characters from all the target characters, remove the first speech features corresponding to the blank characters, and align the target characters with the speech to be tested according to the text appearance time corresponding to each target character to obtain speech-text alignment information;

[0109] The accent recognition module 1004 is used to input the speech-text alignment information and the first speech feature into the accent recognition model to determine the target accent type of the speech to be tested.

[0110] It can be understood that the specific implementation of the accent recognition device 1000 is basically the same as the specific implementation of the above-mentioned accent recognition method, and will not be described in detail here.

[0111] In addition, refer to Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0112] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0113] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the accent recognition method of the embodiment of the present application, for example, executing the above-described Figure 1 Steps S110 to S140 of the method, Figure 2 Steps S210 to S240 of the method, Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S420 of the method, Figure 5 Steps S510 to S540 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Steps S710 to S720 of the method;

[0114] Input / output interface 1103, used to implement information input and output;

[0115] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0116] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0117] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0118] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned accent recognition method, for example, to execute the above-described Figure 1 Steps S110 to S140 of the method, Figure 2 Steps S210 to S240 of the method, Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S420 of the method, Figure 5 Steps S510 to S540 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Method steps S710 to S720.

[0119] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0121] It can be understood by those skilled in the art that Figures 1 to 7 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0122] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0124] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0125] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0130] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for accent recognition, characterized in that: include: Get the voice to be tested; Inputting the speech to be tested into a speech recognition model, performing frame processing and feature extraction processing on the speech to be tested to obtain a plurality of first speech features, processing the first speech features, and determining a text probability distribution, wherein the text probability distribution includes recognition probabilities of a plurality of recognized characters and a text appearance time corresponding to each of the recognized characters; Determine the target character according to the recognition probability, remove blank characters from all the target characters, remove the first speech features corresponding to the blank characters, and align the target characters with the speech to be tested according to the text appearance time corresponding to each of the target characters to obtain speech-text alignment information; The speech-text alignment information and the first speech feature are input into an accent recognition model to determine a target accent type of the speech to be tested.

2. The accent recognition method according to claim 1, characterized in that: The speech recognition model includes a first encoding module, a linear mapping module and an activation module. The method includes inputting the speech to be tested into the speech recognition model, performing frame processing and feature extraction processing on the speech to be tested to obtain a first speech feature, processing the first speech feature, and determining a text probability distribution, including: Inputting the speech to be tested into a speech recognition model, performing frame processing on the speech to be tested to obtain a plurality of speech frames; Performing feature extraction processing on each of the speech frames through the first encoding module to obtain a first speech feature; Recognize each of the first speech features through the linear mapping module to obtain a recognized character; The activation module performs activation processing on each of the recognized characters to obtain a text probability distribution.

3. The accent recognition method according to claim 1, characterized in that: Before aligning the target characters with the speech to be tested according to the text appearance time corresponding to each target character to obtain speech-text alignment information, the method further includes: When the texts corresponding to the target characters appear continuously and the target characters are the same, determining the character with the highest probability according to the recognition probabilities corresponding to the target characters; Other target characters that are identical to and continuous with the character with the highest probability are eliminated, and the first speech features corresponding to the eliminated target characters are eliminated.

4. The accent recognition method according to claim 1, characterized in that: The step of inputting the speech-text alignment information and the first speech feature into an accent recognition model to determine a target accent type of the speech to be tested includes: Calling an accent recognition model to transform the first speech feature to obtain a target accent feature; The target accent type of the speech to be tested is determined according to the target accent feature.

5. The accent recognition method according to claim 4, characterized in that: The accent recognition model includes a second encoding module, a quantization module and an attention module. The calling of the accent recognition model to transform the first speech feature to obtain a target accent feature includes: Performing convolution processing on the first speech feature again by the second encoding module to obtain a second speech feature; Performing discrete processing on the second speech feature by the quantization module to obtain a plurality of third speech features; Adjusting each of the third speech features through the attention module to obtain a plurality of fourth speech features; A target accent feature is determined according to the fourth speech feature.

6. The accent recognition method according to claim 5, characterized in that: The determining the target accent feature according to the fourth speech feature comprises: determining an initial accent feature based on the fourth speech feature; A mean accent feature and a variance accent feature are determined based on the initial accent feature, and a target accent feature is determined according to the mean accent feature and the variance accent feature.

7. The accent recognition method according to claim 4, characterized in that: The step of determining the target accent type of the speech to be tested according to the target accent feature includes: Obtaining standard accent features of the speech-text alignment information; The target accent feature is aligned with the speech-to-text alignment information, and the target accent type of the speech to be tested is determined based on the aligned standard accent feature and the target accent feature.

8. An accent recognition device, characterized in that: include: An acquisition module is used to acquire the voice to be tested; A speech recognition module, used for inputting the speech to be tested into a speech recognition model, performing frame processing and feature extraction processing on the speech to be tested to obtain a plurality of first speech features, processing the first speech features, and determining a text probability distribution, wherein the text probability distribution includes recognition probabilities of a plurality of recognized characters and a text appearance time corresponding to each of the recognized characters; A processing module, used to determine the target characters according to the recognition probability, remove blank characters from all the target characters, remove the first speech features corresponding to the blank characters, and align the target characters with the speech to be tested according to the text appearance time corresponding to each of the target characters to obtain speech-text alignment information; The accent recognition module is used to input the speech-text alignment information and the first speech feature into an accent recognition model to determine a target accent type of the speech to be tested.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the accent recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the accent recognition method according to any one of claims 1 to 7 is implemented.

Images