Voice error detection method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a speech error detection method and device, electronic equipment and storage medium, and relates to the technical field of speech processing. The method comprises obtaining a to-be-detected speech of a target title and a plurality of pronunciation data of the target title, the plurality of pronunciation data comprising target pronunciation data corresponding to the to-be-detected speech; inputting the to-be-detected speech into a pre-trained speech error detection model, and combining the plurality of pronunciation data to obtain a speech error detection result of the to-be-detected speech relative to the target pronunciation data. The method provided in the present disclosure can effectively handle the multi-pronunciation problem in speech error detection, improve the speech error detection effect in the multi-pronunciation scene, and avoid the problem of inaccurate results caused by relying on a single pronunciation mode for speech error detection of a multi-pronunciation title.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of speech processing, and particularly relates to a speech error detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Speech error detection refers to detecting the speech uttered by a user, so as to determine whether the speech uttered by the user is standard or has errors. In a speech error detection task, error detection for multi-pronunciation words (for example, there are multiple acceptable pronunciations for the same word) is often encountered. In the related art, for error detection for multi-pronunciation words in a speech error detection task, a fixed single pronunciation manner is still used as the error detection standard, and it is impossible to perform targeted error detection according to the pronunciation manner corresponding to the speech uttered by the user. SUMMARY

[0003] Therefore, the present disclosure provides a speech error detection method and device, an electronic device, and a storage medium, which can perform targeted error detection on the speech uttered by the user according to the pronunciation manner corresponding to the speech uttered by the user.

[0004] In a first aspect, a speech error detection method is provided, including: obtaining a to-be-detected speech of a target corpus and multiple pronunciation data of the target corpus, the multiple pronunciation data including target pronunciation data corresponding to the to-be-detected speech; inputting the to-be-detected speech into a pre-trained speech error detection model in combination with the multiple pronunciation data to obtain a speech error detection result of the to-be-detected speech relative to the target pronunciation data.

[0005] In some embodiments, inputting the to-be-detected speech into the pre-trained speech error detection model in combination with the multiple pronunciation data to obtain the speech error detection result of the to-be-detected speech relative to the target pronunciation data includes: performing an encoding operation on the multiple pronunciation data respectively to obtain multiple pronunciation feature vectors; inputting the to-be-detected speech into the speech error detection model in combination with the multiple pronunciation feature vectors to obtain the speech error detection result.

[0006] In some embodiments, inputting the multiple pronunciation feature vectors and the to-be-detected speech into the speech error detection model to obtain the speech error detection result includes: fusing the multiple pronunciation feature vectors to obtain a fused feature vector, the fused feature vector being used to express each pronunciation data in the multiple pronunciation data simultaneously; inputting the fused feature vector and the to-be-detected speech into the speech error detection model to obtain the speech error detection result.

[0007] In some embodiments, fusing the multiple pronunciation feature vectors to obtain the fused feature vector includes: taking a mean value of the multiple pronunciation feature vectors as the fused feature vector; and / or inputting the multiple pronunciation feature vectors into a pre-trained vector fusion model to obtain the fused feature vector.

[0008] In some embodiments, the plurality of pronunciation data includes a plurality of pronunciation of the title of the book, each of which is represented by a phoneme.

[0009] In some embodiments, the speech error detection model includes a pronunciation prediction model and a pronunciation error detection model connected in sequence; the pronunciation prediction model is configured to determine, from the plurality of pronunciation data, target pronunciation data corresponding to the speech to be detected; and the pronunciation error detection model is configured to detect errors in the speech to be detected according to the target pronunciation data.

[0010] In some embodiments, the training process of the speech error detection model includes: predicting, based on the pronunciation prediction model, a target pronunciation sample corresponding to the speech sample from the plurality of pronunciation samples, obtaining a first loss between the real pronunciation sample corresponding to the speech sample and the target pronunciation sample; inputting the speech sample and the real pronunciation sample into the pronunciation error detection model to obtain a second loss between the error detection label of the speech sample and the error detection result output by the pronunciation error detection model; and adjusting parameters of the speech error detection model based on the first loss and the second loss to complete the training of the speech error detection model.

[0011] In some embodiments, the speech error detection method further includes: if the real pronunciation sample does not exist in the plurality of pronunciation samples, any pronunciation sample in the plurality of pronunciation samples is used as the real pronunciation sample.

[0012] In a second aspect, a speech error detection apparatus is provided, including: an acquisition module configured to acquire speech to be detected of a title and a plurality of pronunciation data of the title, the plurality of pronunciation data including target pronunciation data corresponding to the speech to be detected; and an error detection module configured to input the speech to be detected into a pre-trained speech error detection model, and obtain a speech error detection result of the speech to be detected relative to the target pronunciation data in combination with the plurality of pronunciation data.

[0013] In a third aspect, an electronic device is provided, including: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the method of the first aspect by executing the executable instructions.

[0014] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0015] The voice error detection method provided in the embodiments of the present disclosure can obtain the to-be-detected voice of the target text and the multiple pronunciation data of the target text, input the to-be-detected voice into the pre-trained voice error detection model, and combine the multiple pronunciation data, so as to obtain the voice error detection result of the to-be-detected voice relative to the target pronunciation data. The method provided in the embodiments of the present disclosure can effectively process the multi-pronunciation problem in voice error detection, improve the voice error detection effect in the multi-pronunciation scene, and avoid the problem that the result is inaccurate when the multi-pronunciation text is still subjected to voice error detection in the single-pronunciation mode. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A system architecture schematic diagram of a voice error detection method in the embodiments of the present disclosure is shown.

[0017] Figure 2 A flowchart of a voice error detection method in the embodiments of the present disclosure is shown.

[0018] Figure 3 A pronunciation feature vector fusion schematic diagram in the embodiments of the present disclosure is shown.

[0019] Figure 4 A flowchart of a voice error detection model training method in the embodiments of the present disclosure is shown.

[0020] Figure 5 A voice error detection model training process schematic diagram in the embodiments of the present disclosure is shown.

[0021] Figure 6 A structure schematic diagram of a voice error detection device in the embodiments of the present disclosure is shown.

[0022] Figure 7 A structure schematic diagram of an electronic device in the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. Features described in the description, examples, or claims can be combined in any suitable manner in one or more implementations.

[0024] In addition, the accompanying drawings are only schematic and are non-limiting illustrative of the disclosure. Identical components have been given the same reference numerals in the various drawings and have not been repeatedly described. Some of the blocks in the drawings are functional blocks that do not necessarily correspond to a physical or logical entity. These functional blocks can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] With the development of computer technology, speech processing and analysis using computer technology are widely used. In the speech error detection task, there are often multiple pronunciation problems, not only including the case that a word itself has multiple pronunciations, but also including the case that in some specific application scenarios, such as word recognition tasks, since the focus is only on the examination of the number of characters, some non-standard pronunciation or accent problems are acceptable.

[0026] The speech error detection method in the related art usually adopts an end-to-end phoneme-level error detection model, but such a method usually uses a fixed standard pronunciation phoneme sequence as a question paper, and cannot perform targeted error detection according to the pronunciation manner corresponding to the speech issued by a user, and lacks attention to and solutions for multiple pronunciation problems.

[0027] Therefore, the scheme provided by the disclosure can obtain a target question paper, multiple pronunciation data of the target question paper, input the speech to be detected for error into a pre-trained speech error detection model, and combine the multiple pronunciation data, so as to obtain a targeted speech error detection result of the speech to be detected for error relative to the target pronunciation data. The multiple pronunciation data include target pronunciation data corresponding to the speech to be detected for error.

[0028] The disclosure provides a speech error detection method, device, electronic equipment and storage medium. The speech error detection device can be integrated in an electronic equipment, which can be a terminal or a server.

[0029] It can be understood that the speech error detection method of the embodiment can be executed on a terminal, or can be executed on a server, or can be executed by the terminal and the server together. The above examples should not be understood as limiting the disclosure.

[0030] Exemplarily, Figure 1 An exemplary system architecture schematic diagram of the speech error detection method or the speech error detection device that can be applied to the embodiment of the disclosure is shown.

[0031] As Figure 1As shown, the system architecture 100 includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected through a network, such as a wired or wireless network connection, etc., wherein the voice error detection device can be integrated in the server.

[0032] The server 102 can be configured to: obtain a to-be-detected voice of a target book and a plurality of pronunciation data of the target book, the plurality of pronunciation data including target pronunciation data corresponding to the to-be-detected voice; input the to-be-detected voice into a pre-trained voice error detection model, and combine the plurality of pronunciation data to obtain a voice error detection result of the to-be-detected voice relative to the target pronunciation data.

[0033] The server 102 can be a single server, or a server cluster or a cloud server composed of multiple servers. For example, the server can be an interworking server or a background server between multiple heterogeneous systems, or an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc. As disclosed in the voice error detection method or device of the present disclosure, the multiple servers can be composed into a blockchain, and the server is a node on the blockchain.

[0034] The terminal 101 can send the to-be-detected voice of the target book to the server 102, or receive the voice error detection result generated by the server 102. The terminal 101 can include a mobile phone, a smart television, a tablet computer, a notebook computer, or a personal computer (PC, Personal Computer), etc. The terminal 101 can also be provided with a client, which can be an application program client or a browser client, etc.

[0035] Those skilled in the art can know that, Figure 1 The number of terminals and servers shown in the above is only illustrative, and any number of terminals and servers can be provided according to actual needs, and the present disclosure does not limit this.

[0036] The present example embodiment will be described in detail below with reference to the accompanying drawings and examples.

[0037] First, the present disclosure provides a voice error detection method, which can be executed by any electronic device with computing processing capability.

[0038] Figure 2 A flowchart of a voice error detection method according to an embodiment of the present disclosure is shown in FIG. 1. Figure 2 As shown, the voice error detection method provided in the present disclosure includes the following steps.

[0039] S201. Obtain the speech to be error - checked of the target question book and various pronunciation data of the target question book. The various pronunciation data includes the target pronunciation data corresponding to the speech to be error - checked.

[0040] It should be noted that the target question book can be the text of the question for speech error - checking, and the speech to be error - checked of the target question book can be the speech obtained by the object to be evaluated reading the text. For example, in the scenario of children's speech evaluation, the target question book can be the text of the question for evaluating the pronunciation accuracy of children, and the speech to be error - checked can be the audio file obtained by children reading the question text.

[0041] It should be noted that the target question book can have various pronunciation data, and the various pronunciation data of the target text can be the phoneme representations of various pronunciations.

[0042] Exemplarily, in the application scenario of English pronunciation evaluation, for the same English word "fast", there are British pronunciation and American pronunciation

[0043] two pronunciation methods, and both of these two pronunciations are correct pronunciations. In the application scenario of children's literacy, for the same Chinese character "省", there are two pronunciations "shěng" and "xǐng", and since this scenario is only used to evaluate children's literacy, both of these two pronunciations are correct pronunciations. It should be noted that the target pronunciation data is the pronunciation data corresponding to the speech to be error - checked. Specifically, when the speech to be error - checked is pronounced correctly, the target pronunciation data is the pronunciation data directly corresponding to the pronunciation method of the speech to be error - checked. For example, for the English word "fast", if the object to be evaluated pronounces it in the British pronunciation way, the target pronunciation data is

[0044] S202. Input the speech to be error - checked into a pre - trained speech error - checking model, and combine with various pronunciation data to obtain the speech error - checking result of the speech to be error - checked relative to the target pronunciation data.

[0045] In some embodiments, in order to enable the speech error - checking model to determine the target pronunciation data from various pronunciation data before error - checking, encoding operations can be performed on the various pronunciation data respectively to obtain multiple pronunciation feature vectors. Then input the speech to be error - checked into the speech error - checking model, and combine with the multiple pronunciation feature vectors, so that the speech error - checking model can select the target pronunciation data from the various pronunciation data according to the multiple pronunciation feature vectors, and further obtain the speech error - checking result of the speech to be error - checked relative to the target pronunciation data.

[0046] It should be noted that the voice error detection result can indicate whether the pronunciation manner of the voice to be detected conforms to the target pronunciation data, which can be directly shown in the form of conformity or non-conformity, or can be shown in the form of similarity, and the embodiments of the present disclosure do not make any limitation in this regard.

[0047] In some embodiments, for a large number of test papers containing multiple pronunciation data, in order to improve the error detection efficiency of the voice error detection model and avoid the low running efficiency caused by decoding multiple pronunciation data respectively, the multiple pronunciation feature vectors can be fused to obtain a fused feature vector in the process of encoding operation. The fused feature vector and the voice to be detected are input into the voice error detection model, and the voice error detection result can be obtained.

[0048] Among them, the fused feature vector can be used to express each of the multiple pronunciation data. That is, the fused feature vector is a new feature expression, and it has a certain correlation with the pronunciation feature vector of each pronunciation data it contains.

[0049] Exemplarily, the way to fuse multiple pronunciation feature vectors to obtain a fused feature vector can be: taking the mean of multiple pronunciation feature vectors as a fused feature vector, and / or inputting multiple pronunciation feature vectors into a pre-trained vector fusion model to obtain a fused feature vector.

[0050] In some embodiments, the vector fusion model can use a relatively simple neural network, for example, a multi-layer perceptron (MLP), to learn the encoding manner of the fused feature. In specific implementation, since the dimension of the input vector of MLP is fixed, and the number of types of multiple pronunciation data is not necessarily fixed, the multi-hot expression manner in multi-label can be used for reference, and the 0 vector is filled in the case of insufficient number of pronunciation data types, so as to ensure that the front dimension of the input MLP is fixed.

[0051] Exemplarily, Figure 3 Taking the English word "fast" as an example, the fusion manner of multiple pronunciation data is shown. "Fast" has two pronunciation manners of British English and American English Among the two pronunciation manners, f, s, t are the same phonemes in the two pronunciation manners, and are different phonemes in the two pronunciation manners. Since the same phonemes between the two pronunciation manners only need to undergo one encoding and one decoding in the calling process of the voice error detection model, they do not need to undergo fusion processing. However, for different phonemes between the two pronunciation manners (for example, Figure 3 , and ), the fusion feature vector can be input into the speech error detection model after fusion to avoid low running efficiency caused by multiple encoding and decoding.

[0052] In some embodiments, when the fusion feature vector is input into the speech error detection model, the encoding information of whether the fusion vector contains multiple pronunciation data can be additionally introduced into the fusion vector to obtain a complete feature vector (i.e. Figure 3 In this way, the speech error detection model can distinguish between the fusion feature vector expressing multiple pronunciation data and the pronunciation feature vector expressing only one pronunciation data when decoding the pronunciation feature vector, reduce the interference between the fusion feature vector and the pronunciation feature vector, and improve the learning effect of the speech error detection model.

[0053] In the encoding process, the multiple pronunciation data are encoded into one fusion vector, so that the speech error detection model provided by the present disclosure only needs to be decoded once during training and application, thereby avoiding low running efficiency caused by multiple decoding.

[0054] In some embodiments, the speech error detection model provided by the present disclosure can include a pronunciation prediction model and a pronunciation error detection model connected in sequence. The pronunciation prediction model is used to determine target pronunciation data corresponding to the speech to be detected in the multiple pronunciation data. The pronunciation error detection model is used to detect the speech to be detected according to the target pronunciation data.

[0055] It is worth noting that, since it is difficult to collect audio data samples of multiple pronunciation types in the training process of the speech error detection model, the present disclosure further provides a training method of a speech error detection model, which can fully train the speech error detection model provided by the present disclosure under the condition of small samples.

[0056] Figure 4 The flowchart of the training method of the speech error detection model provided by the present disclosure is shown in FIG. 1. As shown in FIG. 1, the method includes the following steps: Figure 4

[0057] S401, based on the pronunciation prediction model, predicting a target pronunciation sample corresponding to the speech sample from multiple pronunciation samples to obtain a first loss between the real pronunciation sample corresponding to the speech sample and the target pronunciation sample.

[0058] It should be noted that the reading material of the speech sample is a reading material containing multiple pronunciation methods, and the multiple pronunciation samples are used to record the phoneme representation of the multiple correct pronunciation methods of the reading material. The target pronunciation sample is a pronunciation sample in the multiple pronunciation samples that can show the pronunciation method of the speech sample, and the real pronunciation sample is used to show the real pronunciation method of the speech sample.

[0059] ​In some embodiments, if there is no real pronunciation sample in the plurality of pronunciation samples, any one of the plurality of pronunciation samples is taken as a real pronunciation sample.

[0060] Specifically, since the plurality of pronunciation samples are all correct pronunciation manners, when the speech sample is an incorrect pronunciation manner, there will be no real pronunciation sample in the plurality of pronunciation samples, at this time, any one of the plurality of pronunciation samples can be randomly taken as a real pronunciation sample to complete the training of the speech error detection model.

[0061] S402, input the speech sample and the real pronunciation sample into the pronunciation error detection model to obtain a second loss between the error detection label of the speech sample and the error detection result output by the pronunciation error detection model.

[0062] In some embodiments, the speech sample is pre-set with an error detection label, and the error detection label is used to show the real error detection result of the speech sample, that is, whether the pronunciation manner of the speech sample conforms to one of the plurality of pronunciation manners recorded in the plurality of pronunciation samples.

[0063] S403, based on the first loss and the second loss, the parameters of the speech error detection model are adjusted to complete the training of the speech error detection model.

[0064] For ease of understanding, the specific process of the speech error detection model training method shown in Figure 5 will be described below. Figure 4 The speech error detection model training method shown in

[0065] As shown in Figure 5 , the structure of the pronunciation prediction model and the pronunciation error detection model of the speech error detection model in the embodiment of the present disclosure can be the same, and both include an encoder, a multi-head attention (Multi-head Attention) layer and an MLP layer connected in series. The encoder is used to convert the to-be-evaluated speech data and the pronunciation data into feature vectors. The multi-head attention layer is used to match and fuse the feature vectors of the to-be-evaluated speech data and the feature vectors of the pronunciation data. Finally, through the MLP layer, the prediction of the target pronunciation data or the prediction of the error detection result is completed.

[0066] First, please refer to the pronunciation prediction model in Figure 5 . In the pronunciation prediction model, for the same English word "fast", there are two pronunciation manners, English pronunciation and American pronunciation , and the pronunciation samples (i.e. phoneme representations of pronunciation manners) corresponding to the two pronunciation manners and the speech samples (x1, x2, x3 and x4 in Figure 5 ) are taken as inputs of the pronunciation prediction model. Among them, for the repeated phonemes between the two pronunciation manners, only one pronunciation feature vector can be input (for example, Figure 5y1, y3 and y4 in the phoneme f, s and t), while the phonemes different between the two pronunciations are input as a fused feature vector after fusion (for example, Figure 5 y3 in the phoneme and ).

[0067] The output of the pronunciation prediction model is a predicted target pronunciation sample, that is, a target pronunciation sample corresponding to the speech sample is predicted from multiple pronunciation samples. At this point, a first loss generated by the pronunciation prediction model can be calculated by comparing the target pronunciation sample with a real pronunciation sample (for example, Figure 5 y1, y3 and y4 in the phoneme f, s and t), while the phonemes different between the two pronunciations are input as a fused feature vector after fusion (for example, ).

[0068] Next, refer to the pronunciation error detection model in Figure 5 . In the pronunciation error detection model, the above-mentioned real pronunciation sample (for example, Figure 5 y1, y3 and y4 in the phoneme f, s and t), while the phonemes different between the two pronunciations are input as a fused feature vector after fusion (for example, ).

[0069] Finally, the first loss and the second loss are added together as the overall loss of the speech error detection model, and the entire model parameters are updated to complete the training of the speech error detection model.

[0070] The speech error detection method provided by the embodiments of the present disclosure can obtain the speech to be detected and multiple pronunciation data of the target book after obtaining the speech to be detected of the target book, input the speech to be detected into the pre-trained speech error detection model, and combine the multiple pronunciation data to obtain the speech error detection result of the speech to be detected relative to the target pronunciation data. The method provided by the embodiments of the present disclosure can effectively handle the multi-pronunciation problem in speech error detection, improve the speech error detection effect in the multi-pronunciation scene, and avoid the problem that the result is inaccurate when the multi-pronunciation book still relies on the single-pronunciation mode for speech error detection.

[0071] The embodiments of the present disclosure further provide a training method of the speech error detection model. The speech error detection model is divided into a pronunciation prediction model and a pronunciation error detection model to calculate the loss respectively. Since the speech sample containing multiple pronunciation modes is only relied on during the training of the pronunciation prediction model, the technical problem of difficulty in collecting the multi-pronunciation speech sample in the related art can be overcome, and a small sample training mode is provided.

[0072] Based on the same inventive concept, the embodiment of the present disclosure also provides a voice error detection device, as follows. Since the principle of solving problems of the device embodiment is similar to the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0073] Figure 6 The structure of a voice error detection device in the embodiment of the present disclosure is shown in the schematic diagram, as shown in the figure, the voice error detection device 600 includes: an acquisition module 601 and an error detection module 602. Figure 6

[0074] Specifically, the acquisition module 601 is configured to acquire the to-be-detected voice of the target corpus and a plurality of pronunciation data of the target corpus, the plurality of pronunciation data including target pronunciation data corresponding to the to-be-detected voice. The error detection module 602 is configured to input the to-be-detected voice into a pre-trained voice error detection model, and combine the plurality of pronunciation data to obtain a voice error detection result of the to-be-detected voice relative to the target pronunciation data.

[0075] In some embodiments, the error detection module 602 is further configured to perform an encoding operation on the plurality of pronunciation data respectively to obtain a plurality of pronunciation feature vectors; input the to-be-detected voice into the voice error detection model, and combine the plurality of pronunciation feature vectors to obtain the voice error detection result.

[0076] In some embodiments, the error detection module 602 is further configured to fuse the plurality of pronunciation feature vectors to obtain a fused feature vector, the fused feature vector being configured to express each of the plurality of pronunciation data simultaneously; input the fused feature vector and the to-be-detected voice into the voice error detection model to obtain the voice error detection result.

[0077] In some embodiments, the error detection module 602 is further configured to take a mean value of the plurality of pronunciation feature vectors as the fused feature vector; and / or input the plurality of pronunciation feature vectors into a pre-trained vector fusion model to obtain the fused feature vector.

[0078] In some embodiments, the plurality of pronunciation data includes a phoneme representation of each of a plurality of pronunciations of the target corpus.

[0079] In some embodiments, the voice error detection model includes a pronunciation prediction model and a pronunciation error detection model connected in sequence; the pronunciation prediction model is configured to determine the target pronunciation data corresponding to the to-be-detected voice in the plurality of pronunciation data; and the pronunciation error detection model is configured to detect the to-be-detected voice according to the target pronunciation data.

[0080] ​In some embodiments, the voice error detection apparatus 600 further comprises a training module configured to: predict, based on the pronunciation prediction model, a target pronunciation sample corresponding to the voice sample from a plurality of pronunciation samples, to obtain a first loss between the target pronunciation sample and a real pronunciation sample corresponding to the voice sample; input the voice sample and the real pronunciation sample into the pronunciation error detection model to obtain a second loss between the error detection label of the voice sample and an error detection result output by the pronunciation error detection model; and perform parameter adjustment on the pronunciation error detection model based on the first loss and the second loss to complete training of the pronunciation error detection model.

[0081] In some embodiments, the training module is further configured to, if the real pronunciation sample does not exist in the plurality of pronunciation samples, take any pronunciation sample in the plurality of pronunciation samples as the real pronunciation sample.

[0082] It should be noted that the voice error detection apparatus provided in the above embodiments is used for voice error detection, and the division of the above functional modules is only used as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the voice error detection apparatus and the voice error detection method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0083] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied as a whole hardware embodiment, a whole software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0084] The electronic device 700 according to this embodiment of the present disclosure will be described below with reference to Figure 7 Figure 7 The electronic device 700 is merely an example and should not impose any limitation on the function and application scope of the embodiments of the present disclosure.

[0085] As shown in Figure 7 The electronic device 700 is in the form of a general computing device. The components of the electronic device 700 can include, but are not limited to, the above-mentioned at least one processing unit 710, the above-mentioned at least one storage unit 720, and a bus 730 connecting different system components, including the storage unit 720 and the processing unit 710.

[0086] ​The storage unit stores program codes which can be executed by the processing unit 710, so that the processing unit 710 performs the steps described in the above “Exemplary Methods” section according to various exemplary embodiments of the present disclosure.

[0087] In some embodiments, the processing unit 710 can perform the following steps of the above method embodiments:

[0088] Obtain the to-be-checked speech of the target text and a plurality of pronunciation data of the target text, the plurality of pronunciation data including target pronunciation data corresponding to the to-be-checked speech; input the to-be-checked speech into a pre-trained speech checking model, and combine the plurality of pronunciation data to obtain a speech checking result of the to-be-checked speech relative to the target pronunciation data.

[0089] The storage unit 720 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 7201, and / or a non-volatile storage unit, such as a read-only memory (ROM) 7203.

[0090] The storage unit (RAM) 7201 and / or the cache storage unit 7202 can further include a read-only storage unit (ROM) 7203.

[0091] The storage unit 720 can also include a program storage unit (program memory) 7204 having a set of one or more program modules 7205 that are executable by the processing unit 710, including an operating system, one or more application programs, other program modules, and program data, and various combinations thereof, which can include implementation of a network environment, each of these examples, or some combination thereof.

[0092] The program modules 7205 include, or are implemented using, but are not limited to, an operating system, one or more application programs, other program modules, and program data, and each of these examples, or some combination thereof, can include implementation of a network environment.

[0093] The bus 730 can be one or more of several types of bus structures including an industry standard architecture (ISA), peripheral component interconnect (PCI) bus, a

[0094] or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0095] The electronic device 700 can also communicate with one or more external devices 740 such as a keyboard or pointing device, a Bluetooth device, etc.; user interfaces and / or peripheral interfaces. Additionally, the electronic device 700 can further communicate with one or more devices that enable user interaction with the electronic device 700 (e.g., a

[0096] communication with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface 750. Still yet, the electronic device 700 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via the network adapter 760.

[0097] or public networks, such as the Internet) communication. As shown Figure 7 The network adapter 760 communicates to the other modules of the electronic device 700 via the bus 730. It should be appreciated that although the network adapter 760 is shown as a separate component from the bus 730, the network adapter 760 can be integrated with the bus 730, and / or the bus 730 can be a part of the network adapter 760.

[0098] The electronic device 700 can use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0099] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software and also can be implemented by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0100] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product stored thereon can implement the above-mentioned methods of the present disclosure. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps described in the above-mentioned “example method” section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0101] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0102] In the present disclosure, the computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave propagating through the program code readable by a machine, a device or an apparatus. Such a propagating data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit the program for use by or in connection with an instruction execution system, device or apparatus.

[0103] Alternatively, program code contained in the computer readable storage medium can be transferred to a computing device for execution using any suitable transmission medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination thereof.

[0104] In an implementation, the program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., or conventional procedural programming languages, such as the "C" programming language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0105] It should be noted that, although several modules or units of the device for action execution are mentioned in the foregoing detailed description, such a division is not mandatory. Indeed, features and functionalities of two or more modules or units described above can be embodied in one module or unit according to embodiments of the present disclosure. Conversely, features and functionalities of one module or unit described above can be further divided into multiple modules or units embodied.

[0106] Furthermore, although the various steps of the methods of the present disclosure are described in a particular order in the figures, this is not required or implied, nor is it necessary to perform all of the steps shown in order to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0107] From the above description of the embodiments, those skilled in the art will readily appreciate that the example embodiments described herein can be implemented by software and / or by hardware. Embodiments according to the present disclosure can therefore be realized in a software selective product stored in a non-transitory storage medium such as a CD-ROM, a USB key, a RAM, a ROM, etc. or transmitted over a network, comprising a plurality of instructions to make a computing device (e.g. a personal computer, a server, a mobile terminal, a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0108] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the disclosure be construed as including any patents, patent applications, publications, publications, or other disclosure of complementary subject matter that is within the scope of the disclosure. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A voice error detection method, characterized by, The method comprises the following steps: acquiring a to-be-checked pronunciation of a target book and a plurality of pronunciation data of the target book, wherein the plurality of pronunciation data comprises target pronunciation data corresponding to the to-be-checked pronunciation; performing an encoding operation on the plurality of pronunciation data respectively to obtain a plurality of pronunciation feature vectors; inputting the to-be-checked pronunciation and the plurality of pronunciation feature vectors into a pre-trained pronunciation checking model, so that the pronunciation checking model selects the target pronunciation data from the plurality of pronunciation data according to the plurality of pronunciation feature vectors, and obtains a pronunciation checking result of the to-be-checked pronunciation relative to the target pronunciation data according to the to-be-checked pronunciation and the target pronunciation data; the pronunciation checking result is used to indicate whether the pronunciation manner of the to-be-checked pronunciation is consistent with the target pronunciation data.

2. The method of claim 1, wherein, The step of inputting the to-be-checked pronunciation and the plurality of pronunciation feature vectors into the pre-trained pronunciation checking model, so that the pronunciation checking model selects the target pronunciation data from the plurality of pronunciation data according to the plurality of pronunciation feature vectors, and obtains the pronunciation checking result of the to-be-checked pronunciation relative to the target pronunciation data according to the to-be-checked pronunciation and the target pronunciation data, comprises the following steps: fusing the plurality of pronunciation feature vectors to obtain a fused feature vector, wherein the fused feature vector is used to simultaneously express each pronunciation data in the plurality of pronunciation data; inputting the fused feature vector and the to-be-checked pronunciation into the pronunciation checking model, so that the pronunciation checking model selects the target pronunciation data from the plurality of pronunciation data according to the fused feature vector, and obtains the pronunciation checking result according to the to-be-checked pronunciation and the target pronunciation data.

3. The method of claim 2, wherein, The step of fusing the plurality of pronunciation feature vectors to obtain a fused feature vector comprises the following steps: taking a mean value of the plurality of pronunciation feature vectors as the fused feature vector; and / or inputting the plurality of pronunciation feature vectors into a pre-trained vector fusion model to obtain the fused feature vector. The pronunciation checking model comprises a pronunciation prediction model and a pronunciation checking model connected in sequence; 4. The method according to any one of claims 1 to 3, characterized in that, the pronunciation prediction model is used to determine the target pronunciation data corresponding to the to-be-checked pronunciation from the plurality of pronunciation data; the pronunciation checking model is used to check the to-be-checked pronunciation according to the target pronunciation data. The training process of the pronunciation checking model comprises the following steps:

5. The method of claim 4, wherein, based on the pronunciation prediction model, predicting a target pronunciation sample corresponding to a voice sample from a plurality of pronunciation samples to obtain a first loss between a real pronunciation sample corresponding to the voice sample and the target pronunciation sample; inputting the voice sample and the real pronunciation sample into the pronunciation checking model to obtain a second loss between a checking label of the voice sample and a checking result output by the pronunciation checking model; based on the first loss and the second loss, adjusting parameters of the pronunciation checking model to complete the training of the pronunciation checking model. The method further comprises the following steps:

6. The method of claim 5, wherein, if the real pronunciation sample does not exist in the plurality of pronunciation samples, taking any pronunciation sample in the plurality of pronunciation samples as the real pronunciation sample. The method comprises the following steps:

7. A voice error detection apparatus, characterized by comprising: ​ An acquisition module is configured to acquire a to-be-checked speech of a target book and a plurality of pronunciation data of the target book, the plurality of pronunciation data comprising target pronunciation data corresponding to the to-be-checked speech; A checking module is configured to respectively perform an encoding operation on the plurality of pronunciation data to obtain a plurality of pronunciation feature vectors; input the to-be-checked speech and the plurality of pronunciation feature vectors into a pre-trained speech checking model, so that the speech checking model selects the target pronunciation data from the plurality of pronunciation data according to the plurality of pronunciation feature vectors, and obtains a speech checking result of the to-be-checked speech relative to the target pronunciation data according to the to-be-checked speech and the target pronunciation data. The speech checking result is used to indicate whether the pronunciation manner of the to-be-checked speech is consistent with the target pronunciation data.

8. An electronic device, comprising: comprise: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the method of any one of claims 1 to 6 via execution of the executable instructions.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the method of any one of claims 1 to 6.

Images