Text error correction processing method and device, computer equipment and readable storage medium

Abstract

The invention provides a text error correction processing method and device, computer equipment and a computer readable storage medium, and belongs to the field of natural language processing. The text error correction processing method comprises the following steps: acquiring a text feature of each target character in a to-be-processed text; obtaining a voice feature of each target character in the to-be-processed text; obtaining a font feature of each target character in the to-be-processed text; based on the text feature of each target character, the voice feature of each target character and the font feature of each target character, performing fusion processing to obtain a feature integration representation of each target character in the to-be-processed text; and performing error correction processing based on the feature integration representation of each target character in the to-be-processed text through a trained error correction model to obtain an error-corrected text of the to-be-processed text. According to the method, text errors can be detected and corrected by using information between different modes, and the text error correction capability is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text error correction processing method, apparatus, computer equipment and computer-readable storage medium. Background Art

[0002] Speech recognition technology is a technology that uses speech as a research object, and allows machines to automatically recognize and understand human spoken language through speech signal processing and pattern recognition. Speech recognition technology is a technology that allows machines to convert speech data into corresponding text through the recognition and understanding process.

[0003] However, due to the influence of subjective or objective factors such as accent and oral habits, speech recognition technology often has the problem of inaccurate recognition. Therefore, it is necessary to correct the speech recognition text, that is, to correct the errors in the speech recognition text to obtain a more accurate speech recognition text. Therefore, in actual business usage scenarios, text error correction is crucial for speech recognition technology; for example, in insurance services, text error correction technology can play an important role. It can automatically identify and correct wrong words or phrases for customers' voice input information, allowing customers to input information more smoothly, thereby improving customer experience and work efficiency. However, in related technologies, the text error correction accuracy of text error correction schemes is low. Summary of the invention

[0004] The present application provides a text error correction processing method, apparatus, computer equipment and computer-readable storage medium, which belong to the field of natural language processing technology and can use information between different modalities to detect and correct text errors and improve text error correction capabilities.

[0005] In a first aspect, the present application provides a text error correction processing method, the method comprising:

[0006] Obtain text features of each target character in the text to be processed;

[0007] Acquire the speech feature of each target character in the text to be processed;

[0008] Obtaining the glyph features of each target character in the text to be processed;

[0009] Performing fusion processing based on the text features of each target character, the phonetic features of each target character, and the glyph features of each target character to obtain a feature integrated representation of each target character in the text to be processed;

[0010] Through the trained error correction model, error correction processing is performed based on the feature integration representation of each target character in the text to be processed, so as to obtain the error-corrected text of the text to be processed.

[0011] In a second aspect, the present application provides a text error correction processing device, the text error correction processing device comprising:

[0012] An acquisition unit, used for acquiring text features of each target character in the text to be processed;

[0013] The acquisition unit is further used to acquire the speech feature of each target character in the text to be processed;

[0014] The acquisition unit is further used to acquire the glyph features of each target character in the text to be processed;

[0015] A fusion unit, used for performing fusion processing based on the text feature of each target character, the voice feature of each target character and the glyph feature of each target character to obtain a feature integrated representation of each target character in the text to be processed;

[0016] The processing unit is used to perform error correction processing based on the feature integration representation of each target character in the text to be processed through a trained error correction model to obtain a corrected text of the text to be processed.

[0017] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the text error correction processing method when executing the computer program.

[0018] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the text error correction processing method.

[0019] In the present application, on the one hand, by acquiring the text features, voice features and glyph features of each target character in the text to be processed, the text features, voice features and glyph features can be used to fuse the three dimensions of semantic, acoustic and visual information to detect and correct text errors, improve the accuracy of the error correction model, and improve the text error correction capability; on the other hand, by fusing the text features, voice features and glyph features to control the information flow between different modalities, the error correction model can better utilize the information between different modalities, thereby improving the accuracy of the error correction model and improving the text error correction capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flow chart of a text error correction processing method provided by an embodiment of the present application;

[0022] Figure 2 This is a flowchart of an embodiment of step 104 in the embodiment of the present application;

[0023] Figure 3 This is a flowchart of an embodiment of step 104A in the embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of a principle structural framework of an error correction model in an embodiment of the present application;

[0025] Figure 5 This is a flowchart of an embodiment of error correction model training in an embodiment of the present application;

[0026] Figure 6 It is a schematic diagram of the structure of an embodiment of a text error correction processing device provided in an embodiment of the present application;

[0027] Figure 7 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0031] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0032] Embodiments of the present application provide a text error correction processing method, apparatus, computer equipment, and computer-readable storage medium.

[0033] The executor of the text error correction processing method of the embodiment of the present application may be the text error correction processing device provided in the embodiment of the present application, or the computer device provided in the embodiment of the present application, wherein the text error correction processing device may be implemented in hardware or software.

[0034] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0035] See also Figure 1 , Figure 1 1 is a flowchart of a text error correction processing method provided by an embodiment of the present application. The text error correction processing method includes steps 101 to 105, wherein:

[0036] 101. Obtain text features of each target character in the text to be processed.

[0037] The target character refers to each character in the text to be processed.

[0038] The text to be processed may be text obtained through speech recognition technology. For example, in an insurance service scenario, the text automatically recognized from the customer's speech input information may be used as the text to be processed.

[0039] Exemplarily, the text encoding sublayer in the error correction model trained in steps 501 to 505 can be applied to obtain the character vector of each target character in the text to be processed and the position vector of each target character; the character vector of each target character and the position vector of each target character are concatenated to obtain the first concatenated vector of each target character; based on the trained text encoding parameters, the first concatenated vector of each target character is encoded to obtain the text features of each target character in the text to be processed. The specific implementation principle can be referred to the relevant description of the text encoding sublayer later, and will not be repeated here for the sake of simplicity.

[0040] 102. Acquire the speech feature of each target character in the text to be processed.

[0041] Exemplarily, the speech coding sublayer in the error correction model trained in steps 501 to 505 can be applied to obtain the audio signal data of the text to be processed; based on the audio signal data of the text to be processed, the speech vector of each target character in the text to be processed is obtained; the position vector of each target character in the text to be processed is obtained; the speech vector of each target character and the position vector of each target character are concatenated to obtain the second concatenated vector of each target character; based on the trained speech coding parameters, the second concatenated vector of each target character is encoded to obtain the speech features of each target character in the text to be processed. The specific implementation principle can be referred to the relevant description of the speech coding sublayer in the following text. For the sake of simplicity, it will not be repeated here.

[0042] 103. Obtain the glyph features of each target character in the text to be processed.

[0043] Exemplarily, the glyph encoding sublayer in the error correction model trained in steps 501 to 505 can be applied to obtain the character retrieval image of each target character in the text to be processed; based on the trained glyph encoding parameters, the character retrieval image of each target character is encoded to obtain the glyph features of each target character in the text to be processed. The specific implementation principle can be referred to the relevant description of the glyph encoding sublayer below, and will not be repeated here for the sake of simplicity.

[0044] 104. Perform fusion processing based on the text feature of each target character, the phonetic feature of each target character, and the glyph feature of each target character to obtain a feature integrated representation of each target character in the text to be processed.

[0045] For example, if the sentence "You didn't do your homework, you are so stupid" is input, the text features of each target character i are obtained through steps 101, 102, and 103 respectively. Voice Features Font features This results in representation vectors of the semantic, acoustic, and visual modalities with the same dimensionality.

[0046] There are many ways to perform the fusion processing in step 104, illustratively including:

[0047] (1) Directly concatenate the text features, phonetic features, and glyph features of each target character in a certain order as the integrated feature representation of each target character.

[0048] (2) In order to enable the error correction model to better utilize the information between different modalities to improve the accuracy of the error correction model, a selective gate unit is designed in this embodiment to control how much information can flow into the mixed multimodal representation. For example, if a character is misspelled because its pronunciation is similar to the correct pronunciation, then more information in the acoustic morphology should flow into the mixed representation, that is, more information about the phonetic features of the character should flow into the feature integrated representation of the character. Figure 2 As shown, at this time, step 104 may specifically include the following steps 1041A to 1044A:

[0049] 1041A. Obtain the text fusion weight of each target character according to the text features of each target character, the voice features of each target character, and the glyph features of each target character by using the trained text selection gate parameters.

[0050] There are multiple ways to obtain the text fusion weight of each target character in step 1041A, illustratively including the following ways ① and ②:

[0051] ① In some embodiments, the text fusion weight of each target character is determined based on the text feature of each target character, the phonetic feature of each target character, the glyph feature of each target character, and the mean value of the text feature of the text to be processed. Figure 3 As shown, at this time, step 1041A may specifically include the following steps A1 to A2:

[0052] A1. Based on the text feature of each target character, obtain the text feature mean of the text to be processed.

[0053] For example, first, the text feature of each target character in the text to be processed can be Substitute into the preset mean calculation formula, for example, the preset mean calculation formula is shown in Formula 1 below, and calculate the text feature mean of the text to be processed

[0054]

[0055] In formula 1, N represents the number of characters in the text to be processed. Represents the text feature of the i-th character in the text to be processed, Represents the mean value of the text features of the text to be processed.

[0056] A2. Obtain the text fusion weight of each target character according to the text features of each target character, the phonetic features of each target character, the glyph features of each target character, and the mean of the text features by using the trained text selection gate parameters.

[0057] For example, the text feature of each target character in the text to be processed is The phonetic features of each target character The glyph features of each target character And the mean of text features Substitute into the preset text fusion calculation formula, where the preset text fusion calculation formula is a calculation formula that contains the trained text selection gate parameters. For example, the preset text fusion calculation formula is shown in the following formula 2, and the text fusion weight of each target character is calculated.

[0058]

[0059] In formula 2, represents the text fusion weight of the i-th character in the text to be processed, Respectively represent the text features, semantic features, and glyph features of the i-th character in the text to be processed, W t 、b t Select gate parameters for text, W t 、b t It can be obtained through learning and training, σ is the activation function, and [·] represents the concatenation of vectors.

[0060] Therefore, on the one hand, by using the mean value of the text features of the text to be processed to determine the text fusion weight of each target character, the overall semantics of the entire input sentence corresponding to the text to be processed can be captured, and the ability of the text fusion weight to capture character spelling errors caused by the overall semantic proximity can be improved, thereby improving the feature expression ability of the feature integrated representation of each target character to improve the text error correction accuracy. On the other hand, by using the text features of each target character, the phonetic features of each target character, and the glyph features of each target character to determine the text fusion weight of each target character, it is possible to combine the three modal information of semantics, acoustics, and vision to determine how many text features flow into the hybrid representation, improve the feature expression ability of the feature integrated representation of each target character, and improve the text error correction accuracy. Therefore, the text fusion weight of each target character can capture the spelling errors of a character due to its semantic similarity to the semantics of the correct character, so that more text features flow into the hybrid representation, and improve the text error correction accuracy when the character is misspelled due to semantic similarity.

[0061] ② In some embodiments, the text fusion weight of each target character is determined based on the text feature of each target character, the voice feature of each target character, and the glyph feature of each target character. In this case, step 1041A may specifically include: The phonetic features of each target character And the glyph features of each target character Substitute into the preset text fusion calculation formula, where the preset text fusion calculation formula is a calculation formula that includes the trained text selection gate parameters. For example, the preset text fusion calculation formula is shown in the following formula 3, and the text fusion weight of each target character is calculated.

[0062]

[0063] 1042A. Obtain the speech fusion weight of each target character according to the text features of each target character, the speech features of each target character, and the glyph features of each target character by using the trained speech selection gate parameters.

[0064] There are multiple ways to obtain the speech fusion weight of each target character in step 1042A, illustratively including the following ways 1 and 2:

[0065] Mode 1: In some embodiments, the speech fusion weight of each target character is determined based on the text feature of each target character, the speech feature of each target character, the glyph feature of each target character, and the mean value of the text feature of the to-be-processed text. In this case, step 1042A may specifically include the following steps B1 to B2:

[0066] B1. Based on the text feature of each target character, obtain the text feature mean of the text to be processed.

[0067] The implementation of step B1 is similar to that of step A1. For details, please refer to the relevant instructions in the previous text and will not be repeated here.

[0068] B2. Obtain the speech fusion weight of each target character according to the text features of each target character, the speech features of each target character, the glyph features of each target character, and the mean of the text features by using the trained speech selection gate parameters.

[0069] For example, the text feature of each target character in the text to be processed is The phonetic features of each target character The glyph features of each target character And the mean of text features Substitute into the preset voice fusion calculation formula, where the preset voice fusion calculation formula is a calculation formula that includes the trained voice selection gate parameters. For example, the preset voice fusion calculation formula is shown in the following formula 4, and the voice fusion weight of each target character is calculated.

[0070]

[0071] In formula 4, represents the speech fusion weight of the i-th character in the text to be processed, Respectively represent the text features, semantic features, and glyph features of the i-th character in the text to be processed, W a 、b a Select the gate parameter for speech, W a 、b a It can be obtained through learning and training, σ is the activation function, and [·] represents the concatenation of vectors.

[0072] Thus, on the one hand, by using the mean value of the text features of the text to be processed to determine the speech fusion weight of each target character, the overall semantics of the entire input sentence corresponding to the text to be processed can be captured, and the ability of the speech fusion weight to capture character spelling errors caused by the proximity of the overall semantics can be improved, thereby improving the feature expression ability of the feature integration representation of each target character to improve the accuracy of text error correction. On the other hand, by using the text features of each target character, the speech features of each target character, and the glyph features of each target character to determine the speech fusion weight of each target character, it is possible to combine the semantic, acoustic, and visual modal information to determine how many speech features flow into the hybrid representation, and improve the feature expression ability of the feature integration representation of each target character to improve the accuracy of text error correction. Thus, the speech fusion weight of each target character can capture the spelling errors of a character due to its pronunciation being similar to that of the correct character, so that more speech features flow into the hybrid representation, and improve the accuracy of text error correction when the character is misspelled due to similar pronunciation.

[0073] Mode 2: In some embodiments, the speech fusion weight of each target character is determined based on the text feature of each target character, the speech feature of each target character, and the glyph feature of each target character. In this case, step 1042A may specifically include: The phonetic features of each target character And the glyph features of each target character Substitute into the preset voice fusion calculation formula, where the preset voice fusion calculation formula is a calculation formula that includes the trained voice selection gate parameters. For example, the preset voice fusion calculation formula is shown in the following formula 5, and the voice fusion weight of each target character is calculated.

[0074]

[0075] 1043A. Obtain the glyph fusion weight of each target character according to the text feature of each target character, the voice feature of each target character and the glyph feature of each target character by using the trained glyph selection gate parameters.

[0076] There are many ways to obtain the glyph fusion weight of each target character in step 1043A, illustratively, including the following ways: <1> and <2> :

[0077] Way <1> : In some embodiments, the glyph fusion weight of each target character is determined based on the text feature of each target character, the voice feature of each target character, the glyph feature of each target character, and the mean value of the text feature of the text to be processed. At this time, step 1043A may specifically include the following steps C1 to C2:

[0078] C1. Based on the text feature of each target character, obtain the text feature mean of the text to be processed.

[0079] The implementation of step C1 is similar to step A1. For details, please refer to the relevant instructions above and will not be repeated here.

[0080] C2. Obtaining the glyph fusion weight of each target character according to the text features of each target character, the voice features of each target character, the glyph features of each target character, and the mean of the text features through the trained glyph selection gate parameters.

[0081] For example, the text feature of each target character in the text to be processed is The phonetic features of each target character The glyph features of each target character And the mean of text features Substitute into the preset glyph fusion calculation formula, where the preset glyph fusion calculation formula is a calculation formula that includes the trained glyph selection gate parameters, such as the preset glyph fusion calculation formula shown in the following formula 6, and calculate the glyph fusion weight of each target character

[0082]

[0083] In formula 6, represents the glyph fusion weight of the i-th character in the text to be processed, Respectively represent the text features, semantic features, and glyph features of the i-th character in the text to be processed, W v 、b v Select the gate parameters for the glyph, W v 、b v It can be obtained through learning and training, σ is the activation function, and [·] represents the concatenation of vectors.

[0084] Thus, on the one hand, by using the mean value of the text features of the text to be processed to determine the glyph fusion weight of each target character, the overall semantics of the entire input sentence corresponding to the text to be processed can be captured, and the glyph fusion weight can be improved to capture the character spelling errors caused by the overall semantic proximity, thereby improving the feature expression ability of the feature integrated representation of each target character to improve the text error correction accuracy. On the other hand, by using the text features of each target character, the voice features of each target character, and the glyph features of each target character to determine the glyph fusion weight of each target character, it is possible to combine the semantic, acoustic, and visual modal information to determine how many glyph features flow into the mixed representation, and improve the feature expression ability of the feature integrated representation of each target character to improve the text error correction accuracy. Thus, the glyph fusion weight of each target character can capture the spelling errors of a character due to the similarity of its glyph to the glyph of the correct character, so that more glyph features flow into the mixed representation, and improve the text error correction accuracy when the character is spelled incorrectly due to glyph approximation.

[0085] Way <2> : In some embodiments, based on the text features of each target character, the voice features of each target character, and the glyph features of each target character, the glyph fusion weight of each target character is determined. At this time, step 1043A may specifically include: The phonetic features of each target character And the glyph features of each target character Substitute into the preset glyph fusion calculation formula, where the preset glyph fusion calculation formula is a calculation formula that includes the trained glyph selection gate parameters, such as the preset glyph fusion calculation formula as shown in the following formula 7, and calculate the glyph fusion weight of each target character

[0086]

[0087] 1044A. Determine a feature integrated representation of each target character in the text to be processed based on the text fusion weight of each target character, the speech fusion weight of each target character, the glyph fusion weight of each target character, the text feature of each target character, the speech feature of each target character and the glyph feature of each target character.

[0088] Exemplarily, the text fusion weight of each target character in the text to be processed is The phonetic fusion weight of each target character The glyph fusion weight for each target character Text features for each target character The phonetic features of each target character And the glyph features of each target character Substitute into the preset fusion formula, for example, the preset fusion formula is shown in Formula 8 below, and calculate the feature integration representation of each target character

[0089]

[0090] 105. Perform error correction processing based on the feature integration representation of each target character in the text to be processed by using the trained error correction model to obtain a corrected text of the text to be processed.

[0091] For example, first, the feature integration representation of all target characters in the to-be-processed text can be packaged into After that, the error correction layer of the trained error correction model is input; then, the error correction layer of the trained error correction model is used to integrate the features of each target character i. The probability of the entire vocabulary is calculated, and the character with the highest probability is selected as the replacement character for the target character i to obtain the error-corrected text of the text to be processed.

[0092] Please refer to Figure 4 In order to better understand the present embodiment, the error correction model in the present embodiment is first introduced below. The error correction model includes a feature fusion coding layer and an error correction layer, which are specifically as follows:

[0093] 1. Feature fusion coding layer, which is used to fuse and encode the text in three modalities: semantic, acoustic and visual, to obtain the integrated feature representation of each character in the text. The feature fusion coding layer can further include a text coding sublayer, a speech coding sublayer, a glyph coding sublayer and a coding fusion sublayer.

[0094] 1.1. The text encoding sublayer is used to encode the text features of each character in the text to obtain the text features of each character in the text. Specifically, the position vector of each character and the character vector of each character can be obtained; the position vector and the character vector of each character are spliced ​​together according to a preset splicing order to obtain the first splicing vector of each character; then, according to the text encoding parameters of the text encoding sublayer, the first splicing vector of each character is further encoded to obtain the text features of each character. Exemplarily, the BERT model can be used as the text encoding sublayer.

[0095] Among them, the text encoding sublayer includes text encoding parameters. After the trained error correction model is obtained through the training method of steps 501 to 505, the text encoding parameters of the text encoding sublayer in the trained error correction model can be extracted as the trained text encoding parameters in step 101, which are used to extract the text features of each target character.

[0096] 1.2. The speech coding sublayer is used to encode the speech features of each character in the text to obtain the speech features of each character in the text. Specifically, based on the audio signal data of the text, the speech vector of each character and the position vector of each character in the text can be obtained; the position vector and the speech quantity of each character are spliced ​​together according to the preset splicing order to obtain the second splicing vector of each character; then, according to the speech coding parameters of the speech coding sublayer, the second splicing vector of each character is further encoded to obtain the speech features of each character. Exemplarily, the speech coding sublayer can use the Mel Spectrograms model to convert the audio signal data of the text into a spectrogram, and then use the spectrogram of the text to extract the spectral characteristics of each character in the text and perform vector representation to obtain the speech vector of each character.

[0097] Among them, the speech coding sublayer includes speech coding parameters. After obtaining the trained error correction model through the training method of steps 501 to 505, the speech coding parameters of the speech coding sublayer in the trained error correction model can be extracted as the trained speech coding parameters in step 102, which are used to extract the speech features of each target character.

[0098] 1.3. The glyph encoding sublayer is used to encode the glyph features of each character in the text to obtain the glyph features of each character in the text. Specifically, first, an image is retrieved for each character in the text, and a character retrieval image of each character; then, according to the glyph encoding parameters of the glyph encoding sublayer, the character retrieval image of each character is further encoded to obtain the glyph features of each character.

[0099] Exemplarily, in order to avoid training from scratch, VGG19 pre-trained on ImageNet can be used to construct a glyph encoding sublayer; VGG (Visual Geometry Group) is a deep convolutional neural network model, and VGG19 pre-trained on ImageNet has a total of 19 layers (including convolutional layers and fully connected layers), including 16 convolutional layers and 3 fully connected layers; the convolutional layer uses a 3x3 convolution kernel, a stride of 1, and a padding of 1, which ensures that the size of the feature map remains unchanged between convolutional layers; the pooling layer uses a 2x2 maximum pooling operation with a stride of 2 to reduce the spatial dimension of the feature map; in this embodiment, the last classification layer of VGG19 is removed, and the last maximum pooling layer of the VGG19 backbone is used as the output to form a glyph encoding sublayer.

[0100] Among them, the glyph coding sublayer includes glyph coding parameters. After the trained error correction model is obtained through the training method of steps 501 to 505, the glyph coding parameters of the glyph coding sublayer in the trained error correction model can be extracted as the trained glyph coding parameters in step 103, which are used to extract the glyph features of each target character.

[0101] 1.4, the coding fusion sublayer is used to perform fusion processing based on the text features of each character, the voice features of each character and the glyph features of each character to obtain the feature integration representation of each character in the text. The detailed implementation can refer to step 104, which is not repeated here to simplify the description. Among them, the coding fusion sublayer includes text selection gate parameters, voice selection gate parameters, and glyph selection gate parameters. After obtaining the trained error correction model through the training method of steps 501 to 505, the text selection gate parameters, voice selection gate parameters, and glyph selection gate parameters of the coding fusion sublayer in the trained error correction model can be extracted as the trained voice selection gate parameters in step 1041A, the trained voice selection gate parameters in step 1042A, and the trained glyph selection gate parameters in step 1043A, which are respectively used to obtain the text fusion weight of each target character, the voice fusion weight of each target character, and the glyph fusion weight of each target character, thereby obtaining the feature integration representation of each target character.

[0102] 2. Error correction layer. In this embodiment, a fully connected layer is used as the error correction layer. The feature integration representation of each character is input into the error correction layer constructed by the fully connected layer to calculate the probability of the entire vocabulary, wherein the character with the highest probability is selected as the replacement character of the character, thereby realizing text error correction.

[0103] Please refer to Figure 5 , exemplarily, the error correction model can be trained by the following steps 501 to 505:

[0104] 501. Perform mask processing on the sample text to obtain a masked text of the sample text.

[0105] The masked text includes a preset proportion of masked characters, and the original characters in the sample text are replaced with phonetic confusing words, graphic confusing words, random words, or original words to obtain the masked characters.

[0106] Exemplarily, during the training process of the error correction model, in this implementation, a certain proportion (for example, 15% in this embodiment) of correct characters are randomly disturbed in the source text, a special character token is used to mask the correct token, and the error correction model is trained to predict the correct character token from the masked character token. In this embodiment, four masking strategies of masking categories are adopted, as follows:

[0107] Category 1: Replacement of phonetic and linguistically confusing words

[0108] Category 2: Confusing Word Replacement

[0109] Category 3: Random word replacement

[0110] Category 4: Original word retention (to prevent overfitting)

[0111] By masking 15% of the character tokens, and the masking strategies of the four masking categories account for 60%, 15%, 10%, and 15% respectively, a portion of the correct tokens in the source text are randomly masked, so that the model can not only correct the original wrong tokens during learning, but also predict the masked tokens based on their contextual information. For example, input the sentence "You didn't do your homework, you are so stupid", the masking strategy of this embodiment no longer just masks out the wrong characters, but also masks out the correct characters to improve the model's ability to learn contextual information. The sentence after masking becomes: "You <mask>Ran Wen is doing homework, you are really a tiger picture", among which, <mask>Indicates that the character "居" is replaced by a random word <mask>Replacement: 'Hu' is replaced with its homophone 'Hu' (tiger), and 'Mei' is replaced with its similar-looking character 'Wen'.

[0112] 502. Through the feature fusion encoding layer in the preset error correction model, perform feature encoding processing on the masked text to obtain the feature integrated representation of each sample character in the masked text.

[0113] Wherein, a sample character refers to each character in the masked text.

[0114] The method for obtaining the feature integrated representation of each sample character in step 502 is similar to the method for obtaining the feature integrated representation of each target character. Specifically, reference can be made to the relevant descriptions in steps 101-103 and steps 1041A-10444A. For the sake of simplicity of expression, it will not be elaborated here.

[0115] 503. Through the error correction layer in the preset error correction model, perform error correction processing according to the feature integrated representation of each sample character in the masked text to obtain the corrected text of the sample text.

[0116] 504. Based on the corrected text of the sample text and the sample text, determine the target loss value of the preset error correction model.

[0117] Wherein, the target loss value includes at least one of a homophone confusion word correction loss value, a similar-looking character confusion word correction loss value, a random word correction loss value, and an original word correction loss value.

[0118] For example, in this embodiment, the error correction layer is trained using the cross-entropy loss function. For example, the loss function of the j-th masked category of the error correction layer is as shown in formula 9 below. Given a training sample (X, Y), the correction loss value of the j-th masked category is defined as:

[0119]

[0120] In formula 9, is the cross-entropy loss, X is the character sequence, Y is the corrected sequence of X, c represents the total number of masked categories. In this embodiment, there are a total of 4 masked categories (respectively: homophone confusion word replacement, similar-looking character confusion word replacement, random word replacement, original word retention), so c = 4 in this embodiment.

[0121] Taking the example that the target loss value includes the correction loss value of the pronunciation confused word, the correction loss value of the shape confused word, the correction loss value of the random word and the correction loss value of the original word, at this time, in step 504, referring to formula 9, based on the corrected text of the sample text and the sample text, the correction loss value of the pronunciation confused word of the preset error correction model is determined; based on the corrected text of the sample text and the sample text, the correction loss value of the shape confused word of the preset error correction model is determined; based on the corrected text of the sample text and the sample text, the correction loss value of the random word of the preset error correction model is determined; based on the corrected text of the sample text and the sample text, the correction loss value of the original word of the preset error correction model is determined; and the result obtained by summing the correction loss value of the pronunciation confused word, the shape confused word, the correction loss value and the original word correction loss value is used as the target loss value.

[0122] 505. Adjust the model parameters of the error correction layer in the preset error correction model based on the target loss value to obtain a trained error correction model.

[0123] From the above content, it can be seen that, on the one hand, by obtaining the text features, voice features and glyph features of each target character in the text to be processed, and using the text features, voice features and glyph features, the three dimensions of semantic, acoustic and visual information can be integrated to detect and correct text errors, improve the accuracy of the error correction model, and improve the text error correction ability; on the other hand, by fusing the text features, voice features and glyph features to control the information flow between different modalities, the error correction model can better utilize the information between different modalities, thereby improving the accuracy of the error correction model and improving the text error correction ability.

[0124] It can be seen that this embodiment improves the text error correction capability through the following aspects. First, multi-feature fusion: by utilizing text features, speech features and glyph features, the three dimensional information of semantics, acoustics and vision can be fused to detect and correct text errors. The error correction model captures the three dimensional information of semantics, acoustics and vision respectively by using a customized text coding sublayer, speech coding sublayer and glyph coding sublayer, and adopts a selective modal fusion mechanism through the coding fusion sublayer to control the information flow between these modalities. This fusion mechanism enables the error correction model to better utilize the information between different modalities, thereby improving the accuracy of the error correction model and improving the text error correction capability.

[0125] Secondly, effective use of correct character tokens: by masking the correct tags in the source text (i.e., the masking strategy of the random word replacement category), the model is encouraged to predict tags only through context. This masking strategy of the random word replacement category allows the model to better utilize contextual information, avoiding the problem that the error rate of incorrect sentences is usually very low (e.g., 10%), which causes the error correction model to only learn to correct errors on limited incorrect tags and simply copy on most tags (correct tags), thereby improving the utilization of correct information in the sentence, thereby improving the accuracy of the error correction model and improving the text correction ability; in addition, this masking strategy of the random word replacement category allows each token to participate in the correction loss, which can be back-propagated without affecting the overall distribution, and can also improve the text correction ability of the error correction model to a certain extent.

[0126] Third, it is highly robust: the mask strategies of the four mask categories can be used as a plug-and-play module to improve the text error correction capability.

[0127] In addition, in order to better implement the text error correction processing method in the embodiment of the present application, based on the text error correction processing method, the embodiment of the present application also provides a text error correction processing device, such as Figure 6 As shown, it is a schematic diagram of the structure of an embodiment of a text error correction processing device provided in an embodiment of the present application. The text error correction processing device 600 includes:

[0128] An acquisition unit 601 is used to acquire text features of each target character in the text to be processed;

[0129] The acquisition unit 601 is further used to acquire the speech feature of each target character in the text to be processed;

[0130] The acquisition unit 601 is further used to acquire the glyph features of each target character in the text to be processed;

[0131] A fusion unit 602 is used to perform fusion processing based on the text feature of each target character, the voice feature of each target character and the glyph feature of each target character to obtain a feature integrated representation of each target character in the text to be processed;

[0132] The processing unit 603 is used to perform error correction processing based on the feature integration representation of each target character in the text to be processed by using the trained error correction model to obtain a corrected text of the text to be processed.

[0133] In some embodiments, the fusion unit 602 is specifically used for:

[0134] By using the trained text selection gate parameters, according to the text features of each target character, the voice features of each target character and the glyph features of each target character, a text fusion weight of each target character is obtained;

[0135] Acquire the speech fusion weight of each target character according to the text feature of each target character, the speech feature of each target character and the glyph feature of each target character by using the trained speech selection gate parameter;

[0136] Obtaining the glyph fusion weight of each target character according to the text features of each target character, the voice features of each target character and the glyph features of each target character by using the trained glyph selection gate parameters;

[0137] Determine a feature integrated representation of each target character in the text to be processed based on the text fusion weight of each target character, the speech fusion weight of each target character, the glyph fusion weight of each target character, the text feature of each target character, the speech feature of each target character and the glyph feature of each target character.

[0138] In some embodiments, the fusion unit 602 is specifically used for:

[0139] Based on the text feature of each target character, obtaining a text feature mean of the text to be processed;

[0140] The text fusion weight of each target character is obtained by using the trained text selection gate parameters according to the text features of each target character, the voice features of each target character, the glyph features of each target character, and the mean of the text features.

[0141] In some embodiments, the text error correction processing device further includes a training unit (not shown in the figure), which is specifically used to:

[0142] Performing masking processing on the sample text to obtain masked text of the sample text, wherein the masked text includes masked characters of a preset proportion, and the original characters in the sample text are replaced with phonetic confusing words, graphic confusing words, random words, or original words to obtain the masked characters;

[0143] Performing feature encoding processing on the masked text through a feature fusion encoding layer in a preset error correction model to obtain a feature integrated representation of each sample character in the masked text;

[0144] Through the error correction layer in the preset error correction model, error correction processing is performed according to the feature integrated representation of each sample character in the masked text to obtain a corrected text of the sample text;

[0145] Based on the corrected text of the sample text and the sample text, determining the target loss value of the preset error correction model, wherein the target loss value includes at least one of a correction loss value of a phonetic-confused word, a correction loss value of a pictographic-confused word, a correction loss value of a random word, and a correction loss value of an original word;

[0146] Based on the target loss value, the model parameters of the error correction layer in the preset error correction model are adjusted to obtain a trained error correction model.

[0147] In some embodiments, the acquisition unit 601 is specifically used to:

[0148] Obtaining a character vector of each target character in the text to be processed and a position vector of each target character;

[0149] Performing splicing processing on the character vector of each target character and the position vector of each target character to obtain a first splicing vector of each target character;

[0150] Based on the trained text encoding parameters, the first concatenated vector of each target character is encoded to obtain the text feature of each target character in the text to be processed.

[0151] In some embodiments, the acquisition unit 601 is specifically used to:

[0152] Acquire audio signal data of the text to be processed;

[0153] Based on the audio signal data of the text to be processed, obtaining a speech vector of each target character in the text to be processed;

[0154] Obtaining a position vector of each target character in the text to be processed;

[0155] Performing splicing processing on the voice vector of each target character and the position vector of each target character to obtain a second splicing vector of each target character;

[0156] Based on the trained speech coding parameters, encoding processing is performed based on the second concatenated vector of each target character to obtain the speech feature of each target character in the text to be processed.

[0157] In some embodiments, the acquisition unit 601 is specifically used to:

[0158] Obtaining a character retrieval image of each target character in the text to be processed;

[0159] Based on the trained glyph encoding parameters, the character retrieval image of each target character is encoded to obtain the glyph features of each target character in the text to be processed.

[0160] In specific implementation, the above-mentioned units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above-mentioned units can refer to the previous text error correction processing method embodiment, which will not be repeated here.

[0161] See also Figure 7 , Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0162] like Figure 7 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0163] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any text error correction processing method.

[0164] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0165] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any text error correction processing method.

[0166] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0168] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0169] Obtain text features of each target character in the text to be processed; obtain speech features of each target character in the text to be processed; obtain glyph features of each target character in the text to be processed; perform fusion processing based on the text features of each target character, the speech features of each target character and the glyph features of each target character to obtain a feature integrated representation of each target character in the text to be processed; perform error correction processing based on the feature integrated representation of each target character in the text to be processed through a trained error correction model to obtain a corrected text of the text to be processed.

[0170] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned text error correction method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0171] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any text error correction processing method provided by the embodiment of the present application.

[0172] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0173] In the above-mentioned text error correction processing device and computer-readable storage medium embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described text error correction processing device, computer-readable storage medium and its corresponding units can refer to the description of the text error correction processing method in the above embodiment, and will not be repeated here.

[0174] The above is a detailed introduction to a text error correction processing method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.< / mask> < / mask> < / mask>

Claims

1. A text error correction processing method, characterized in that: The method comprises: Obtain text features of each target character in the text to be processed; Acquire the speech feature of each target character in the text to be processed; Obtaining the glyph features of each target character in the text to be processed; Performing fusion processing based on the text features of each target character, the phonetic features of each target character, and the glyph features of each target character to obtain a feature integrated representation of each target character in the text to be processed; Through the trained error correction model, error correction processing is performed based on the feature integration representation of each target character in the text to be processed, so as to obtain the error-corrected text of the text to be processed.

2. The text error correction processing method according to claim 1, characterized in that: The fusing process based on the text feature of each target character, the phonetic feature of each target character and the glyph feature of each target character to obtain the feature integrated representation of each target character in the text to be processed includes: By using the trained text selection gate parameters, according to the text features of each target character, the voice features of each target character and the glyph features of each target character, a text fusion weight of each target character is obtained; Acquire the speech fusion weight of each target character according to the text feature of each target character, the speech feature of each target character and the glyph feature of each target character by using the trained speech selection gate parameter; Obtaining the glyph fusion weight of each target character according to the text features of each target character, the voice features of each target character and the glyph features of each target character by using the trained glyph selection gate parameters; Determine a feature integrated representation of each target character in the text to be processed based on the text fusion weight of each target character, the speech fusion weight of each target character, the glyph fusion weight of each target character, the text feature of each target character, the speech feature of each target character and the glyph feature of each target character.

3. The text error correction processing method according to claim 2, characterized in that: The method of obtaining the text fusion weight of each target character according to the text feature of each target character, the voice feature of each target character and the glyph feature of each target character by using the trained text selection gate parameter includes: Based on the text feature of each target character, obtaining a text feature mean of the text to be processed; The text fusion weight of each target character is obtained by using the trained text selection gate parameters according to the text features of each target character, the voice features of each target character, the glyph features of each target character, and the mean of the text features.

4. The text error correction processing method according to claim 1, characterized in that: The trained error correction model is trained in the following way: Performing masking processing on the sample text to obtain masked text of the sample text, wherein the masked text includes masked characters of a preset proportion, and the original characters in the sample text are replaced with phonetic confusing words, graphic confusing words, random words, or original words to obtain the masked characters; Performing feature encoding processing on the masked text through a feature fusion encoding layer in a preset error correction model to obtain a feature integrated representation of each sample character in the masked text; Through the error correction layer in the preset error correction model, error correction processing is performed according to the feature integrated representation of each sample character in the masked text to obtain a corrected text of the sample text; Based on the corrected text of the sample text and the sample text, determining the target loss value of the preset error correction model, wherein the target loss value includes at least one of a correction loss value of a phonetic-confused word, a correction loss value of a pictographic-confused word, a correction loss value of a random word, and a correction loss value of an original word; Based on the target loss value, the model parameters of the error correction layer in the preset error correction model are adjusted to obtain a trained error correction model.

5. The text error correction processing method according to claim 1, characterized in that: The step of obtaining the text feature of each target character in the text to be processed includes: Obtaining a character vector of each target character in the text to be processed and a position vector of each target character; Performing splicing processing on the character vector of each target character and the position vector of each target character to obtain a first splicing vector of each target character; Based on the trained text encoding parameters, the first concatenated vector of each target character is encoded to obtain the text feature of each target character in the text to be processed.

6. The text error correction processing method according to claim 1, characterized in that: The step of obtaining the speech feature of each target character in the text to be processed includes: Acquire audio signal data of the text to be processed; Based on the audio signal data of the text to be processed, obtaining a speech vector of each target character in the text to be processed; Obtaining a position vector of each target character in the text to be processed; Performing splicing processing on the voice vector of each target character and the position vector of each target character to obtain a second splicing vector of each target character; Based on the trained speech coding parameters, encoding processing is performed based on the second concatenated vector of each target character to obtain the speech feature of each target character in the text to be processed.

7. The text error correction processing method according to claim 1, characterized in that: The step of obtaining the glyph features of each target character in the text to be processed includes: Obtaining a character retrieval image of each target character in the text to be processed; Based on the trained glyph encoding parameters, the character retrieval image of each target character is encoded to obtain the glyph features of each target character in the text to be processed.

8. A text error correction processing device, characterized in that: The text error correction processing device comprises: An acquisition unit, used for acquiring text features of each target character in the text to be processed; The acquisition unit is further used to acquire the speech feature of each target character in the text to be processed; The acquisition unit is further used to acquire the glyph features of each target character in the text to be processed; A fusion unit, used for performing fusion processing based on the text feature of each target character, the voice feature of each target character and the glyph feature of each target character to obtain a feature integrated representation of each target character in the text to be processed; The processing unit is used to perform error correction processing based on the feature integration representation of each target character in the text to be processed through a trained error correction model to obtain a corrected text of the text to be processed.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the text error correction processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the text error correction processing method described in any one of claims 1 to 7.

Images