Text processing method and apparatus
By dynamically adjusting the regularization model parameters and combining the Transformer model and XGBoost algorithm, standard text is predicted character by character, solving the accuracy problem of fixed text regularization model parameters and achieving high-accuracy conversion from non-standard text to standard text.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-23
- Publication Date
- 2026-03-13
AI Technical Summary
Once existing text regularization models are trained, their parameters are fixed, resulting in insufficient accuracy when converting non-standard text to standard text, especially when processing dates and numbers, which can easily lead to confusion.
By acquiring category labels from non-standard text, dynamically adjusting the model parameters of the regularization model, using the Transformer model for text processing, and combining the XGBoost algorithm and pointer generation network, standard text is predicted character by character.
It improves the accuracy of converting non-standard text to standard text and ensures the correct conversion of special symbols such as dates and numbers.
Smart Images
Figure CN112632914B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of speech and semantic recognition, and in particular to a text processing method and apparatus. Background Art
[0002] Text normalization is an essential part of a conversation system. For example, in speech synthesis and automatic speech recognition, non-standard text is converted into standard text representing a standard expression. For example, special symbols such as numbers, dates, currencies, and abbreviations are converted into pre-set standard text. For example, the number 123$ representing a price is converted into English: one hundred and twenty-three dollars, or into Chinese: one hundred and twenty-three US dollars.
[0003] Currently, once the regularization model for text normalization is trained, the model parameters of the regularization model will remain unchanged. When using this regularization model to perform regularization processing on non-standard text, the resulting standard text is often not accurate enough. For example, when performing regularization processing on dates and numbers, it is easy to be confused and unable to distinguish the differences, resulting in the same standard text. Summary of the Invention
[0004] Embodiments of the present invention provide a text processing method and apparatus, which can determine the value of the model parameters according to the non-standard category corresponding to the non-standard text, thereby improving the accuracy of converting non-standard text into standard text.
[0005] In a first aspect, embodiments of the present invention provide a text processing method, including:
[0006] Obtain non-standard text and determine the non-standard category corresponding to the non-standard text;
[0007] According to the non-standard category, determine the update value of the model parameters of the regularization model;
[0008] Update the value of the model parameters of the regularization model using the update value, and process the non-standard text using the updated regularization model to obtain at least one vector;
[0009] According to the at least one vector, determine the standard text corresponding to the non-standard text.
[0010] In a possible implementation, the at least one vector includes a first vector and at least one second vector, and one second vector corresponds to one character position in the standard text, and the second vector is used to determine the character at the character position in the standard text;
[0011] The first vector is the vector output by the encoder in the updated regularization model, and the second vector is the vector output by the decoder in the updated regularization model.
[0012] In one possible implementation, the standard text corresponding to the non-standard text is determined based on the at least one vector;
[0013] The second vector is processed by the linear layer and softmax layer in the neural network to obtain the third vector corresponding to the second vector. The third vector is used to represent the probability that the character at the character position corresponding to the second vector is a preset character in the preset vocabulary.
[0014] Based on the first vector and the second vector, a fourth vector corresponding to the second vector is obtained. The fourth vector is used to represent the importance of the characters contained in the non-standard text to the characters at the predicted character positions corresponding to the second vector.
[0015] The standard text corresponding to the non-standard text is determined based on at least one third vector corresponding to the at least one second vector and at least one fourth vector corresponding to the at least one second vector, wherein one second vector corresponds to one third vector and one second vector corresponds to one fourth vector.
[0016] In one possible implementation, determining the standard text corresponding to the non-standard text based on at least one third vector corresponding to the at least one second vector and at least one fourth vector corresponding to the at least one second vector includes:
[0017] For each of the at least one second vector, obtain the weight value corresponding to the second vector;
[0018] Based on the third vector corresponding to the second vector, the fourth vector corresponding to the second vector, and the weight value corresponding to the second vector, the final character at the character position corresponding to the second vector is obtained;
[0019] Based on the final character corresponding to each of the at least one second vector, construct the standard text corresponding to the non-standard text.
[0020] In one possible implementation, obtaining the weight value corresponding to the second vector includes:
[0021] Obtain the feature vectors corresponding to the characters contained in the non-standard text;
[0022] Based on the feature vector and the second vector, the weight value corresponding to the second vector is obtained.
[0023] In one possible implementation, obtaining the weight value corresponding to the second vector based on the feature vector and the second vector includes:
[0024] Based on the feature vector, the context vector corresponding to the non-standard text is obtained;
[0025] Based on the context vector and the second vector, the weight value corresponding to the second vector is obtained.
[0026] In one possible implementation, the regularization model is a Transformer model.
[0027] Secondly, embodiments of the present invention provide a text processing apparatus, comprising:
[0028] An acquisition unit is used to acquire non-standard text and determine the non-standard category corresponding to the non-standard text;
[0029] The first determining unit is used to determine the updated values of the model parameters of the regularized model based on the non-standard category.
[0030] The model update unit is used to update the values of the model parameters of the regularization model using the update value, and to process the non-standard text using the updated regularization model to obtain at least one vector.
[0031] The second determining unit is used to determine the standard text corresponding to the non-standard text based on the at least one vector.
[0032] In one possible implementation, the at least one vector includes a first vector and at least one second vector, wherein the second vector corresponds to a character position in the standard text, and the second vector is used to determine the character at the character position;
[0033] The first vector is the vector output by the encoder in the updated regularization model, and the second vector is the vector output by the decoder in the updated regularization model.
[0034] In one possible implementation, the second determining unit is specifically used for:
[0035] The second vector is processed by the linear layer and softmax layer in the neural network to obtain the third vector corresponding to the second vector. The third vector is used to represent the probability that the character at the character position corresponding to the second vector is a preset character in the preset vocabulary.
[0036] Based on the first vector and the second vector, a fourth vector corresponding to the second vector is obtained. The fourth vector is used to represent the importance of the characters contained in the non-standard text to the characters at the predicted character positions corresponding to the second vector.
[0037] The standard text corresponding to the non-standard text is determined based on at least one third vector corresponding to the at least one second vector and at least one fourth vector corresponding to the at least one second vector, wherein one second vector corresponds to one third vector and one second vector corresponds to one fourth vector.
[0038] In one possible implementation, the second determining unit is specifically used for:
[0039] For each of the at least one second vector, obtain the weight value corresponding to the second vector;
[0040] Based on the third vector corresponding to the second vector, the fourth vector corresponding to the second vector, and the weight value corresponding to the second vector, the final character at the character position corresponding to the second vector is obtained;
[0041] Based on the final character corresponding to each of the at least one second vector, construct the standard text corresponding to the non-standard text.
[0042] In one possible implementation, the second determining unit is specifically used for:
[0043] Obtain the feature vectors corresponding to the characters contained in the non-standard text;
[0044] Based on the feature vector and the second vector, the weight value corresponding to the second vector is obtained.
[0045] In one possible implementation, the second determining unit is specifically used for:
[0046] Based on the feature vector, the context vector corresponding to the non-standard text is obtained;
[0047] Based on the context vector and the second vector, the weight value corresponding to the second vector is obtained.
[0048] In one possible implementation, the regularization model is a Transformer model.
[0049] Thirdly, embodiments of the present invention provide a text processing device, the text processing device including a processor, a memory and a communication interface, the processor, the memory and the communication interface being interconnected, wherein the communication interface is used to receive and send data, the memory is used to store program code, and the processor is used to call the program code to execute the method described in the first aspect.
[0050] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the methods described above.
[0051] In this embodiment of the invention, non-standard text is obtained, and the non-standard category corresponding to the non-standard text is determined. The updated value of the model parameters of the regularization model is determined according to the non-standard category, and the updated value is used to update the value of the model parameters of the regularization model. Thus, the updated regularization model is used to perform regularization processing on the non-standard text. In this application, the model parameters of the regularization model are not fixed, but dynamically adjusted according to the non-standard category corresponding to the non-standard text. This allows the updated regularization model to perform regularization processing on the non-standard text more accurately, improving the accuracy of converting non-standard text into standard text. Attached Figure Description
[0052] To illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0053] Figure 1 A flowchart of a text processing method provided in an embodiment of the present invention;
[0054] Figure 2 A system architecture diagram for text processing provided in an embodiment of the present invention;
[0055] Figure 3 A schematic block diagram illustrating a text processing method provided in an embodiment of the present invention;
[0056] Figure 4 This is a schematic diagram of the structure of a text processing device provided in an embodiment of the present invention;
[0057] Figure 5 This is a schematic diagram of another text processing device provided in an embodiment of the present invention. Detailed Implementation
[0058] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.
[0059] The following will be combined with the appendix Figure 1 -Appendix Figure 3 This paper provides a detailed description of a text processing method provided by an embodiment of the present invention.
[0060] Please see Figure 1 This is a flowchart illustrating a text processing method according to an embodiment of the present invention. Figure 1 As shown, the text processing method of this embodiment of the invention may include the following steps S101-S104.
[0061] S101, Obtain non-standard text and determine the non-standard category corresponding to the non-standard text;
[0062] In this embodiment, fixed-length text is input into the extreme gradient boosting (XGBoost) algorithm, a tool for massively parallel boosted trees, which is an open-source boosted tree toolkit. The algorithm used by XGBoost is an improvement on gradient boosting decision tree (GBDT), and can be used for both classification and regression problems. In this embodiment, the XGBoost algorithm is used to classify characters in the text.
[0063] Specifically, the XGBoost algorithm outputs category labels corresponding one-to-one with the character positions in the text. Optionally, if the input text is in English, each standard word (i.e., words contained in the standard text) is called a loanword, and each standard word occupies one character position and has one category label. Each non-standard word (i.e., characters contained in the non-standard text) occupies one character position and has one category label. The category label indicates the category of the text, where the category corresponding to non-standard text is the non-standard category. Similarly, if the input text is in Chinese, each Chinese character or word can occupy one character position and have one category label. This category label indicates the category to which the corresponding Chinese character, word, or character belongs. The category corresponding to non-standard text is the non-standard category, and the category corresponding to standard text is the standard category. For example, non-standard categories may include, but are not limited to, dates, currencies, addresses, letters, cardinal numbers, ordinal numbers, URLs, units of measurement, fractional forms, decimal forms, phone numbers, time, place values, and punctuation. Standard categories can be loanwords. Each category is assigned a category label in turn. For example, for the 15 categories mentioned above, the category labels can be numbers from 1 to 15.
[0064] For example, such as Figure 2As shown, the input text to the XGBoost algorithm is 'totally 1 2 3 $', and the corresponding XGBoost algorithm output is '15 2 2 2 2'. '15' indicates that 'totally' is a foreign word, and the four '2's represent the categories of '1 2 3 $' as currency. Then, based on the category labels, the non-standard text (the non-standard part of the input text, i.e., the part that needs regularization, referred to as non-standard text) and its category labels are sent to the regularization system for regularization processing, outputting the standard text corresponding to the non-standard text. The processing procedure of the regularization system is detailed in steps S102-S104 and will not be elaborated here. For example, in the fixed-length text 'totally 1 2 3 $', the non-standard text input to the regularization system is '1 2 3 $', and the category label is 2, indicating that the non-standard category is currency.
[0065] S102, Based on the non-standard category, determine the updated values of the model parameters of the regularized model;
[0066] In this embodiment, the regularization model can be a Transformer model, which includes an encoder and a decoder. The updated values of the model parameters of the regularization model can be determined based on the non-standard categories corresponding to the non-standard text. Different non-standard categories correspond to different updated values for the model parameters. These updated values can be updated model weight values, meaning the model parameters of the regularization model are dynamically adjusted based on the non-standard categories of the non-standard text, rather than using static model parameters. Specifically, the category labels corresponding to the non-standard categories can be input into the parameter generation network. The parameter generation network generates updated values for different model parameters based on different category labels. That is, the input to the parameter generation network is a label from 0 to 15, and each non-standard text corresponds to only one label (e.g., the label for non-standard text '12 3 $' is 2). The output of the parameter generation network is the value of each parameter in the Transformer model.
[0067] S103, the updated value is used to update the values of the model parameters of the regularization model, and the updated regularization model is used to process the non-standard text to obtain at least one vector;
[0068] In this embodiment, after the model parameters of the regularization model are updated with new parameter values, non-standard text can be input into the updated regularization model for processing, thereby obtaining at least one vector output by the regularization model. This at least one vector may include a first vector Vector_E output by the encoder and at least one second vector Vector_D_i output by the decoder, where i represents the i-th output vector of the decoder. Each second vector corresponds to a character position in the standard text, which is the standard text corresponding to the input non-standard text. That is, the number of characters in the standard text is the same as the number of second vectors. For example, if the standard text contains 4 characters, then the regularization model outputs 4 second vectors accordingly.
[0069] Optionally, the dimension of the first vector can be [n * 768], where n represents the number of characters included in the non-standard text of the input encoder, and the dimension of the second vector Vector_D_i can be [1 * 768].
[0070] S104, determine the standard text corresponding to the non-standard text based on the at least one vector.
[0071] Understandably, the entire prediction process of the Transformer model is character-by-character prediction, meaning the decoder outputs only one vector Vector_D_i corresponding to the current character position each time, until the decoder predicts "[SEP]". Optionally, the first vector Vector_E output by the encoder and the second vector Vector_D_i output by the decoder are processed each time to obtain the character at the current character position. For example, the first vector Vector_E output by the encoder and the second vector Vector_D_i output by the decoder can be input into the pointer generation network for processing to obtain the character at the current character position. The processing procedure of the pointer generation network is described below, including the following steps:
[0072] Step 1: In this embodiment, for each of the at least one second vector, the second vector can be processed by a linear layer and a softmax layer in the neural network to obtain a third vector corresponding to the second vector. The third vector is used to represent the probability that the character at the character position corresponding to the second vector predicted by the decoder is any of the preset characters in the preset vocabulary. The third vector is Project, then Project = Softmax(Linear(Vector_D_i)), its dimension is [1 * vocab_size], where vocab_size is the size of the preset vocabulary, and the Project vector represents the probability of all characters in the preset vocabulary at the character position currently decoded by the decoder.
[0073] Step 2: For each of the at least one second vector, a fourth vector corresponding to the second vector can be obtained based on the first and second vectors. This fourth vector represents the importance of non-standard characters not included in the input encoder to the character at the predicted character position of the second vector. The fourth vector is denoted as attm. Specifically, the attention module calculates the weight score of the first vector Vector_E relative to the second vector Vector_D_i, using the formula attm = Attention(Vector_E, Vector_D_i). Attm is a vector with dimension [1 * n], where n is the number of non-standard characters in the input encoder, and attm represents the importance (score) of each character in the non-standard text of the input encoder relative to the character at the current character position predicted by the decoder.
[0074] Step 3: After obtaining the third vector Project and the fourth vector attm corresponding to Vector_D_i, the standard text corresponding to the non-standard text can be determined based on the third vector Project and the fourth vector attm.
[0075] Specifically, based on the formula Calculate the context vector corresponding to the first vector Vector_E, with dimensions [1 * 768], where n represents the number of characters input to the encoder, and i represents the i-th position. This represents the feature vector (with dimensions [1 * 768]) corresponding to the i-th character in the first vector Vector_E. This represents the score at the i-th position of the fourth vector attm (which is a scalar value).
[0076] The context vector and the second vector Vector_D_i are summed and then passed through a linear layer to calculate the weight value (scalar value between 0 and 1) P corresponding to the second vector Vector_D_i. The formula is P = Linear(context + Vector_D_i).
[0077] Based on the size of the preset vocabulary, the fourth vector `attm` corresponding to the second vector is adjusted, specifically according to the formula: `attm = attm / (vocab_size / n)`. Then, some probability values in the Project vector are adjusted. Specifically, if the character at position j of the preset vocabulary appears at position i of the input non-standard text, the probability value j in Project needs to be adjusted. The formula is `attm_i_j * (1-P) + Project_j * P`, where Project_j represents the value at position j in Project (i.e., the predicted word position j), and `attm_i_j` represents the score at position i of the input non-standard text. It's understandable that if the character at position j of the preset vocabulary does not appear in the input non-standard text, the probability value j in Project is not adjusted. The final adjusted Project is Project_Out. This vector Project_Out represents the probability distribution of character positions in the corresponding output standard text. Specifically, this vector represents the probability of each character in the preset vocabulary that the decoder can output in the current step. The character with the highest probability is selected, and the character at the character position in the current step of the decoder is obtained. All the predicted characters are then used to construct the standard text.
[0078] The following example illustrates the calculation process: The preset vocabulary is a dictionary, assuming a total dictionary size (vocab_size) of 10 characters (w1, w2, w3, w4, w5, w6, w7, w8, w9, w10), with each index corresponding to the character's position in the dictionary; the weight value P = 0.3, and the input text consists of four characters: [w5, w10, w3, w8], resulting in an attm of [0.2, 0.1, 0.4, 0.3]. The decoder's current Project = [0.02, 0.28, 0.09, 0.01, 0.05, 0.02, 0.03, 0.3, 0.1, 0.1]. After formula adjustment, attm = [0.08, 0.04, 0.16, ...]. [0.12], by summing the score of each character in the dictionary with the corresponding dictionary position in the vector Project, we get: Project = [0.02,0.28, 0.4*(1-0.3)+0.09*0.3, 0.01, 0.2*(1-0.3)+ 0.05*0.3, 0.02, 0.03, 0.3*(1-0.3)+0.3*0.3, 0.1, 0.1*(1-0.3)+0.1*0.3]=[0.02, 0.28, 0.307, 0.01, 0.155,0.02, 0.03, 0.3, 0.1, 0.1], where the maximum probability value is 0.307, located in the third position of the dictionary, which means the standard text of the encoder in the current step is w3.
[0079] It should be noted that the decoder stops predicting when the standard text predicted by the pointer network in the current step is [SEP], indicating that all characters in the standard text corresponding to the non-standard text have been predicted. At this point, the system has completed the regularization of the parts that need to be regularized.
[0080] Finally, the standard text categories determined by the XGBoost algorithm and the standard text obtained by the final regularization system are concatenated back according to their original positions to obtain the final output text.
[0081] Please refer to Figure 3 This is a schematic block diagram of a text processing method provided in this application. The block diagram can also be... Figure 2 The diagram shows a schematic block diagram of the regularization system. The processing procedure for this schematic block diagram is described below:
[0082] The network is generated by inputting the labels corresponding to the non-standard text as input parameters. These labels are used to represent the non-standard categories corresponding to the non-standard text.
[0083] The parameter generation network determines the updated values of the model parameters of the Transformer model based on the labels, and transmits the updated values to the Transformer model to update the model parameters.
[0084] The Transformer model is updated after non-standard text input and processed to obtain the first vector output by the encoder and at least one second vector output by the decoder in the Transformer model.
[0085] The first vector and at least one second vector are input to a pointer generation network to process the standard text corresponding to the non-standard text.
[0086] Understandable, Figure 3 The specific descriptions of the embodiments can be found in the descriptions of the foregoing method embodiments, and will not be repeated here.
[0087] In this embodiment of the invention, non-standard text is obtained, and the non-standard category corresponding to the non-standard text is determined. The updated value of the model parameters of the regularization model is determined according to the non-standard category, and the updated value is used to update the value of the model parameters of the regularization model. Thus, the updated regularization model is used to perform regularization processing on the non-standard text. In this application, the model parameters of the regularization model are not fixed, but dynamically adjusted according to the non-standard category corresponding to the non-standard text. This allows the updated regularization model to perform regularization processing on the non-standard text more accurately, thereby improving the accuracy of the obtained standard text corresponding to the non-standard text.
[0088] Please see Figure 4 This is a schematic diagram of the structure of a text processing device provided in an embodiment of the present invention. Figure 4 As shown, the text processing apparatus of this embodiment may include:
[0089] The acquisition unit 10 is used to acquire non-standard text and determine the non-standard category corresponding to the non-standard text;
[0090] The first determining unit 11 is used to determine the updated values of the model parameters of the regularized model based on the non-standard category.
[0091] The model update unit 12 is used to update the values of the model parameters of the regularization model using the update value, and to process the non-standard text using the updated regularization model to obtain at least one vector.
[0092] The second determining unit 13 is used to determine the standard text corresponding to the non-standard text based on the at least one vector.
[0093] In one possible implementation, the at least one vector includes a first vector and at least one second vector, wherein the second vector corresponds to a character position in the standard text, and the second vector is used to determine the character at the character position;
[0094] The first vector is the vector output by the encoder in the updated regularization model, and the second vector is the vector output by the decoder in the updated regularization model.
[0095] In one possible implementation, the second determining unit 13 is specifically used for:
[0096] The second vector is processed by the linear layer and softmax layer in the neural network to obtain the third vector corresponding to the second vector. The third vector is used to represent the probability that the character at the character position corresponding to the second vector is a preset character in the preset vocabulary.
[0097] Based on the first vector and the second vector, a fourth vector corresponding to the second vector is obtained. The fourth vector is used to represent the importance of the characters contained in the non-standard text to the characters at the predicted character positions corresponding to the second vector.
[0098] The standard text corresponding to the non-standard text is determined based on at least one third vector corresponding to the at least one second vector and at least one fourth vector corresponding to the at least one second vector, wherein one second vector corresponds to one third vector and one second vector corresponds to one fourth vector.
[0099] In one possible implementation, the second determining unit 13 is specifically used for:
[0100] For each of the at least one second vector, obtain the weight value corresponding to the second vector;
[0101] Based on the third vector corresponding to the second vector, the fourth vector corresponding to the second vector, and the weight value corresponding to the second vector, the final character at the character position corresponding to the second vector is obtained;
[0102] Based on the final character corresponding to each of the at least one second vector, construct the standard text corresponding to the non-standard text.
[0103] In one possible implementation, the second determining unit 13 is specifically used for:
[0104] Obtain the feature vectors corresponding to the characters contained in the non-standard text;
[0105] Based on the feature vector and the second vector, the weight value corresponding to the second vector is obtained.
[0106] In one possible implementation, the second determining unit 13 is specifically used for:
[0107] Based on the feature vector, the context vector corresponding to the non-standard text is obtained;
[0108] Based on the context vector and the second vector, the weight value corresponding to the second vector is obtained.
[0109] In one possible implementation, the regularization model is a Transformer model.
[0110] In this embodiment of the invention, non-standard text is obtained, and the non-standard category corresponding to the non-standard text is determined. The updated value of the model parameters of the regularization model is determined according to the non-standard category, and the updated value is used to update the value of the model parameters of the regularization model. Thus, the updated regularization model is used to perform regularization processing on the non-standard text. In this application, the model parameters of the regularization model are not fixed, but dynamically adjusted according to the non-standard category corresponding to the non-standard text. This allows the updated regularization model to perform regularization processing on the non-standard text more accurately, thereby improving the accuracy of the obtained standard text corresponding to the non-standard text.
[0111] Please refer to Figure 5 This is a schematic diagram of another text processing device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the text processing device 1000 may include: at least one processor 1001, such as a CPU, at least one communication interface 1003, a memory 1004, and at least one communication bus 1002. The communication bus 1002 is used to implement communication between these components. The communication interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1004 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 5 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and program instructions.
[0112] exist Figure 5 In the text processing device 1000 shown, the processor 1001 can be used to load program instructions stored in the memory 1004 and specifically perform the following operations:
[0113] Obtain non-standard text and determine the non-standard category corresponding to the non-standard text;
[0114] Based on the non-standard category, determine the updated values of the model parameters for the regularized model;
[0115] The updated values are used to update the values of the model parameters of the regularization model, and the updated regularization model is used to process the non-standard text to obtain at least one vector.
[0116] Based on the at least one vector, determine the standard text corresponding to the non-standard text.
[0117] Optionally, the at least one vector contains at least two vectors, including a first vector and at least one second vector, wherein a second vector corresponds to a character position in the standard text, and the second vector is used to determine the character at the character position;
[0118] The first vector is the vector output by the encoder in the updated regularization model, and the second vector is the vector output by the decoder in the updated regularization model.
[0119] It should be noted that the specific execution process can be found in [link to relevant documentation]. Figure 1 The specific details of the method embodiments shown are not elaborated here.
[0120] For specific execution steps, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0121] This invention also provides a computer storage medium that can store multiple instructions adapted for loading and execution by a processor as described above. Figure 1 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 1 The specific details of the illustrated embodiments will not be elaborated here.
[0122] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
Claims
1. A text processing method characterized by, The method comprises: obtaining non-standard text and determining a non-standard category corresponding to the non-standard text; determining an updated value of a model parameter of a regularization model according to the non-standard category; updating the value of the model parameter of the regularization model by using the updated value, and processing the non-standard text by using the updated regularization model to obtain at least one vector, the at least one vector comprising a first vector and at least one second vector, wherein one second vector corresponds to a character position in the standard text, and the second vector is used to determine a character at the character position in the standard text; the first vector is a vector output by an encoder in the updated regularization model, and the second vector is a vector output by a decoder in the updated regularization model; processing the second vector by using a linear layer and a softmax layer in a neural network to obtain a third vector corresponding to the second vector, the third vector being used to represent probabilities of each preset character in a preset character table for a character at the character position corresponding to the second vector; obtaining a fourth vector corresponding to the second vector according to the first vector and the second vector, the fourth vector being used to represent an importance of a character included in the non-standard text for predicting a character at the character position corresponding to the second vector; obtaining a weight value corresponding to each second vector in the at least one second vector; obtaining a final character of the character position corresponding to the second vector according to the third vector corresponding to the second vector, the fourth vector corresponding to the second vector, and the weight value corresponding to the second vector; constructing a standard text corresponding to the non-standard text according to the final character corresponding to each second vector in the at least one second vector; wherein one second vector corresponds to one third vector, and one second vector corresponds to one fourth vector.
2. The method of claim 1, wherein, The method comprises: obtaining a feature vector corresponding to a character included in the non-standard text; obtaining the weight value corresponding to the second vector according to the feature vector and the second vector.
3. The method of claim 2, wherein, The method comprises: obtaining a context vector corresponding to the non-standard text according to the feature vector; obtaining the weight value corresponding to the second vector according to the context vector and the second vector.
4. The method according to any one of claims 1 to 3, characterized in that, The regularization model is a Transformer model.
5. A text processing apparatus characterized by comprising: The device comprises units for performing the method according to any one of claims 1-4, and the device comprises: an obtaining unit configured to obtain non-standard text and determine a non-standard category corresponding to the non-standard text; a first determining unit configured to determine an updated value of a model parameter of a regularization model according to the non-standard category; a model updating unit configured to update the value of the model parameter of the regularization model by using the updated value, and process the non-standard text by using the updated regularization model to obtain at least one vector; A second determining unit is configured to determine the standard text corresponding to the non-standard text according to the at least one vector.
6. A text processing apparatus characterized by comprising: The apparatus comprises a processor, a memory and a communication interface, which are connected with each other. The communication interface is configured to receive and send data. The memory is configured to store program code. The processor is configured to invoke the program code and execute the method according to any one of claims 1 to 4.
7. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Text organization method and device, storage medium and electronic equipment
CN107562718A