Model training method, text error correction method, device, equipment, medium and product

Abstract

The invention discloses a model training method and device, a text error correction method and device, equipment, a medium and a product, and is applied to the technical field of artificial intelligence. The method comprises the following steps: firstly, carrying out word segmentation on a training text, and mapping to high-dimensional embedded representation so as to obtain target word vector representation through conversion, namely, the process of BERT embed; according to the scheme, the target word vector representation is input into the initial text error correction model for reinforcement learning, the method combines BERT embdingings and the aggregation operation of the reinforcement learning unit, training is completed only through few texts, the semantic relation can be captured, error correction of grammar, punctuation marks and words can be achieved, and meanwhile the text representation and classification capacity is improved. The method can adapt to an official document error correction scene, and has a good error correction effect.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a model training method, a text error correction method, a device, a equipment, a medium and a product. Background Art

[0002] Official documents are official documents used for notification, reporting and record keeping in official occasions such as government agencies, enterprises and academic institutions. They contain important information such as policy decisions, business guidelines and legal provisions, which have a direct impact on the reader's understanding and behavior. Any errors in official documents (content errors and format errors, etc.) may lead to misunderstandings and operational errors. Therefore, the accuracy and rigor of official documents are crucial.

[0003] The current text error correction method mainly uses machine learning to predict and proofread text. This method requires a large amount of training corpus, but the official document training corpus is relatively small. The trained model lacks the ability to generate and identify professional descriptions in specific fields such as official documents. It can be seen that the traditional text error correction model cannot adapt to the official document text error correction scenario, and the error correction effect is poor. Summary of the invention

[0004] The embodiments of the present application provide a model training method, a text error correction method, an apparatus, a device, a medium and a product, which can adapt to the official document error correction scenario and have a good error correction effect.

[0005] On the one hand, an embodiment of the present application provides a method for training a text error correction model, comprising:

[0006] Obtaining the training text with errors and the corresponding correct reference text;

[0007] Segmenting the training text and mapping it to a high-dimensional embedding representation to convert it into a target word vector representation;

[0008] Inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network;

[0009] Performing the first target action on the target word vector representation to update the target word vector representation;

[0010] Determining a reward value for executing the first target action based on the updated target word vector representation and the reference text;

[0011] In the case where the reward value does not satisfy the iteration stop condition, the policy network is updated based on the reward value, and the step of inputting the target word vector representation into the pre-built initial text error correction model is returned to determine the first target action for the target word vector representation based on the policy network;

[0012] When the reward value satisfies the iteration stop condition, the initial text error correction model is determined as the text error correction model.

[0013] On the other hand, the step of inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network includes:

[0014] Inputting the target word vector representation into the initial text error correction model;

[0015] Determining an action probability distribution based on the target word vector representation and the weight matrix of the policy network;

[0016] Based on the action probability distribution, the first target action is selected from the action space.

[0017] On the other hand, selecting the first target action from the action space based on the action probability distribution includes:

[0018] Determining the selection probability of each action in the action space based on the action probability distribution;

[0019] Based on the selection probability, the first target action is determined from each action in the action space.

[0020] On the other hand, when the reward value does not satisfy the iteration stop condition, updating the policy network based on the reward value includes:

[0021] When the reward value does not satisfy the iteration stop condition, updating the action value network of the initial text error correction model based on the reward value;

[0022] Based on the updated action-value network, a weight matrix of the policy network is determined.

[0023] On the other hand, determining the reward value for executing the first target action based on the updated target word vector representation and the reference text includes:

[0024] Based on the reference text, determining the vocabulary accuracy and sentence connection accuracy of the updated target word vector representation;

[0025] The reward value is determined based on the vocabulary accuracy and the sentence connection accuracy.

[0026] On the other hand, determining the reward value based on the vocabulary accuracy and the sentence connection accuracy includes:

[0027] Based on the cross entropy loss function, determining a first cross entropy loss value corresponding to the vocabulary accuracy, and determining a second cross entropy loss value corresponding to the sentence connection accuracy; the first cross entropy loss value is positively correlated with the vocabulary accuracy, and the second cross entropy loss value is positively correlated with the sentence connection accuracy;

[0028] The reward value is determined based on the first cross entropy loss value and the second cross entropy loss value.

[0029] On the other hand, after segmenting the training text and mapping it to a high-dimensional embedding representation to convert it into a target word vector representation, the method further includes:

[0030] Selecting a target vocabulary from the target word vector representation at a first preset ratio;

[0031] The following operations are performed on the target vocabulary representation:

[0032] Based on a second preset ratio, masking the words of the target vocabulary;

[0033] Based on a third preset ratio, the vocabulary of the target vocabulary is replaced; the first preset ratio is less than a first ratio threshold, the second preset ratio is less than a second ratio threshold, and the third preset ratio is less than a third ratio threshold.

[0034] On the other hand, when the reward value satisfies the iteration stop condition, after determining the initial text error correction model as the text error correction model, the method further includes:

[0035] Obtain the training document text for the corresponding scenario;

[0036] The text error correction model is fine-tuned based on the training document text.

[0037] On the other hand, an embodiment of the present application provides a text error correction method, which is applied to a text error correction model, wherein the text error correction model is trained by the training method of the text error correction model as described above, and the text error correction method includes:

[0038] Receiving the original text to be corrected;

[0039] Inputting the original text into the text error correction model to determine a second target action for the original text based on a policy network;

[0040] Performing the second target action on the original text to obtain a target text;

[0041] Based on the target text, error correction is performed on the original text.

[0042] On the other hand, the embodiment of the present application provides a training device for a text error correction model, including:

[0043] An acquisition module is used to acquire the training text with errors and the corresponding correct reference text;

[0044] A conversion module, used to segment the training text and map it to a high-dimensional embedding representation to obtain a target word vector representation;

[0045] An input module, used for inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network;

[0046] A first execution module, configured to execute the first target action on the target word vector representation to update the target word vector representation;

[0047] A first determination module, configured to determine a reward value for executing the first target action based on the updated target word vector representation and the reference text;

[0048] An updating module, configured to update the policy network based on the reward value when the reward value does not satisfy the iteration stop condition, and return to the step of inputting the target word vector representation into the pre-built initial text error correction model to determine a first target action for the target word vector representation based on the policy network;

[0049] The second determination module is used to determine the initial text error correction model as the text error correction model when the reward value satisfies the iteration stop condition.

[0050] On the other hand, an embodiment of the present application provides a text error correction device, which is applied to a text error correction model, wherein the text error correction model is trained by the training method of the text error correction model as described above, and the text error correction device comprises:

[0051] A receiving module, used for receiving the original text to be corrected;

[0052] A third determination module, configured to input the original text into the text error correction model to determine a second target action for the original text;

[0053] A second execution module, configured to execute the second target action on the original text to obtain a target text;

[0054] The error correction module is used to correct the original text based on the target text.

[0055] In another aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0056] When the processor executes the computer program instructions, the training method of the text error correction model and the text error correction method as described above are implemented.

[0057] On the other hand, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the training method of the text error correction model and the text error correction method as described above are implemented.

[0058] On the other hand, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the training method of the text error correction model and the text error correction method as described above.

[0059] A training method for a text error correction model provided in an embodiment of the present application first performs word segmentation on the training text and maps it to a high-dimensional embedding representation to convert the target word vector representation, i.e., the process of BERT embedding. Then, the target word vector representation is input into the initial text error correction model for reinforcement learning, and the first target action is first determined; then, the first target action is performed on the target word vector representation to update the target word vector representation, and the reward value for performing the first target action is determined. In the case where the reward value does not meet the iteration stop condition, the policy network is updated based on the reward value, and the step of determining the first target action is returned. In the case where the reward value meets the iteration stop condition, the initial text error correction model is determined as the text error correction model. The scheme proposed in the embodiment of the present application combines the aggregation operation of BERT embeddings and reinforcement learning units, requires only a small amount of text to complete the training, and can capture semantic relationships, realize grammar, punctuation, and word correction, and improve text representation and classification capabilities. It can adapt to the official document text error correction scenario and has a good error correction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0061] Figure 1 A flowchart of a method for training a text error correction model provided by an embodiment of the present application is shown;

[0062] Figure 2 A training flow chart of a text error correction model provided by an embodiment of the present application is shown;

[0063] Figure 3 A flowchart of a text error correction method provided by an embodiment of the present application is shown;

[0064] Figure 4 A schematic diagram of the structure of a training device for a text error correction model provided in an embodiment of the present application is shown;

[0065] Figure 5 A schematic diagram of the structure of a text error correction device provided in an embodiment of the present application is shown;

[0066] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0067] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0068] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "include..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0069] The accuracy and rigor of official documents are reflected in several aspects: first, the accuracy of the content, ensuring that the information is conveyed clearly and correctly; second, the standardization of the format, including structure, wording and punctuation, to ensure the formality and professionalism of the document. Therefore, accurate error correction of official documents is not only necessary to improve work efficiency and document quality, but also a necessary means to prevent possible misunderstandings and wrong decisions. Especially in automatically generated official documents, due to the limitations of the development of artificial intelligence, errors in wording and punctuation are particularly common and need to be corrected through effective technical means. When traditional text correction solutions are applied to the field of official documents, they may face problems such as lack of training data, expression sensitivity, and contextual coherence.

[0070] The document error correction model provided in the embodiment of the present application is applicable to the field of natural language processing, and can be used for the construction of an automated error correction system and the development of document processing software, thereby improving the efficiency and quality of document processing. It is applicable to the error correction of various types of documents, including but not limited to reports, contracts, memoranda, and notices. It can be implemented by a computer, server, or other computing device, and its software module can be stored on a storage medium of the computing device, and interact with the user through corresponding input and output interfaces.

[0071] In order to solve the problems of the traditional solution, the embodiment of the present application provides a model training method, a text error correction method, a device, a equipment, a medium and a product. The following first introduces the training method of the text error correction model provided by the embodiment of the present application. Figure 1 A flow chart of a method for training a text error correction model provided by an embodiment of the present application is shown; Figure 1 As shown, the method comprises the following steps:

[0072] S101: Acquire erroneous training text and the corresponding correct reference text.

[0073] Before training, it is necessary to obtain training data, namely training text and reference text. The training text contains grammatical, word, and punctuation errors. The reference text is the correct text corresponding to the training text. There is no limit on the number of training texts and reference texts, and there is no limit on the specific source. They can be obtained from the historical official document library.

[0074] S102: Segment the training text into words and map them to high-dimensional embedding representations to convert them into target word vector representations.

[0075] After obtaining the training text, word segmentation and word embedding operations are performed to segment the training text with errors, and the word segmentation results are converted into word vector representations so as to be input into the Bidirectional Encoder Representations from Transformers (BERT) model (i.e., the initial text error correction model) for processing.

[0076] Assuming that the input training text is "In order to further improve the work efficiency of each department", after word segmentation, the token sequence is obtained: {[CLS], in order to, further, improve, the, work, efficiency, of, each department, [SEP]}. Each word will be mapped to a high-dimensional embedding representation, which includes word vectors (Token embeddings), sentence embeddings (Segment embeddings) and positional embeddings (Positional embeddings). Segment embeddings are used to mark two different sentences, and positional embeddings are added to each token embedding to indicate its position in the sentence, which is learned by the model. Assuming the embedding dimension is 768, the word vector representation matrix of the input model is:

[0077] E=[E [CLS] ,E 为了 ,E 进一步 ,E 提高 ,E 各部门 ,E 的 ,E 工作 ,E 效率 ,E [SEP] ];

[0078] For each Token, calculate its Token Embeddings, Segment Embeddings, and Position Embeddings, and add them together to get the word vector representation.

[0079] S103: Input the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network.

[0080] This application uses reinforcement learning to build an update strategy for the BERT model.

[0081] The word vector representation matrix E is used as the initial state s0 of the model, that is:

[0082] s0=E;

[0083] According to the current status t (the first is the initial state s0) and the policy network π(a|s t )Select an action a t , the selected action a t Determined as the attention weight, the probability distribution of the policy network is:

[0084] π(a|s t )=softmax(w π s t +b π );

[0085] Among them, w π is the trainable weight matrix, b π is the bias vector, π(a|s t ) is the state s t The probability distribution of each possible action a; Softmax is a normalized exponential function.

[0086] Based on the above probability distribution, the first target action represented by the target word vector is determined. That is, each action in the action space has a corresponding selection probability, and the first target action is selected based on the corresponding selection probability.

[0087] S104: Perform a first target action on the target word vector representation to update the target word vector representation.

[0088] The first target action is an error correction action for the target word vector representation, such as replacing vocabulary and punctuation, changing the order of sentences, etc. After the first target action is executed, an updated target word vector representation is obtained.

[0089] S105: Determine a reward value for executing the first target action based on the updated target word vector representation and the reference text.

[0090] As mentioned above, if the reference text is the correct text corresponding to the training text, the updated target word vector representation can be compared with the reference text to determine the reward value for executing the first target action. When making the comparison, the reference text can be converted into the corresponding word vector representation in the same way in advance.

[0091] S106: Determine whether the reward value satisfies the iteration stop condition; if not, proceed to S107; if so, proceed to S108.

[0092] The iteration stop condition can be set in advance, for example, the reward value tends to remain unchanged, indicating that the training text has been corrected.

[0093] S107: Update the policy network based on the reward value and return to S103.

[0094] When the reward value does not meet the iteration stop condition, the policy network is updated based on the obtained reward value, that is, the probability distribution of the above policy network is updated, and the process returns to S103 to continue to determine a new first target action, and execute the first target action for the updated target word vector representation.

[0095] S108: Determine the initial text error correction model as the text error correction model.

[0096] When the reward value does not meet the iteration stopping condition, it indicates that the initial text correction model has learned enough features, so the updated initial text correction model is determined as the final text correction model. Based on this text correction model, accurate error correction of official documents can be achieved.

[0097] In view of the determination of traditional solutions and the current demand for official document generation, the embodiment of the present application proposes a document error correction training method based on BERT and reinforcement learning. Among them, BERT segments and encodes the text content of the official document and obtains word vectors. The reinforcement learning unit explores the vocabulary based on its reward feedback mechanism, corrects the word vector selection through the feedback reward obtained for the selected action after the state transfer, and continues to train until the correct "cloze"; punctuation marks are classified and encoded. Unlike the error correction of the text part, the punctuation error correction uses the complete sentence as the input part of BERT, and based on the reinforcement learning unit, the punctuation prediction of the sentence approximates the correct punctuation marks. Among them, the classification labels of punctuation marks are: none, comma, period, question mark, exclamation mark, ellipsis and colon.

[0098] The embodiment of the present application uses the method of temporal difference learning, using the current state s t , the action taken a t , after executing the action, according to the reward function R(s t ,a t ) The reward r t+1 and the subsequent new state s t+1 The four-tuple composed of s is used to update the samples generated by each state in a single step. t Represents the current state, a tis the action chosen in this state, r t+1 is the reward after the action is executed, s t+1 It is an action t After the impact on the environment t The new state of the transformation estimates the action-value (Q-value) of the same (s, a) under different sampledata, as shown in the following formula:

[0099] Q(s t ,a t )←Q(s t ,a t )+α[r t→t+1 +γmaxQ(s t+1 ,a)-Q(s t ,a t )];

[0100] Among them, α and γ are learning rate and discount coefficient, α∈[0,1], maxQ(s t+1 ,a) means in state s t The maximum Q value is selected from all actions under t ←s t+1 , repeat the process until s t Arrive at the final state and update the parameters of the action-value network through the Bellman equation.

[0101] In the above scheme, the corresponding feature vector of each word is obtained through the reward and feedback mechanism and reinforcement learning training is performed. It is no longer necessary to calculate the query matrix, key matrix and value matrix to obtain the attention relationship of each word.

[0102] The text correction model provided in the embodiment of the present application realizes intelligent correction functions for errors such as missing words, redundancy, disorder, improper collocation, and improper use of punctuation marks in official document correction scenarios such as notices and regulations.

[0103] By introducing reinforcement learning units, we can use the reward and punishment mechanism of trial and error feedback to improve the training method of the error correction model, improve the robustness and fine-grained error correction ability of the model, and feed the word vector representation constructed by BERT into the reinforcement learning network to train the approximate correct words and punctuation marks, integrate multi-dimensional semantic feature information, and enable the model to improve its semantic understanding and text and punctuation prediction capabilities.

[0104] In addition, by combining BERT embeddings with reinforcement learning methods and jointly training, BERT's attention mechanism is replaced, the model's trial-and-error tolerance is enhanced, and the degree of approximation between the convergence value and the theoretical value of the optimal document content is effectively improved. It can perform semantic understanding and feature learning of text data from multiple dimensions such as text and punctuation, improving the effect of the error correction model.

[0105] A training method for a text error correction model provided in an embodiment of the present application first performs word segmentation on the training text and maps it to a high-dimensional embedding representation to convert the target word vector representation, i.e., the process of BERT embeddings. Then, the target word vector representation is input into the initial text error correction model for reinforcement learning, and the first target action is first determined; then, the first target action is performed on the target word vector representation to update the target word vector representation, and the reward value for performing the first target action is determined. In the case where the reward value does not meet the iteration stop condition, the policy network is updated based on the reward value, and the above step of determining the first target action is returned. In the case where the reward value meets the iteration stop condition, the initial text error correction model is determined as the text error correction model. The scheme proposed in the embodiment of the present application combines the aggregation operation of BERT embeddings and reinforcement learning units, requires only a small amount of text to complete the training, and can capture semantic relationships, realize grammar, punctuation and word correction, and improve text representation and classification capabilities. It can adapt to the official document text error correction scenario and has a good error correction effect.

[0106] During the training of the text error correction model, the performance of the reinforcement learning training process can be further optimized by dynamically adjusting the experience replay space size parameter. This parameter gradually decreases as the training progresses, forgetting unnecessary long-term experience. As the algorithm gradually approaches the optimal value, recent experience is considered more important, reducing the need to explore early experience.

[0107] Specifically, the size parameter of the experience replay space decreases monotonically with the progress of training, and is adaptively adjusted according to the training cycle and the convergence of the results to improve the utilization of the experience data. This adjustment can be achieved through the function E r ←a e *E p +b e To indicate that E p is a constant parameter, E r is the experience playback space size parameter, a e and b e is the adjustment coefficient, which means that as the training progress increases, the size of the experience replay space decreases monotonically.

[0108] In addition, in order to adapt to changes in training conditions, the relationship between the size of the experience replay space and the reward can be expressed by the function E r ←a r *log(b r *E p +c r )+d r To represent, where a r 、b r , c and dr The same is true for the correlation coefficient to ensure dynamic adaptive characteristics. When epoch (number of training times) is 1, assuming E p =0.9, E p ∈[0,1], the experience playback space can accommodate more experience in each round of training, which is convenient for updating the input matrix of the word embedding vector in the document from the initialization value. When the epoch is 1000, assuming E p =0.5, the word and context logic at this position are basically clear, the experience of previous exploration is forgotten and the nearby experience is focused on to enhance the accuracy of relationship mapping.

[0109] Long-term storage of useless experience may lead to negative feedback and affect learning efficiency. As the training cycle progresses, the algorithm performance gradually stabilizes and the negative feedback phenomenon gradually decreases. Dynamic experience playback parameters, through adaptive means, gradually forget long-term experience in the training process, and focus on short-term experience to improve the accuracy of document content prediction.

[0110] Figure 2 FIG. 1 shows a training flow chart of a text error correction model provided by an embodiment of the present application. Figure 2 As shown, DQN (Deep Q-Network) technology is used, and the DQN agent 201 selects actions according to the current policy network. During the training process, in order to balance exploration and utilization and prevent the current policy network from falling into a vicious cycle, a greedy strategy is used: a random action is selected with probability ε, and the optimal action under the current policy is selected with probability 1-ε. This ensures that the agent 201 not only repeats the existing optimal action, but also explores new actions to discover a potentially better strategy.

[0111] DQN uses a deep neural network (estimation network 202) to approximate the Q-value function Q(s,a), where s is the state and a is the action. For each given state, the neural network outputs the Q-values ​​of each possible action, which represent the expected cumulative rewards that can be obtained by performing these actions.

[0112] When the agent 201 performs an action, the environment 204 will feedback a new state s' and reward R. DQN uses the experience replay space to store multiple state transition samples (s, a, R, s'), that is, a four-tuple. In order to update the parameters of the estimation network 202, DQN calculates the target Q value Q target :

[0113] In order to improve the stability of training, DQN introduces an independent target network 203, whose parameters are updated synchronously from the estimation network 202 every certain number of steps. In each training, the target Q value is calculated using the parameters of the target network 203 instead of the latest parameters of the estimation network 202. This approach can reduce oscillation and instability during training.

[0114] The whole process is repeated continuously, and the agent 201 continuously interacts with the environment 204, predicts the action value of the current state through the estimation network 202, and optimizes the parameters of the estimation network 202 through experience replay and the target network 203, gradually approaching the optimal strategy.

[0115] In the above embodiment, after the target word vector representation is input into the pre-built initial text error correction model, it is not limited how to determine the first target action based on the policy network. In a specific implementation, the initial text error correction model includes an action value network, that is, for a text, different actions have different action values, and the first target action can be determined based on the action value. This embodiment provides a specific implementation scheme, including:

[0116] The target word vector representation is input into the initial text error correction model. Then, the action probability distribution is determined based on the target word vector representation and the weight matrix of the policy network. Finally, the first target action is selected from the action space based on the action probability distribution.

[0117] In practical applications, the selection probability of each action in the action space can be determined based on the action probability distribution; then, based on the selection probability, the first target action can be determined from each action in the action space. That is, at a certain stage of the training model, there are multiple actions that may be selected, and different actions have different selection probabilities. The selection probability can be determined based on the action value. The higher the action value of the corresponding action, the higher the probability of selection.

[0118] The embodiment of the present application determines the action probability distribution based on the weight matrix, and then selects the action from the action space based on the selection probabilities corresponding to different actions in the action probability distribution, which can train a model with better error correction effect and improve the accuracy of model error correction.

[0119] After obtaining the reward value, the text error correction model needs to be updated. The embodiment of the present application provides a specific updating method. When the reward value does not meet the iteration stop condition, the action value network of the initial text error correction model is updated based on the reward value; based on the updated action value network, the weight matrix of the strategy network is determined. This embodiment updates the action value through the reward value, so that the action value corresponding to the better action is higher.

[0120] The above does not limit how to determine the reward value. This embodiment provides a specific implementation scheme. First, based on the reference text, the vocabulary accuracy and sentence connection accuracy of the updated target word vector representation are determined; then, based on the vocabulary accuracy and sentence connection accuracy, the reward value is determined. The embodiment of the present application can accurately determine the corresponding reward value based on the vocabulary accuracy and sentence connection accuracy, so that the policy network can be effectively updated.

[0121] In specific implementation, based on the cross-entropy loss function, a first cross-entropy loss value corresponding to the vocabulary accuracy and a second cross-entropy loss value corresponding to the sentence connection accuracy can be determined; then, based on the first cross-entropy loss value and the second cross-entropy loss value, a reward value can be determined.

[0122] In this embodiment, the first cross entropy loss value is positively correlated with the vocabulary accuracy, and the second cross entropy loss value is positively correlated with the sentence connection accuracy.

[0123] For example, to perform action a t After that, update the state to get s t+1 , and feedback rewards based on the execution of the action. In the Masked Language Model (MLM) task, the reward function is:

[0124]

[0125] Among them, P pred_mlm is the probability distribution predicted by the model, P true_mlm is the actual vocabulary distribution, which is a one-hot vector, is the cross entropy loss function. If the model predicts the correct word, the cross entropy loss is low and the reward is high; if it predicts the wrong word, the reward is low.

[0126] In the Next Sentence Prediction (NSP) task, assuming that the input is two sentences "In order to further improve the work efficiency of each department" and "Fully mobilize the work enthusiasm of all employees", the model needs to predict whether these two sentences are continuous. The reward function of the NSP task is:

[0127]

[0128] Among them, P pred_nsp is the probability distribution predicted by the model, P true_nsp is the actual label, which is a one-hot vector. Then the comprehensive reward function is:

[0129] R(s t ,a t )=αR MLM(s t ,a t )+βR NSP (s t ,a t );

[0130] Among them, α and β are weight coefficients, and the values ​​of α and β are dynamically adjusted according to the performance of the model on different tasks.

[0131] The embodiment of the present application provides a specific reward value calculation method, which can obtain an accurate reward value by calculating the cross entropy loss value through the cross entropy loss function.

[0132] Compared with common natural language texts, official documents usually have less lexical ambiguity and contextual changes. A smaller mask ratio allows the model to focus more on the semantics and structure of the entire sentence, rather than focusing too much on local lexical errors. Official documents usually have rigorous language structures and professional terminology, and the errors in them may involve more subtle issues in grammar and word usage. Therefore, appropriately reducing the mask ratio helps BERT better capture the overall context and structure of the text and correct the expression of the official document narrative.

[0133] Therefore, after the training text is segmented and mapped to a high-dimensional embedding representation to convert the target word vector representation, the embodiment of the present application further includes:

[0134] Selecting a target word from the target word vector representation at a first predetermined ratio;

[0135] Perform the following operations on the target vocabulary representation:

[0136] masking the words of the target vocabulary based on a second preset ratio;

[0137] Based on the third preset ratio, the vocabulary of the target vocabulary is replaced. The first preset ratio is less than the first ratio threshold, the second preset ratio is less than the second ratio threshold, and the third preset ratio is less than the third ratio threshold. Here, the specific values ​​of the first ratio threshold, the second ratio threshold, and the third ratio threshold can be set according to actual conditions.

[0138] During the pre-training process, the BERT model randomly selects 10% of the tokens in the input target word vector, and masks 80% of the selected tokens using [MASK], randomly replaces 10% of the tokens with another word, and keeps the other 10% unchanged, and tries to predict the words at the masked position through the masked language model task. Assuming that one of the tokens that need to be masked and replaced in the full text falls into the matrix E, the input matrix after masking and replacement may be expressed as:

[0139] E=[E [CLS],E 为了 ,E 进一步 ,E [MASK] ,E 各部门 ,E 的 ,E 工作 ,E 速率 ,E [SEP] ];

[0140] The embodiment of the present application adopts a reduced mask ratio, so that BERT can better capture the overall context and structure in the official document field, so that the model can focus on the expression of clear official document types such as "notice" or "system", and improve the accuracy of grammar, vocabulary, punctuation and other error corrections.

[0141] In practical applications, after the text error correction model is determined, it may be used in different official document scenarios, and the parameters of the text error correction model can be fine-tuned according to the specific scenarios to achieve better error correction effects. That is, first obtain the training official document text for the corresponding scenario; then fine-tune the text error correction model based on the training official document text.

[0142] The embodiment of the present application is fine-tuned on the basis of the trained BERT model, and the official document corpus in a specific scenario is fed into the model for further training to adapt to the task of official document error correction. By collecting the official document data disclosed by enterprises and the Internet as input data and providing it to the model, the loss function L(Y, f(x)) is calculated, where Y and f(x) are the true label and model output respectively, assuming "to further improve the work efficiency of each department" and "to further reduce the work rate of each department", and the model parameters are updated through the back-propagation algorithm adaptive motion estimation algorithm (Adam).

[0143] The method provided in the embodiment of the present application fine-tunes the BERT model based on a domain-specific data set to adapt to the language characteristics of official documents in different fields, improve the professionalism and applicability of error correction, and thus better apply to different types of official document error correction tasks.

[0144] In order to solve the above technical problems, the embodiment of the present application further provides a text error correction method, which is applied to a text error correction model. The text error correction model is trained by the training method of the text error correction model in the above embodiment. Figure 3 FIG. 1 is a flow chart of a text error correction method provided by an embodiment of the present application. Figure 3 As shown, the text error correction method includes the following steps:

[0145] S301: receiving an original text to be corrected.

[0146] S302: Inputting the original text into the text error correction model to determine a second target action for the original text based on the policy network.

[0147] It should be noted that when determining the second target action here, the second target action with the highest action value performed on the original text can be determined based on the action value network of the text error correction model.

[0148] S303: Perform a second target action on the original text to obtain a target text.

[0149] S304: Correct the original text based on the target text.

[0150] The text correction method provided in the embodiment of the present application corresponds to the training method of the text correction model in the above embodiment, so both have the same beneficial effects and will not be repeated here.

[0151] In order to solve the above technical problems, the present application also provides a training device for a text error correction model. Figure 4 The structure diagram of the training device of the text error correction model provided by the embodiment of the present application is shown. Figure 4 As shown, the device includes the following modules:

[0152] An acquisition module 401 is used to acquire a training text with errors and a corresponding correct reference text;

[0153] A conversion module 402 is used to segment the training text and map it to a high-dimensional embedding representation to obtain a target word vector representation;

[0154] An input module 403 is used to input the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network;

[0155] A first execution module 404 is used to execute a first target action on the target word vector representation to update the target word vector representation;

[0156] A first determination module 405 is used to determine a reward value for performing a first target action based on the updated target word vector representation and the reference text;

[0157] An updating module 406, configured to update the policy network based on the reward value and trigger the input module 403 when the reward value does not satisfy the iteration stop condition;

[0158] The second determination module 407 is used to determine the initial text error correction model as the text error correction model when the reward value meets the iteration stop condition.

[0159] In some embodiments, the input module 403 is specifically configured to:

[0160] Input the target word vector representation into the initial text error correction model;

[0161] Determine the action probability distribution based on the target word vector representation and the weight matrix of the policy network;

[0162] Based on the action probability distribution, the first target action is selected from the action space.

[0163] In some embodiments, the input module 403 is specifically configured to:

[0164] Based on the action probability distribution, determine the selection probability of each action in the action space;

[0165] Based on the selection probability, the first target action is determined from each action in the action space.

[0166] In some embodiments, the update module 406 is specifically configured to:

[0167] When the reward value does not meet the iteration stop condition, the action value network of the initial text error correction model is updated based on the reward value;

[0168] Based on the updated action-value network, determine the weight matrix of the policy network.

[0169] In some embodiments, the first determining module 405 is specifically configured to:

[0170] Based on the reference text, determine the vocabulary accuracy and sentence connection accuracy of the updated target word vector representation;

[0171] The reward value is determined based on the vocabulary accuracy and sentence connection accuracy.

[0172] In some embodiments, the first determining module 405 is specifically configured to:

[0173] Based on the cross entropy loss function, a first cross entropy loss value corresponding to the vocabulary accuracy is determined, and a second cross entropy loss value corresponding to the sentence connection accuracy is determined; the first cross entropy loss value is positively correlated with the vocabulary accuracy, and the second cross entropy loss value is positively correlated with the sentence connection accuracy;

[0174] A reward value is determined based on the first cross entropy loss value and the second cross entropy loss value.

[0175] In some embodiments, the training device of the text error correction model further includes:

[0176] A selection module, configured to select a target word from the target word vector representation at a first preset ratio after segmenting the training text and mapping it to a high-dimensional embedding representation to convert the target word vector representation;

[0177] The third execution module is used to perform the following operations on the target vocabulary representation:

[0178] masking the words of the target vocabulary based on a second preset ratio;

[0179] Based on a third preset ratio, the vocabulary of the target vocabulary is replaced; the first preset ratio is less than a first ratio threshold, the second preset ratio is less than a second ratio threshold, and the third preset ratio is less than a third ratio threshold.

[0180] In some embodiments, the acquisition module 401 is further used to acquire the training document text in the corresponding scenario after determining the initial text error correction model as the text error correction model when the reward value meets the iteration stop condition;

[0181] The training device for the text error correction model also includes: a fine-tuning module for fine-tuning the text error correction model based on the training document text.

[0182] In order to solve the above technical problems, the embodiment of the present application further provides a text correction device, which is applied to a text correction model. The text correction model is trained by the above text correction model training method. Figure 5 The structure diagram of the text error correction device provided by the embodiment of the present application is shown. Figure 5 As shown, the text error correction device comprises:

[0183] A receiving module 501 is used to receive an original text to be corrected;

[0184] A third determination module 502 is used to input the original text into a text error correction model to determine a second target action for the original text;

[0185] The second execution module 503 is used to execute a second target action on the original text to obtain a target text;

[0186] The error correction module 504 is used to correct the original text based on the target text.

[0187] The device provided in the embodiment of the present application is the same as the method in the above embodiment, so both have the same embodiments and beneficial effects, which will not be repeated here.

[0188] Figure 6 FIG. 1 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0189] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0190] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0191] The memory 602 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0192] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement any one of the text error correction model training methods and text error correction methods in the above embodiments.

[0193] In one example, the electronic device may further include a communication interface 603 and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected via the bus 604 and communicate with each other.

[0194] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0195] Bus 604 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 604 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0196] In addition, in combination with the training method of the text error correction model and the text error correction method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the text error correction model training methods and text error correction methods in the above embodiments is implemented.

[0197] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, implements any one of the text error correction model training methods and text error correction methods in the above embodiments.

[0198] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0199] The functional blocks shown in the block diagram above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0200] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0201] Aspects of the present disclosure are described above with reference to the flowcharts and / or block diagrams of a model training method, text error correction method, device, equipment, medium, and product according to an embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0202] The above contents are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A training method for a text error correction model, characterized in that: include: Obtaining the training text with errors and the corresponding correct reference text; Segmenting the training text and mapping it to a high-dimensional embedding representation to convert it into a target word vector representation; Inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network; Performing the first target action on the target word vector representation to update the target word vector representation; Determining a reward value for executing the first target action based on the updated target word vector representation and the reference text; In the case where the reward value does not satisfy the iteration stop condition, the policy network is updated based on the reward value, and the step of inputting the target word vector representation into the pre-built initial text error correction model is returned to determine the first target action for the target word vector representation based on the policy network; When the reward value satisfies the iteration stop condition, the initial text error correction model is determined as the text error correction model.

2. The training method of the text error correction model according to claim 1, characterized in that: The step of inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network includes: Inputting the target word vector representation into the initial text error correction model; Determining an action probability distribution based on the target word vector representation and the weight matrix of the policy network; Based on the action probability distribution, the first target action is selected from the action space.

3. The training method of the text error correction model according to claim 2, characterized in that: The selecting the first target action from the action space based on the action probability distribution includes: Determining the selection probability of each action in the action space based on the action probability distribution; Based on the selection probability, the first target action is determined from each action in the action space.

4. The training method of the text error correction model according to claim 2, characterized in that: When the reward value does not satisfy the iteration stop condition, updating the policy network based on the reward value includes: When the reward value does not satisfy the iteration stop condition, updating the action value network of the initial text error correction model based on the reward value; Based on the updated action-value network, a weight matrix of the policy network is determined.

5. The training method of the text error correction model according to claim 1, characterized in that: The step of determining a reward value for executing the first target action based on the updated target word vector representation and the reference text includes: Based on the reference text, determining the vocabulary accuracy and sentence connection accuracy of the updated target word vector representation; The reward value is determined based on the vocabulary accuracy and the sentence connection accuracy.

6. The method for training a text error correction model according to claim 5, characterized in that: The step of determining the reward value based on the vocabulary accuracy and the sentence connection accuracy includes: Based on the cross entropy loss function, determining a first cross entropy loss value corresponding to the vocabulary accuracy, and determining a second cross entropy loss value corresponding to the sentence connection accuracy; the first cross entropy loss value is positively correlated with the vocabulary accuracy, and the second cross entropy loss value is positively correlated with the sentence connection accuracy; The reward value is determined based on the first cross entropy loss value and the second cross entropy loss value.

7. The method for training a text error correction model according to any one of claims 1 to 6, characterized in that: After the training text is segmented and mapped to a high-dimensional embedding representation to convert the target word vector representation, the method further includes: Selecting a target vocabulary from the target word vector representation at a first preset ratio; The following operations are performed on the target vocabulary representation: Based on a second preset ratio, masking the words of the target vocabulary; Based on a third preset ratio, the vocabulary of the target vocabulary is replaced; the first preset ratio is less than a first ratio threshold, the second preset ratio is less than a second ratio threshold, and the third preset ratio is less than a third ratio threshold.

8. The method for training a text error correction model according to claim 1, characterized in that: In the case where the reward value satisfies the iteration stop condition, after determining the initial text error correction model as the text error correction model, the method further includes: Obtain the training document text for the corresponding scenario; The text error correction model is fine-tuned based on the training document text.

9. A text error correction method, characterized in that: Applied to a text error correction model, the text error correction model is trained by the text error correction model training method according to any one of claims 1 to 8, the text error correction method comprising: Receiving the original text to be corrected; Inputting the original text into the text error correction model to determine a second target action for the original text based on a policy network; Performing the second target action on the original text to obtain a target text; Based on the target text, error correction is performed on the original text.

10. A training device for a text error correction model, characterized in that: include: An acquisition module is used to acquire the training text with errors and the corresponding correct reference text; A conversion module, used to segment the training text and map it to a high-dimensional embedding representation to obtain a target word vector representation; An input module, used for inputting the target word vector representation into a pre-built initial text error correction model to determine a first target action for the target word vector representation based on a policy network; A first execution module, configured to execute the first target action on the target word vector representation to update the target word vector representation; A first determination module, configured to determine a reward value for executing the first target action based on the updated target word vector representation and the reference text; An updating module, configured to update the policy network based on the reward value when the reward value does not satisfy the iteration stop condition, and return to the step of inputting the target word vector representation into the pre-built initial text error correction model to determine a first target action for the target word vector representation based on the policy network; The second determination module is used to determine the initial text error correction model as the text error correction model when the reward value satisfies the iteration stop condition.

11. A text error correction device, characterized in that: Applied to a text error correction model, the text error correction model is trained by the text error correction model training method according to any one of claims 1 to 8, and the text error correction device comprises: A receiving module, used for receiving the original text to be corrected; A third determination module, configured to input the original text into the text error correction model to determine a second target action for the original text; A second execution module, configured to execute the second target action on the original text to obtain a target text; The error correction module is used to correct the original text based on the target text.

12. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the training method of the text error correction model as described in any one of claims 1 to 8 and the text error correction method as described in claim 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the text error correction model training method as described in any one of claims 1 to 8 and the text error correction method as described in claim 9.

14. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the text error correction model training method as described in any one of claims 1 to 8 and the text error correction method as described in claim 9.

Images