A training method, device, medium and product for a text error correction model

Abstract

The present invention provides a training method, device, medium and product for a text error correction model. The method comprises the following steps: obtaining a sample text list, segmenting any sample text in the sample text list to obtain sample segments; inputting any sample segment into a preset text error correction model, obtaining a first loss function value and a second loss function value, and obtaining a total loss function value based on the first loss function value and the second loss function value; the text error correction model can be adjusted with the total loss function value until the text error correction model converges, thereby improving the accuracy of the text error correction model for text, while ensuring that the text is correctly corrected, reducing the error correction of the text by the error correction model and enhancing the replication of the correct text, thereby reducing false positives during text error correction.

Description

Technical Field

[0001] The present invention relates to the field of text processing technology, and in particular to a training method, equipment, medium and product of a text error correction model. Background Art

[0002] Text error correction is a fundamental problem in Natural Language Processing (NLP). It can usually be placed before other NLP tasks such as text retrieval, text classification, machine translation or sequence labeling to improve the effectiveness of input text and prevent the adverse effects of spelling errors.

[0003] In the prior art, by training a neural network model, wherein the neural network model includes a memory bank gating layer, a coding layer and an output layer, the coding vectors of semantic units selected from the historical text of the sample text can be added to the memory bank, thereby combining the coding vectors of the semantic units in the memory bank to realize error correction of the sample text. However, this only enriches the error correction information used by the text error correction model, and does not solve the problem that when generating sample sentences in the text error correction model of the copy attention mechanism, more words are changed, thereby resulting in lower accuracy of the error correction results. Summary of the invention

[0004] The object of the present invention is to provide a training method for a text error correction model, which can reduce the erroneous correction of the text by the error correction model and enhance the replication of the correct text while ensuring the correct error correction of the text, thereby reducing false positives during text error correction, and comprises the following steps:

[0005] S100, obtain sample text list A = {A 1 , A 2 , A 3 , ..., A m}, where A i refers to the i-th sample text, i=1…m, m is the number of sample texts;

[0006] S200, A i Input into the preset text error correction model to obtain A i The corresponding original text fragment list B i = {B i1 , B i2 , B i3 , ..., B in}, B ix It means in A i The xth original text segment in , x = 1...n, n is the number of original text segments;

[0007] S300, obtain B iThe corresponding actual text fragment list C i ={C i1 , C i2 , C i3 , ..., C iS}, C iy A i The corresponding y-th actual text segment, y = 1...s, s is A i The actual number of text fragments in

[0008] S400, according to B i and C i , get the first loss function value I corresponding to A 1 ;

[0009] S500, when B ix =C iy When B ix As the target text segment is marked as 0 and when B ix ≠C iy When B ix As the target text segment, it is marked as 1 to construct A i Corresponding target text segment list B' i = {B' i1 , B' i2 , B' i3 , ..., B' in}, B' ix A i The corresponding tag of the x-th target text segment;

[0010] S600, according to B i and B' i , get the second loss function value I corresponding to A 2 ;

[0011] S700, based on I 1 and I 2 , get the total loss function value I 0 And according to I 0 , train the preset text error correction model until the preset text error correction model converges, where I 0 Meet the following conditions:

[0012] I 0 =I 1 +I 2 .

[0013] The present invention also protects an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method of the text error correction model as mentioned above.

[0014] The present invention also protects a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement the above-mentioned text error correction model training method.

[0015] The present invention also protects a computer program product, including a computer program, wherein the computer program is executed by a processor to perform the above-mentioned text error correction model training method.

[0016] The present invention obtains a sample text list, segments any sample text in the sample text list, and obtains sample segments; inputs any sample segment into a preset text error correction model, obtains a first loss function value and a second loss function value, and obtains a total loss function value based on the first loss function value and the second loss function value; the text error correction model can be adjusted with the total loss function value until the text error correction model converges, thereby improving the accuracy of text error correction by the text error correction model, while ensuring that the text is correctly corrected, reducing the error correction of the text by the error correction model and enhancing the replication of the correct text, thereby reducing false positives during text error correction.

[0017] In addition, before decoding, the decoder in the encoder-decoder of the present invention needs to use the text error correction model to classify the labels of each input text segment vector to obtain the error correction label of each input text segment vector, and the error correction label is used to indicate whether the corresponding text segment is wrong. The correctness of the input text segment encoding can be explicitly enhanced through the text segment label classification task, thereby reducing the modification of the correct text segment by the model while ensuring the correction of the erroneous text segment, thereby improving the accuracy of text error correction.

[0018] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a method for training a text error correction model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, an embodiment of the present invention provides a method for training a text error correction model, comprising the following steps:

[0023] S100, obtain sample text list A = {A 1 , A 2 , A 3 , ..., A m}, where A i It refers to the i-th sample text, i=1…m, and m is the number of sample texts.

[0024] S200, get A i The corresponding original text fragment list B i = {B i1 , B i2 , B i3 , ..., B in}, B ix It means in A i The xth original text segment in , x = 1...n, n is the number of original text segments.

[0025] Specifically, B i By applying A i The sample text is segmented and processed according to the type of the sample text to obtain original text fragments. Those skilled in the art know how to segment the sample text by any method in the prior art, which will not be described in detail here.

[0026] Preferably, when the sample text is Chinese text, the original text segment is a single character or a single word.

[0027] Specifically, the preset text error correction model includes: an encoding layer, a decoding layer, copy attention, and encoding layer-decoding layer attention, wherein the copy attention and the encoding layer-decoding layer attention both adopt a multi-head attention mechanism.

[0028] S300, obtain B i The corresponding actual text fragment list C i ={C i1 , C i2 , C i3 , ..., C iS}, C iy A i The corresponding y-th actual text segment, y = 1...s, s is A i The actual number of text fragments in .

[0029] Specifically, those skilled in the art will know that C i According to A i The sample text is obtained from the corresponding reference text, which will not be described in detail here. The reference text refers to the text with no error sentences corresponding to the sample text.

[0030] S400, according to B i and C i , get the first loss function value I corresponding to A 1 .

[0031] Specifically, step S400 also includes the following steps to determine I 1 :

[0032] S401, B i Input to the encoding layer to obtain B i The corresponding encoding vector H i ;

[0033] S403, H i Input to the decoding layer to obtain B i The corresponding decoding vector U i = {U i1 , U i2 , U i3 , ..., U in},U ix Refers to B ix The corresponding decoding vector;

[0034] S405, according to U ix and C iy , get U ix The corresponding first probability value P ix ;

[0035] S407, according to P ix , get the first loss function value I corresponding to A 1 , I 1 Meet the following conditions:

[0036]

[0037] Furthermore, H i Meet the following conditions:

[0038] H i =f(B i1 , B i2 , ..., B in ), where f() refers to the nonlinear transformation function in the coding layer.

[0039] Furthermore, U ix Meet the following conditions:

[0040] U ix =g(U i1 , U i2 , ..., U ix-1 , H i ), where g() refers to the nonlinear transformation function in the decoding layer.

[0041] Furthermore, P ix Meet the following conditions:

[0042] P ix =S(U ix ), wherein S() is a normalization function; preferably, the normalization function is a softmax function.

[0043] Specifically, those skilled in the art will know that C i According to A i The sample text is obtained from the corresponding reference text, which will not be described in detail here. The reference text refers to the text with no error sentences corresponding to the sample text.

[0044] S500, when B ix =C iy When B ix As the target text segment is marked as 0 and when B ix ≠C iy When B ix As the target text segment, it is marked as 1 to construct A i Corresponding target text segment list B' i = {B' i1 , B' i2 , B' i3 , ..., B' in}, B' ix Ai The corresponding tag of the x-th target text segment.

[0045] S600, according to B i and B' i , get the second loss function value I corresponding to A 2 .

[0046] Specifically, step S600 also includes the following steps to obtain I 2 :

[0047] S601, B i Input to the encoding layer to obtain B i The corresponding encoding vector H i ={H i1 , H i2 , H i3 , ..., H in}, H ix Refers to B ix The corresponding encoding vector;

[0048] S603, according to H ix and B' ix , get H ix The corresponding second probability value P' ix ;

[0049] S607, according to P' ix , get the second loss function value I corresponding to A 2 , I 2 Meet the following conditions:

[0050]

[0051] Specifically, P' ix Meet the following conditions:

[0052] P' ix =S(H ix ).

[0053] S700, based on I 1 and I 2 , get the total loss function value I 0 And according to I 0 , train the preset text error correction model until the preset text error correction model converges, where I 0 Meet the following conditions:

[0054] I 0 =I 1 +I 2 .

[0055] An embodiment of the present invention obtains a sample text list, segments any sample text in the sample text list, and obtains sample segments; inputs any sample segment into a preset text error correction model, obtains a first loss function value and a second loss function value, and obtains a total loss function value based on the first loss function value and the second loss function value; the text error correction model can be adjusted with the total loss function value until the text error correction model converges, thereby improving the accuracy of text error correction by the text error correction model, while ensuring that the text is correctly corrected, reducing the error correction of the text by the error correction model and enhancing the replication of the correct text, thereby reducing false positives during text error correction.

[0056] In a specific embodiment, the method further comprises the following steps:

[0057] S1. Get several sample texts from A and construct a target text list in, It refers to the pth target text, p=1…q, q is the number of target texts.

[0058] Preferably, q=m.

[0059] Specifically, in The number of abnormal statements is consistent with the number of non-abnormal statements; it can be understood that the number of erroneous statements is consistent with the number of non-erroneous statements. Those skilled in the art know how to use the existing technology to mark erroneous statements.

[0060] S2. According to Get the first target text list E = {E 1 , E 2 , E 3 ,……,E N}, E t refers to the tth first target text, t = 1 ... N, N is the number of first target text segments, and the second target text list E' = {E' 1 , E' 2 , E' 3 , ..., E' M}, E' r It refers to the rth second target text, r=1…M, M is the second target text.

[0061] Specifically, when the target text is consistent with the reference text corresponding to the target text, the target text is determined to be the first target text, and the second target text is determined to be Other target texts except the second target text in .

[0062] Specifically, the first target text is a non-abnormal text segment, and the second target text is an abnormal text; it can be understood that: the first target text is an error-free sentence, and the second target text is an error-containing sentence. When training the error correction model, equal amounts of correct text segments and erroneous text segments can be input into the error correction model. When the correct text segment is input, the attention of the encoding layer-decoding layer is removed, and more text segments will be copied from the input to reduce changes to the correct text. For the erroneous input text, fewer text segments will be copied from the input when generating text segments to ensure the correction of the erroneous text segments.

[0063] S3. Input E into the encoding layer, copy attention and decoding layer to obtain E tj The corresponding probability value F tj , where E tj It refers to the jth text segment in the tth first target text, j=1...z, and z is the number of text segments in the first target text.

[0064] F tj =(1-α r )×F 1 tj +α r ×F 2 tj .

[0065] Specifically, F 1 tj Get references S401-S407.

[0066] Specifically, F 2 tj Meet the following conditions:

[0067] F 2 tj =S(A tj ).

[0068] Specifically, α t Meet the following conditions:

[0069] α t =H(G tj ), where H() is the activation function, i.e. H() is the sigmoid function.

[0070] Furthermore, 0≤α t ≤1.

[0071] Furthermore, G tj Meet the following conditions:

[0072] Among them, W T is the weight matrix of the copied attention.

[0073] Furthermore, V t Meet the following conditions:

[0074] Among them, W T v is the VALUE weight matrix, For E t The corresponding coding layer vector; The method for obtaining is referred to the steps S401-S407 in the above embodiment, and will not be repeated here.

[0075] Furthermore, A tj Meet the following conditions:

[0076] A tj =Q T tj ×K t , A tj and A T tj are transposed matrices of each other.

[0077] Furthermore, Q tj Meet the following conditions:

[0078] Among them, Q tj and Q T tj are transposed matrices, W T q is the QUERY weight matrix, It means E tj The corresponding decoding layer vector.

[0079] Furthermore, K t Meet the following conditions:

[0080] Among them, W T k is the KEY weight matrix.

[0081] The above W T q , W T k , W T v These are all pre-set weight matrices in the multi-head attention mechanism.

[0082] S4. Input E' into the encoding layer, decoding layer, encoding layer-decoding layer attention and copy attention to obtain E' r The probability value corresponding to any text segment in can be understood as: E' r The probability value corresponding to any text segment in Ftj The method of obtaining is the same and will not be repeated here.

[0083] Before decoding, the decoder in the encoder-decoder of the present invention needs to use the text error correction model to classify the labels of each input text segment vector to obtain the error correction label of each input text segment vector, and the error correction label is used to indicate whether the corresponding text segment is wrong. The correctness of the input text segment encoding can be explicitly enhanced through the text segment label classification task, thereby reducing the modification of the correct text segment by the model while ensuring the correction of the erroneous text segment, thereby improving the accuracy of text error correction.

[0084] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0085] The electronic device of the embodiment of the present invention exists in various forms, including but not limited to:

[0086] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0087] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0088] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0089] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0090] (5) Other electronic devices with data interaction functions.

[0091] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0092] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0093] An embodiment of the present invention provides a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the above-mentioned text error correction model training method.

[0094] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0095] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0096] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A training method for a text error correction model, characterized in that: The steps include: S100, obtain sample text list A = {A1, A2, A3, ..., A m }, where A i refers to the i-th sample text, i=1…m, m is the number of sample texts; S200, A i Input into the preset text error correction model to obtain A i The corresponding original text fragment list B i = {B i1 , B i2 , B i3 , ..., B in }, B ix It means in A i The xth original text segment in , x=1...n, n is the number of original text segments, wherein the preset text error correction model includes: an encoding layer, a decoding layer, an attention of the encoding layer and the decoding layer, and a copy attention; S300, obtain B i The corresponding actual text fragment list C i ={C i1 , C i2 , C i3 , ..., C iS }, C iy A i The corresponding y-th actual text segment, y = 1...s, s is A i The actual number of text fragments in S400, according to B i and C i , obtain the first loss function value I1 corresponding to A, wherein the step S400 also includes the following steps to determine I1: S401, B i Input to the encoding layer to obtain B i The corresponding encoding vector H i ; S403, H i Input to the decoding layer to obtain B i The corresponding decoding vector U i = {U i1 , U i2 , U i3 , ..., U in },U ix Refers to B ix The corresponding decoding vector; S405, according to U ix and C iy , get U ix The corresponding first probability value P ix ; S407, according to P ix , obtain the first loss function value I1 corresponding to A, I1 meets the following conditions: S500, when B ix =C iy When B ix As the target text segment is marked as 0 and when B ix ≠C iy When B ix As the target text segment, it is marked as 1 to construct A i Corresponding target text segment list B' i = {B' i1 , B' i2 , B' i3 , ..., B' in }, B' ix A i The corresponding tag of the x-th target text segment; S600, according to B i and B' i , obtain the second loss function value I2 corresponding to A; S700, based on I1 and I2, obtain a total loss function value I0, and train a preset text error correction model according to I0 until the preset text error correction model converges, wherein I0 meets the following conditions: I0=I1+I2.

2. The training method of the text error correction model according to claim 1, characterized in that: H i Meet the following conditions: H i =f(B i1 , B i2 , ..., B in ), where f() refers to the nonlinear transformation function in the coding layer.

3. The training method of the text error correction model according to claim 1, characterized in that: U ix Meet the following conditions: U ix =g(U i1 , U i2 , ..., U ix-1 , H i ), where g() refers to the nonlinear transformation function in the decoding layer.

4. The training method of the text error correction model according to claim 3 is characterized in that: P ix Meet the following conditions: P ix =S(U ix ), where S() is the normalization function.

5. The training method of the text error correction model according to claim 4 is characterized in that: The normalization function is a softmax function.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method of the text error correction model as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method of the text error correction model as claimed in any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that The computer program is executed by a processor to implement the training method of the text error correction model as described in any one of claims 1 to 5.

Images