Sentence inference method and training method and device, electronic equipment and storage medium

Abstract

The sentence inference method and the training method, the device, the electronic equipment and the storage medium, wherein the training method of the sentence inference comprises: inputting a first training corpus set and a second training corpus set into a first neural network model, training the first neural network model, obtaining a first sentence inference initial model; inputting the second training corpus set into a second neural network model, training the second neural network model, obtaining a second sentence inference initial model; inputting a third training corpus set, a fourth training corpus set and a fifth training corpus set into the first sentence inference initial model, fine-tuning the first sentence inference initial model, obtaining a first sentence inference model; inputting the fourth training corpus set and the fifth training corpus set into the second sentence inference initial model, fine-tuning the second sentence inference initial model, obtaining a second sentence inference model; and outputting the trained sentence inference model. By using the above scheme, when inferring whether a sentence has an error and a specific error type, the computational amount can be reduced, and system resources can be saved.

Description

Technical Field

[0001] This specification relates to the field of computer natural language processing technology, and in particular to a sentence inference method and training method, apparatus, electronic device and storage medium. Background Technology

[0002] For language learners (e.g., English, Chinese, etc.), spoken expression is prone to errors, which can be categorized into grammatical errors and semantic ambiguity.

[0003] Currently, inferring the error type of spoken expressions mainly involves feature extraction, such as extracting features like sentence perplexity, grammatical error rate, and calculating the alignment value between the input corpus and the sentences back-translated from the input corpus. The extracted features are then used to train a classifier, which is then used to predict the error type of spoken expressions.

[0004] However, predicting the error types in spoken expressions requires extracting multiple features, which involves a large amount of computation and consumes a significant amount of system resources. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a statement inference method, training method, apparatus, electronic device, and storage medium, which can reduce the amount of computation and save system resources when inferring whether a statement has an error and the specific type of error.

[0006] First, the embodiments in this specification provide a rich training method for statement inference, including:

[0007] A first training corpus and a second training corpus of the target language type are obtained respectively, wherein the second training corpus includes a semantically ambiguous training corpus and a grammatically incorrect training corpus;

[0008] The first training corpus and the second training corpus are input into a preset first neural network model, and the first neural network model is trained until it reaches a preset first stopping condition to obtain a first sentence inference initial model; the second training corpus is input into a preset second neural network model, and the second neural network model is trained until it reaches a preset second stopping condition to obtain a second sentence inference initial model;

[0009] A third training corpus, a fourth training corpus, and a fifth training corpus containing annotation information are obtained respectively, wherein: the third training corpus includes correct corpus; the fourth training corpus includes semantically unclear corpus; and the fifth training corpus includes grammatically incorrect corpus.

[0010] The third, fourth, and fifth training corpora are input into the first initial sentence inference model, and the first initial sentence inference model is fine-tuned until it reaches a preset third stopping condition to obtain the first sentence inference model; and the fourth and fifth training corpora are input into the second initial sentence inference model, and the second initial sentence inference model is fine-tuned until it reaches a preset fourth stopping condition to obtain the second sentence inference model;

[0011] Output the trained statement inference model, including the first statement inference model and the second statement inference model.

[0012] This specification also provides a statement inference method in its embodiments, including: obtaining the statement to be inferred;

[0013] The statement to be inferred is input into the statement inference model for inference processing and the inference result is output.

[0014] The statement inference model includes a first statement inference model and a second statement inference model, and is obtained using the statement inference training method described in any of the foregoing embodiments.

[0015] The inference process and output of the inference result include:

[0016] The statement to be inferred is input into the first statement inference model to obtain a first inference result. If the first inference result is of the first type, the first inference result is used as the inference result and output.

[0017] If the first inference result is of the second type, the statement to be inferred is input into the second statement inference model to obtain the second inference result, and the second inference result is used as the inference result and output.

[0018] Accordingly, embodiments of this specification also provide a statement inference training device, including:

[0019] The first corpus acquisition unit is adapted to acquire a first training corpus set and a second training corpus set of the target language type, respectively, wherein the second training corpus set includes a semantically ambiguous training corpus set and a grammatically incorrect training corpus set;

[0020] The first training unit is adapted to input the first training corpus set and the second training corpus set into a preset first neural network model, train the first neural network model until the first neural network model reaches a preset first stopping condition, and obtain the first sentence inference initial model;

[0021] The second training unit is adapted to input the second training corpus into a preset second neural network model, train the second neural network model until the second neural network model reaches a preset second stopping condition, and obtain the initial model for second sentence inference;

[0022] The second corpus acquisition unit is adapted to acquire a third training corpus set, a fourth training corpus set, and a fifth training corpus set containing annotation information, wherein: the third training corpus set includes correct corpus; the fourth training corpus set includes semantically unclear corpus; and the fifth training corpus set includes grammatically incorrect corpus.

[0023] The first fine-tuning unit is adapted to input the third training corpus, the fourth training corpus, and the fifth training corpus into the first sentence inference initial model, and to fine-tune the first sentence inference initial model until the first sentence inference initial model reaches the preset third stopping condition, thereby obtaining the first sentence inference model.

[0024] The second fine-tuning unit is adapted to input the fourth training corpus set and the fifth training corpus set into the second sentence inference initial model, fine-tune the second sentence inference initial model until the second sentence inference initial model reaches the preset fourth stopping condition, and obtain the second sentence inference model;

[0025] The inference model output unit is adapted to output the trained statement inference model, including the first statement inference model and the second statement inference model.

[0026] Accordingly, embodiments of this specification also provide a statement inference apparatus, including:

[0027] The unit for obtaining statements to be inferred is suitable for obtaining statements to be inferred.

[0028] The processing unit is adapted to input the statement to be inferred into the statement inference model for inference processing and output the inference result;

[0029] The statement inference model includes a first statement inference model and a second statement inference model;

[0030] The inference process and output of the inference result include:

[0031] The statement to be inferred is input into the first statement inference model to obtain a first inference result. If the first inference result is of the first type, the first inference result is used as the inference result and output.

[0032] If the first inference result is of the second type, the statement to be inferred is input into the second statement inference model to obtain the second inference result, and the second inference result is used as the inference result and output.

[0033] This specification also provides an electronic device, including a memory and a processor, wherein the memory is adapted to store one or more computer instructions, characterized in that, when the processor executes the computer instructions, it performs the steps of the training method for statement inference described in any of the foregoing embodiments, or performs the steps of the statement inference method in the foregoing embodiments.

[0034] This specification also provides a computer-readable storage medium storing computer instructions that, when executed, can perform the steps of the training method for statement inference described in any of the foregoing embodiments, or perform the steps of the statement inference method in the foregoing embodiments.

[0035] The sentence inference training method described in this specification trains a first neural network model using a first training corpus and a second training corpus, and then fine-tunes the initial first sentence inference model using a third, fourth, and fifth training corpus containing labeled information. This method eliminates the need to extract multiple features from the sentence, thus obtaining a first sentence inference model that can infer whether a sentence is incorrect. Furthermore, by training a second neural network model using the second training corpus and fine-tuning the initial second sentence inference model using a fourth and fifth training corpus containing labeled information, the method eliminates the need to extract multiple features from the sentence, thus obtaining a second sentence inference model that can infer the specific error type of the sentence. Therefore, this method reduces the computational load during training and saves system resources.

[0036] The statement inference method described in this specification can infer whether there are errors in spoken expressions using the obtained first statement inference model. When the first statement inference model infers that there are errors in the statement, the second statement inference model infers the type of error in the statement. This eliminates the need to extract multiple features of spoken expressions, thus reducing computation and saving system resources.

[0037] Further, a sixth training corpus is obtained; the source corpus in the sixth training corpus is input into a preset translation model for translation processing to obtain the translation result of the target language type corresponding to the source corpus; each segment of the source corpus is aligned with its corresponding translation result, and the aligned part of the translation result is deleted; the aligned part of the source corpus is input into the translation model for translation to obtain the corresponding literal translation result; the literal translation result is embedded into the translation result after deleting the aligned part to obtain the semantically ambiguous training corpus. By obtaining the sixth training corpus and performing translation, alignment, and literal translation result embedding operations on it, the data volume of the obtained semantically ambiguous training corpus can be expanded, thereby improving the inference accuracy of the first and second sentence inference initial models when training the first neural network model and the second neural network model.

[0038] Furthermore, by acquiring a first verification corpus set and a second verification corpus set to verify whether the first neural network model and the second neural network model have completed training, wherein the corpora in the first verification corpus set and the second verification corpus set are corpora of the target language type, and by continuously inputting the first verification corpus set and the second verification corpus set into the first neural network model multiple times, multiple two-dimensional vectors corresponding to the first verification corpus set and the second verification corpus set can be obtained. If the two-dimensional vectors obtained in multiple consecutive traversals are the same, the training of the first neural network model is stopped. As can be seen from the above, by inputting the first verification corpus set and the second verification corpus set into the first neural network model, and stopping the training of the first neural network when it is determined that the two-dimensional vectors obtained in multiple consecutive traversals are the same, the training time can be reduced while ensuring the inference accuracy of the initial model for inferring the first sentence. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this specification, the drawings used in the description of the embodiments of this specification or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart of a training method for statement inference in an embodiment of this specification is shown;

[0041] Figure 2 This specification illustrates a flowchart of an embodiment for obtaining an initial model for inferring the first statement;

[0042] Figure 3This document illustrates a flowchart of a fine-tuning process for an initial model of first statement inference, as described in an embodiment of this specification.

[0043] Figure 4 A flowchart of a statement inference method according to an embodiment of this specification is shown;

[0044] Figure 5 A schematic diagram of the structure of a statement inference training device according to an embodiment of this specification is shown;

[0045] Figure 6 A schematic diagram of the structure of a statement inference device according to an embodiment of this specification is shown;

[0046] Figure 7 A schematic diagram of the result of an electronic device according to an embodiment of this specification is shown. Detailed Implementation

[0047] As described in the background section, current methods primarily rely on feature extraction to infer whether spoken expressions contain errors. However, using these methods requires significant computation for each error inference, consuming substantial system resources.

[0048] To address the aforementioned issues, this specification provides a statement inference scheme that first infers whether an oral statement contains errors, and if an error is found, infers the type of error, and then provides correct guidance to the speaker based on the specific error type.

[0049] In the embodiments of this specification, in order to accurately infer whether there are errors in spoken sentences and the specific types of errors, different corpora can be used to train a preset neural network model to obtain two different sentence inference models. One sentence inference model is used to infer whether there are errors in spoken sentences, and the other sentence inference model is used to infer the specific types of errors.

[0050] On the one hand, the embodiments of this specification provide a training method for sentence inference. By training a preset neural network model with a training corpus, and fine-tuning the initial sentence inference model obtained by training with labeled information, a first sentence inference model that infers whether a sentence is wrong and a second inference statement that infers the specific error type of the sentence can be obtained without extracting multiple features of the sentence. Therefore, the amount of computation in the training process can be reduced and system resources can be saved.

[0051] On the other hand, the embodiments of this specification also provide a sentence inference method, which can infer whether there are errors in spoken expressions by using a first sentence inference model obtained through training, and when the first sentence inference model infers that there are errors in the sentence, the second sentence inference model infers the type of error in the sentence. There is no need to extract multiple features of spoken expressions, so the amount of computation can be reduced and system resources can be saved.

[0052] More specifically, the training method for sentence inference in the embodiments of this specification includes: acquiring a first training corpus set and a second training corpus set of the target language type, wherein the second training corpus set includes a semantically ambiguous training corpus set and a grammatically incorrect training corpus set; inputting the first training corpus set and the second training corpus set into a preset first neural network model, training the first neural network model until the first neural network model reaches a preset first stopping condition, to obtain a first initial model for sentence inference; inputting the second training corpus set into a preset second neural network model, training the second neural network model until the second neural network model reaches a preset second stopping condition, to obtain a second initial model for sentence inference; and acquiring a third training corpus set, a fourth training corpus set, and a fifth training corpus set containing annotation information. The training corpus includes: a third training corpus containing correct data; a fourth training corpus containing semantically ambiguous data; and a fifth training corpus containing grammatically incorrect data. The third, fourth, and fifth training corpora are input into the first initial statement inference model, and the first initial statement inference model is fine-tuned until it reaches a preset third stopping condition, thus obtaining a first statement inference model. The fourth and fifth training corpora are then input into the second initial statement inference model, and the second initial statement inference model is fine-tuned until it reaches a preset fourth stopping condition, thus obtaining a second statement inference model. Finally, the trained statement inference model is output, comprising the first statement inference model and the second statement inference model.

[0053] To enable those skilled in the art to better understand and implement the embodiments of this specification, the following detailed description is provided with reference to the accompanying drawings and specific application examples.

[0054] Specific reference Figure 1 The training method for statement inference shown in the embodiments of this specification can be performed according to the following steps:

[0055] S11, Obtain the first training corpus of the target language type.

[0056] In practice, the wider the source of the first training corpus of the target language type, the better the versatility and accuracy of the sentence inference model trained using the first training corpus. Therefore, the first corpus can be obtained from multiple different fields.

[0057] For example, if the user is a student, the relevant text data can be obtained from the textbook as the first training corpus, so that the student can judge whether there are any errors in their spoken expression through the subsequently trained model.

[0058] In practice, one or more target language types can be selected as the first corpus according to actual needs. For example, the target language type can be Chinese, English, German, French, etc.

[0059] In specific implementation, the first training corpus may be a single sentence, a paragraph containing multiple sentences, or a document containing many sentences. This specification does not limit the specific form of the acquired training corpus; each corpus in the first training corpus has corresponding annotation information.

[0060] S12, Obtain the second training corpus of the target language type.

[0061] The second training corpus includes a semantically ambiguous training corpus and a grammatically incorrect training corpus.

[0062] In practice, the acquisition method of the second training corpus of the target language type can be the same as the acquisition process of the first training corpus, wherein each corpus in the second training corpus has corresponding annotation information.

[0063] S13, input the first training corpus set and the second training corpus set into the preset first neural network model, train the first neural network model until the first neural network model reaches the preset first stopping condition, and obtain the first sentence inference initial model.

[0064] In practice, the corpora from the first and second training corpora can be mixed together and used as input to the first neural network model. Since each corpus in both sets has corresponding labeled information, the first neural network model can be trained in a manner similar to supervised learning, enabling it to classify the input corpora and distinguish between correct and incorrect corpora. Furthermore, by setting a first stopping condition, the training time can be reduced while maintaining the inference accuracy of the initial model for the first sentence inference.

[0065] S14, input the second training corpus into the preset second neural network model, train the second neural network model until the second neural network model reaches the preset second stopping condition, and obtain the initial model for second sentence inference.

[0066] In practice, the corpora from the second training corpus can be mixed together and used as input to the second neural network model. Since each corpus in the second training corpus has corresponding labeled information, it can perform a learning method similar to supervised learning on the second neural network model, enabling the model to classify the input corpora, distinguishing between semantically ambiguous and grammatically incorrect corpora. Moreover, by setting a second stopping condition, the training time can be reduced while ensuring the inference accuracy of the initial model for inferring the second sentence.

[0067] S15, Obtain the third training corpus containing annotation information.

[0068] The third training corpus includes correct corpora.

[0069] In practice, the wider the source of the first training corpus containing correct data and the more standardized the annotation information, the more beneficial it is for subsequent fine-tuning of the initial model for first-sentence inference, which can increase the universality and accuracy of the trained first-sentence inference model. Therefore, on the one hand, in order to make the source of the third training corpus wider, data can be obtained from different fields as much as possible; on the other hand, in order to ensure that all data in the first training corpus are correct data, proofread data can be obtained.

[0070] S16, obtain the fourth and fifth training corpora containing annotation information.

[0071] The fourth training corpus includes semantically ambiguous corpus; the fifth training corpus includes grammatically incorrect corpus.

[0072] In practice, the process of obtaining the fourth and fifth training corpora containing annotation information can be the same as the process of obtaining the third training corpus in step S15, and will not be elaborated further here. The difference is that all the corpora in the obtained fourth and fifth training corpora are erroneous corpora. Specifically, the corpora in the fourth training corpus are semantically ambiguous, and the corpora in the fifth training corpus are grammatically incorrect.

[0073] S17, input the third training corpus set, the fourth training corpus set, and the fifth training corpus set into the first sentence inference initial model, fine-tune the first sentence inference initial model until the first sentence inference initial model reaches the preset third stopping condition, and obtain the first sentence inference model.

[0074] In practice, the corpora from the third, fourth, and fifth training corpora can be mixed together and used as input to the initial model for the first sentence inference. Since each corpus in these corpora contains corresponding annotation information, the initial model for the first sentence inference can be trained in a supervised learning manner. This allows the second neural network model to classify the input corpora and output corresponding discrete values ​​to characterize whether the input corpora are correct or incorrect. Furthermore, by setting a third stopping condition, the training time can be reduced while maintaining the inference accuracy of the obtained first sentence inference model.

[0075] S18, input the fourth training corpus set and the fifth training corpus set into the second sentence inference initial model, fine-tune the second sentence inference initial model until the second sentence inference initial model reaches the preset fourth stopping condition, and obtain the second sentence inference model.

[0076] In practice, the corpora from the fourth and fifth training corpora can be mixed together and used as input to the initial model for second sentence inference. Since each corpus in both sets contains corresponding annotation information, the initial model can be trained in a supervised learning manner, enabling the second neural network model to classify the input corpus and output corresponding discrete values ​​to characterize whether the input corpus is semantically ambiguous or grammatically incorrect. Furthermore, by setting a fourth stopping condition, the training time can be reduced while maintaining the inference accuracy of the obtained second sentence inference model.

[0077] S19, Output the trained statement inference model, including the first statement inference model and the second statement inference model.

[0078] In specific implementation, through steps S11 to S18, a first statement inference model and a second statement inference model can be obtained. The first statement inference model can be used to infer whether a statement has an error, and the second statement inference model can be used to infer the specific error type of the statement.

[0079] As can be seen from the above, by adopting the sentence inference scheme in the embodiments of this specification, the preset neural network model is trained by training the training corpus, and the initial sentence inference model obtained by training is fine-tuned by training corpus containing labeled information. Without extracting multiple features of the sentence, a first sentence inference model for inferring whether the sentence is wrong and a second inference sentence for inferring the specific error type of the sentence can be obtained. Therefore, the amount of computation can be reduced and system resources can be saved.

[0080] It should be noted that there is no necessary order between some steps in the above embodiments. They can be executed simultaneously or sequentially without causing contradictions, and the order can be changed. For example, when actually executing the steps of the statement inference training method provided in this specification, steps S11 and S12 can be executed simultaneously, and S13 and S14 can be executed simultaneously. This specification does not impose specific restrictions on the order of steps in the embodiments.

[0081] In practice, since the number of semantically ambiguous training corpora is small, while training a neural network model requires a large amount of corpus, if a small amount of semantically ambiguous training corpus is used to train the first neural network model directly, the model trained will have poor generality and may be difficult to infer whether the error type of spoken expression belongs to semantic ambiguity or grammatical error. Alternatively, all the error types of spoken expression obtained may be grammatical errors. Therefore, it is necessary to construct a large amount of semantically ambiguous corpus.

[0082] In practice, a large amount of semantically ambiguous corpus can be constructed in various ways. For example, a large training set containing correct sentences can be obtained from the network, and a large amount of semantically ambiguous corpus can be generated by lexicographical translation of the training set.

[0083] In the embodiments of this specification, a large amount of semantically ambiguous training corpus can be obtained by following these steps: obtaining a sixth training corpus; inputting the source corpus in the sixth training corpus into a preset translation model for translation processing to obtain the translation result of the target language type corresponding to the source corpus; aligning each segment of the source corpus with its corresponding translation result, and deleting the content of the aligned part of the translation result; inputting the aligned part of the source corpus into the translation model for translation to obtain the corresponding literal translation result; embedding the literal translation result into the translation result after deleting the aligned part to obtain the semantically ambiguous training corpus.

[0084] By acquiring the sixth training corpus and performing translation, alignment, and embedding of the literal translation results, the amount of data in the semantically ambiguous training corpus can be expanded. Consequently, when training the first neural network model and the second neural network model, the inference accuracy of the first and second sentence inference initial models can be improved.

[0085] As a specific example, if the semantically ambiguous training corpus of the target language type is English, and the sixth training corpus is Chinese, then the trained Chinese-English translation model can be obtained, and the sixth training corpus can be translated.

[0086] If one of the corpora in the sixth training corpus is U: {I like eating apples very much.}, after translation by the Chinese-English translation model, the translation result V: {I like eating apples very much.} can be obtained.

[0087] Select one of the segments U1: {I very.}, and use the alignment tool to align the segment U1 with the translation result V. Among them, "我 (I)" corresponds to "I", "非常 (very much)" corresponds to "very much", and delete "I" and "very much" in the translation result V to get V1: {like eating apples.}

[0088] Input "我 (I)" and "非常 (very much)" into the Chinese-English translation model for translation respectively, and get the translation results "me", "I" corresponding to "我 (I)", and get the translation results "very", "a lot of", "extremely" corresponding to "非常 (very much)".

[0089] Embed the obtained literal translation results into V1 to get the semantically unclear training corpora W1: {I very like eating apples.}, W2: {I a lot of like eating apples.}, W3: {I extremely like eating apples.}, W4: {me very like eating apples.}, etc.

[0090] It can be understood that the above method of obtaining a large number of semantically unclear training corpora is only an example. In specific implementation, according to actual needs, a translation model and the sixth training corpus adapted to the target language type can be selected.

[0091] In specific implementation, corresponding operations can also be directly performed on the first training corpus to generate a large number of semantically unclear corpora. In the embodiments of this specification, the following at least one method can be used to obtain the semantically unclear training corpus:

[0092] 1) Exchange the word order of the corresponding corpora in the first training corpus;

[0093] 2) Delete some words of the corresponding corpora in the first training corpus;

[0094] 3) Remove the segments at the preset positions of each corpus in the first training corpus with the same probability.

[0095] For example, remove the segments in the second half of each corpus in the first training corpus.

[0096] In some embodiments of this specification, statements that are deemed semantically ambiguous by online users can also be obtained as semantically ambiguous training corpus.

[0097] After obtaining the first training corpus and the second training corpus, and before inputting them into the preset first neural network model and the second neural network model, word segmentation can be performed on them respectively to obtain word vectors corresponding to each corpus in the first training corpus and the second training corpus, and then the obtained word vectors can be input into the neural network model.

[0098] Using a first training corpus set and a second training corpus set that have undergone word segmentation, the first neural network model and the second neural network model can be trained to obtain a first initial inference model for inferring whether the input corpus is erroneous and a second initial inference model for inferring the specific error type of the input corpus.

[0099] In the embodiments of this specification, the input corpus can be subjected to two binary classification inferences to determine whether the input corpus is erroneous and the specific type of error. The first classification inference infers whether the input corpus contains an error, and if an error is found, a second classification inference is performed to infer whether the error type is semantic ambiguity or grammatical error. Therefore, in the embodiments of this specification, two neural network models can be trained using a first training corpus set and a second training corpus set. Moreover, in order to speed up the training process and shorten the training time, the two neural network models can have the same architecture and only need to be trained with different training corpora to obtain a model that can perform two binary classification inferences on the input corpus.

[0100] The following detailed explanation, with reference to the accompanying drawings and specific examples, details the process of obtaining the initial model from the first statement in the embodiments of this specification.

[0101] Reference Figure 2 The flowchart shown in this specification illustrates an embodiment of obtaining an initial model for inferring a first statement. In some embodiments of this specification, the first neural network model includes a plurality of sequentially coupled coding layers and a fully connected layer coupled to the last coding layer.

[0102] The process of training the first neural network model until it reaches a preset first stopping condition to obtain the initial model for first statement inference can be performed in the following steps:

[0103] S21, the first training corpus set and the second training corpus set are sequentially input into multiple encoding layers of the first neural network model to obtain corresponding encoding vectors, and the first encoding vector in the encoding vector is output by the last encoding layer to the fully connected layer to obtain the two-dimensional vector corresponding to the encoding vector.

[0104] In specific implementation, the corpora in the first training corpus and the corpora in the second training corpus can be mixed together and used as the input of the first neural network model. The first neural network model encodes each corpus in the first training corpus and the second training corpus multiple times, and the last encoding layer inputs the first encoded vector to the fully connected layer. The first encoded vector can be used as the feature vector representation of the entire corpus. The fully connected layer performs operations on the encoded vectors of each corpus to obtain the corresponding two-dimensional vector [a, b], where a + b = 1, and a and b are both in the range (0, 1).

[0105] In practice, each corpus is encoded multiple times through multiple encoding layers of the first neural network model, resulting in an encoded vector in the form of: [CLS][t1][t2]…[t i ]…[t n [SEP], where [t1][t2]…[t i ]…[t n [] represents each token corresponding to each corpus. [CLS] is located at the first position of the entire encoding vector and is used to characterize the feature representation of the entire encoding vector. [SEP] is located at the last position of the entire encoding vector and is used to characterize the end position of the corpus. i≤n and i≥1.

[0106] It is understandable that when training the first neural network model using the first training corpus and the second training corpus, the first training corpus and the second training corpus, which meet a preset number, can be randomly selected from the training corpus to train the first neural network model.

[0107] S22, calculate the cross-entropy between the two-dimensional vector and the one-hot vectors of the first training corpus and the second training corpus, and use it as the loss function.

[0108] S23, minimize the loss function, update the parameters of the first neural network model, and stop updating the parameters of the first neural network model when the first neural network model reaches the first stopping condition, to obtain the initial model for the first statement inference.

[0109] In practice, gradient descent or backpropagation methods can be used to fine-tune the parameters of the first neural network model to reduce the value of the loss function. When the first neural network model reaches the first stopping condition, training of the first neural network model is stopped, and the initial model for the first statement inference is obtained.

[0110] In practice, the timing for stopping training the first neural network model can be determined by setting the number of training iterations. Specifically, if the number of training iterations for the first neural network model has not reached the preset number, training continues until the preset number of training iterations is reached.

[0111] In another specific implementation, the timing for stopping the training of the first neural network model can be determined based on the test results of the trained first neural network model. Specifically, a first verification corpus set and a second verification corpus set are obtained to verify whether the first neural network model and the second neural network model have been trained successfully, wherein the corpus in the first verification corpus set and the second verification corpus set are corpora of the target language type; the first verification corpus set and the second verification corpus set are continuously input into the first neural network model multiple times to obtain multiple two-dimensional vectors corresponding to the first verification corpus set and the second verification corpus set; if the result obtained by the traversal process of the first preset number of iterations meets the first preset condition, the training of the first neural network model is stopped to obtain the initial model for the first sentence inference.

[0112] By inputting the first and second validation corpora into the first neural network model, and stopping the training of the first neural network when the two-dimensional vectors obtained from multiple consecutive traversals are the same, the training time can be reduced while ensuring the inference accuracy of the initial model for inferring the first sentence.

[0113] As a specific example, in the embodiments of this specification, each corpus in the first and second verification corpora uses one-hot encoding. The first neural network model is verified m times using the first and second verification corpora, and the learning rate of the first neural network model is set to le during each verification. -5 .

[0114] If the results obtained in the m-th, m-1-th, and m-2-th verifications are the same, for example, if the results of the m-th, m-1-th, and m-2-th verifications of the first neural network model are all 1 or 0, then training of the first neural network model is stopped; or, if the results obtained in the m-th, m-1-th, and m-2-th verifications are not an improvement over the results of the previous m-3 verifications, for example, if the results of the m-th, m-1-th, and m-2-th verifications are not better than the best result among the previous m-3 verifications, then training of the first neural network model is stopped, where m is an integer greater than 3.

[0115] In practice, since the first training corpus and the second training corpus are mixed together and used as input to the first neural network model, the obtained first verification corpus and second verification corpus can include verification corpus of correct statements, verification corpus of semantically ambiguous statements, and verification corpus of grammatically incorrect statements.

[0116] Understandably, in practice, different first stopping conditions can be selected according to actual needs to obtain the first statement inference initial model that meets the requirements.

[0117] In the embodiments of this specification, when the first training corpus set and the second training corpus set are obtained, a portion of the corpus in the first training corpus set and the second training corpus set can be used as verification corpus. The initial model for inferring the first sentence obtained by training can be tested using the verification corpus set. If the test result meets the expected result, the training ends; otherwise, the first neural network model can continue to be trained using the first training corpus set and the second training corpus set.

[0118] As can be seen from the above, the annotation information of the first training corpus and the second training corpus may be inaccurate. If the initial model for inferring the first sentence obtained from training is directly used to infer the input corpus, incorrect results may be obtained. For example, if the input corpus is incorrect, the inference result may indicate that the input corpus is correct.

[0119] Based on this, in order to improve the accuracy of the initial model for inferring the first sentence, the initial model for inferring the first sentence can be fine-tuned using a third training corpus, a fourth training corpus, and a fifth training corpus containing labeled information.

[0120] Reference Figure 3 The flowchart shown in this embodiment of the specification illustrates a fine-tuning process for the initial model of the first statement inference. Specifically, in this embodiment, the process can be performed according to the following steps:

[0121] S31, the third training corpus set, the fourth training corpus set, and the fifth training corpus set are sequentially input into multiple encoding layers of the first neural network model to obtain corresponding encoding vectors, and the first encoding vector in the encoding vector is output by the last encoding layer to the fully connected layer to obtain the two-dimensional vector corresponding to the encoding vector.

[0122] S32, calculate the cross-entropy between the two-dimensional vector and the one-hot vectors of the third training corpus, the fourth training corpus, and the fifth training corpus, and use it as the loss function.

[0123] S33, minimize the loss function, update the parameters of the initial model for the first statement inference, and stop updating the parameters of the initial model for the first statement inference when the initial model for the first statement inference reaches the third stopping condition, thus obtaining the first statement inference model.

[0124] The specific implementation process of steps S31 to S33 can refer to the aforementioned process of training the first neural network model using the first training corpus and the second training corpus to obtain the initial model for first sentence inference, and will not be elaborated further here. The difference lies in that, to reduce fluctuations in model parameters during fine-tuning, the learning rate during fine-tuning should be lower than the learning rate during training. For example, during fine-tuning, the model's learning rate can be set to 1e. -6 .

[0125] It is understandable that during the fine-tuning of the initial model for the first statement inference, the timing for stopping the fine-tuning can be determined. For the specific implementation method, please refer to the scheme for determining the timing for stopping the training of the first neural network model in the aforementioned embodiment, which will not be repeated here.

[0126] After the above series of training, verification, and fine-tuning processes, a first statement inference model can be obtained to infer whether the input corpus is an erroneous statement. However, the first statement inference model cannot determine the specific error type when the input statement is an erroneous statement. Therefore, in specific implementation, it is also necessary to train a second statement inference model that can determine the specific error type when the first statement inference model infers that the input corpus is an erroneous statement.

[0127] In the embodiments of this specification, the second neural network model can be trained using a second training corpus to obtain an initial model for second sentence inference, and the initial model for second sentence inference can be fine-tuned using a fourth training corpus and a fifth training corpus containing labeled information to obtain a second sentence inference model.

[0128] In some embodiments of this specification, the architecture of the second neural network model is the same as that of the first neural network model, both including multiple coding layers coupled in sequence and a fully connected layer coupled to the last coding layer;

[0129] The step of training the second neural network model until it reaches a second preset stopping condition to obtain the initial model for second statement inference includes:

[0130] The second training corpus is sequentially input into multiple encoding layers of the second neural network model to obtain corresponding encoding vectors. The first encoding vector in the encoding vector is output from the last encoding layer to the fully connected layer to obtain a two-dimensional vector corresponding to the encoding vector. The cross-entropy between the two-dimensional vector and the one-hot vector of the second training corpus is calculated and used as a loss function. The loss function is minimized to update the parameters of the second neural network model. When the second neural network model reaches the second stopping condition, the update of the parameters of the second neural network model is stopped to obtain the initial model for the second sentence inference.

[0131] The detailed process of training the second neural network model using the second training corpus can be referred to the aforementioned process of training the first neural network model using the first and second training corpora to obtain the initial model for first sentence inference, and will not be described again here. The difference is that the corpus input to the second neural network model is only the second training corpus.

[0132] In practice, when the second neural network model is trained a certain number of times, its inference accuracy tends to be the same. Even if the second neural network model is trained many more times, its inference accuracy will not improve. Therefore, in the embodiments of this specification, it is necessary to determine when to stop training the second neural network model.

[0133] Specifically, a third verification corpus is obtained to verify whether the second neural network model has been trained, wherein the corpus in the third verification corpus is the corpus of the target language type; the third verification corpus is continuously input into the second neural network model multiple times to obtain multiple two-dimensional vectors corresponding to the third verification corpus; if the result obtained by the traversal process of the second preset number of consecutive iterations meets the second preset condition, the training of the second neural network model is stopped to obtain the initial model for the second sentence inference.

[0134] The specific implementation process of determining the cessation of training the second neural network model using the third verification corpus is described in the aforementioned embodiment regarding the scheme for determining the cessation of training the first neural network model. The difference lies in that, when using the third verification corpus to verify the second neural network model, the consecutive preset second number and the second preset condition for determining the cessation of training can be different from or the same as the consecutive preset first number and the first preset condition used when using the first and second verification corpora to verify the second neural network model.

[0135] In specific implementation, the third verification corpus may include semantically ambiguous corpus and grammatically incorrect corpus.

[0136] By inputting the third verification corpus into the second neural network model, and stopping the training of the second neural network model when the two-dimensional vectors obtained from multiple consecutive traversals are the same, the training time can be reduced while ensuring the inference accuracy of the initial model for inferring the second sentence.

[0137] In other embodiments of this specification, other methods may be used to determine when to stop training the second neural network model. For specific methods and their implementation processes, please refer to the scheme for determining when to stop training the first neural network model in the foregoing embodiments.

[0138] As can be seen from the above, since the annotation information of the second training corpus is inaccurate, if the initial model for inferring the second sentence obtained from the training is directly used to infer the input corpus, it may produce incorrect results. For example, if the input corpus is semantically ambiguous, the inference result indicates that the input corpus is grammatically incorrect, which makes it impossible to provide correct guidance based on the error type.

[0139] Based on this, in order to improve the accuracy of the initial model for second sentence inference, the trained initial model for second sentence inference can be fine-tuned using a fourth training corpus and a fifth training corpus containing labeled information.

[0140] In the embodiments described in this specification, the initial model for inferring the second statement can be fine-tuned according to the following steps:

[0141] The fourth and fifth training corpora are sequentially input into multiple encoding layers of the second sentence inference initial model to obtain corresponding encoding vectors. The first encoding vector in the encoding vectors is output from the last encoding layer to the fully connected layer to obtain the two-dimensional vector corresponding to the encoding vector. The cross-entropy between the two-dimensional vector and the one-hot vectors of the fourth and fifth training corpora is calculated and used as the loss function. The loss function is minimized to update the parameters of the second sentence inference initial model. When the second sentence inference initial model reaches the fourth stopping condition, the updating of the parameters of the second sentence inference initial model is stopped, thus obtaining the second sentence inference model.

[0142] The fine-tuning process of the initial model for second sentence inference using the fourth and fifth training corpora can be referenced from the process in the aforementioned embodiment of fine-tuning the initial model for first sentence inference using the third, fourth, and fifth training corpora to obtain the initial model for first sentence inference. This will not be elaborated further here. The difference lies in that the corpora input to the model are the fourth and fifth training corpora.

[0143] It is understandable that during the fine-tuning of the initial model for the second statement inference, the timing for stopping the fine-tuning of the second initial model for inference can be determined. For the specific implementation method, please refer to the scheme for determining the timing for stopping the fine-tuning of the first initial model for inference in the aforementioned embodiment.

[0144] By training, validating, and fine-tuning neural network models using different input corpora, a first sentence inference model that distinguishes whether the input corpus is erroneous and a second sentence inference model that distinguishes the specific error type of the input corpus can be obtained. Using the first sentence inference model and the second sentence inference model, inference can be performed on the input corpus.

[0145] Reference Figure 4 The flowchart shown in this specification illustrates a statement inference method. In some embodiments of this specification, the method specifically includes the following steps:

[0146] S41, Obtain the statement to be inferred.

[0147] In practice, the statement to be inferred can be a corpus of spoken expressions converted by a speech recognition device.

[0148] S42, input the statement to be inferred into the statement inference model for inference processing and output the inference result.

[0149] The statement inference model includes a first statement inference model and a second statement inference model, and is obtained using the statement inference training method described in any of the foregoing embodiments.

[0150] The inference process and output of the inference result include:

[0151] The statement to be inferred is input into the first statement inference model to obtain a first inference result. If the first inference result is of the first type, the first inference result is used as the inference result and output.

[0152] If the first inference result is of the second type, the statement to be inferred is input into the second statement inference model to obtain the second inference result, and the second inference result is used as the inference result and output.

[0153] In specific implementation, the statement inference model may include a first statement inference model and a second statement inference model obtained by training a preset neural network model using the statement inference training method described in any of the foregoing embodiments.

[0154] When outputting the inference result of the statement to be inferred, the statement to be inferred can be input into the first statement inference model to obtain the two-dimensional vector [a1, b1] of the statement to be inferred, and a1+b1=1. Based on the magnitude of a1 and b1, the first inference result of the corresponding type is output.

[0155] If b1 in the two-dimensional vector [a1, b1] is not less than a1, it indicates that the statement to be inferred is a correct statement, and the first inference result is of the first type, which is output as the inference result. If b1 in the two-dimensional vector [a1, b1] is less than a1, it indicates that the statement to be inferred is an incorrect statement, and the first inference result is of the second type. In this case, the statement to be inferred needs to be input into the second statement inference model to obtain the two-dimensional vector [a2, b2] of the statement to be inferred, and the corresponding second inference result is output according to the size of a2 and b2.

[0156] If b2 in the two-dimensional vector [a2, b2] is not less than a2, the second inference result is obtained that the statement to be inferred is a grammatically incorrect statement, and is output as the inference result; if b2 in the two-dimensional vector [a2, b2] is not less than a2, it means that the second inference result is obtained that the statement to be inferred is a grammatically unclear statement, and is output as the inference result.

[0157] By using the above statement inference method, the statement to be inferred can be input into the trained first statement inference model and second statement inference model to infer whether there is an error and the specific type of error. There is no need to extract multiple features, thus reducing the amount of computation and saving system resources.

[0158] Accordingly, embodiments of this specification also provide an apparatus corresponding to the training method for the above statement inference, which will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0159] Reference Figure 5 The diagram shown is a structural schematic of a sentence inference training device according to an embodiment of this specification. In this embodiment, the sentence inference training device 50 may include:

[0160] The first corpus acquisition module 51 is adapted to acquire a first training corpus set and a second training corpus set of the target language type, respectively, wherein the second training corpus set includes a semantically ambiguous training corpus set and a grammatically incorrect training corpus set;

[0161] The first training module 52 is adapted to input the first training corpus set and the second training corpus set into a preset first neural network model, train the first neural network model until the first neural network model reaches a preset first stopping condition, and obtain the first sentence inference initial model;

[0162] The second training module 53 is adapted to input the second training corpus into a preset second neural network model, train the second neural network model until the second neural network model reaches a preset second stopping condition, and obtain the initial model for second sentence inference.

[0163] The second corpus acquisition module 54 is adapted to acquire a third training corpus set, a fourth training corpus set, and a fifth training corpus set containing annotation information, respectively, wherein: the third training corpus set includes correct corpus; the fourth training corpus set includes semantically unclear corpus; and the fifth training corpus set includes grammatically incorrect corpus.

[0164] The first fine-tuning unit 55 is adapted to input the third training corpus set, the fourth training corpus set, and the fifth training corpus set into the first sentence inference initial model, fine-tune the first sentence inference initial model until the first sentence inference initial model reaches the preset third stopping condition, and obtain the first sentence inference model;

[0165] The second fine-tuning unit 56 is adapted to input the fourth training corpus set and the fifth training corpus set into the second sentence inference initial model, fine-tune the second sentence inference initial model until the second sentence inference initial model reaches the preset fourth stopping condition, and obtain the second sentence inference model.

[0166] The inference model output unit 57 is adapted to output the trained statement inference model, including the first statement inference model and the second statement inference model.

[0167] The specific implementation of each module of the above statement inference training device can be found in the detailed descriptions of the corresponding embodiments above, and will not be elaborated here.

[0168] Accordingly, embodiments of this specification also provide apparatus corresponding to the above statement inference method, which will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0169] Reference Figure 6 The diagram shown is a structural schematic of a statement inference device according to an embodiment of this specification. In this embodiment, the statement inference device 60 may include:

[0170] The statement to be inferred acquisition unit 61 is adapted to acquire the statement to be inferred.

[0171] The processing unit 62 is adapted to input the statement to be inferred into the statement inference model for inference processing and output the inference result.

[0172] The statement inference model includes a first statement inference model and a second statement inference model;

[0173] The inference process and output of the inference result include:

[0174] The statement to be inferred is input into the first statement inference model to obtain a first inference result. If the first inference result is of the first type, the first inference result is used as the inference result and output.

[0175] If the first inference result is of the second type, the statement to be inferred is input into the second statement inference model to obtain the second inference result, and the second inference result is used as the inference result and output.

[0176] By using the above-mentioned statement inference device, the statement to be inferred can be input into the trained first statement inference model and second statement inference model to infer whether there is an error and the specific type of error. There is no need to extract multiple features, thus reducing the amount of computation and saving system resources.

[0177] This specification also provides an electronic device for inferring whether a statement is an erroneous statement and, if so, the specific type of error. For example... Figure 7 As shown, the electronic device 70 may include a memory 71 and a processor 72, wherein the memory 71 is adapted to store one or more computer instructions, and when the processor 72 executes the computer instructions, it performs the steps of the training method for statement inference described in any of the foregoing embodiments, or performs the steps of the statement inference method described in any of the foregoing embodiments.

[0178] In specific implementation, such as Figure 7 As shown, the electronic device 70 may also include an expansion interface 73, which is adapted to connect with other devices to realize data interaction.

[0179] Specifically, electronic device 70 can be a general-purpose or special-purpose computer device, or more specifically, a server or computer terminal, such as a personal computer or portable terminal device.

[0180] In a practical implementation, the memory 71, processor 72, and expansion interface 73 can be connected via a bus.

[0181] In specific implementations, the processor can be implemented by processing chips such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA), or by an application-specific integrated circuit (ASIC) or one or more integrated circuits configured to implement the embodiments of this specification.

[0182] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0183] This specification also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed, they can perform the steps of the statement inference training method described in any of the foregoing embodiments, or perform the steps of the statement inference method described in any of the foregoing embodiments. The computer-readable storage medium can be any suitable readable storage medium such as an optical disc, a hard disk drive, or a solid-state drive. The instructions stored on the computer-readable storage medium execute the steps of the statement inference training method described in any of the foregoing embodiments, or perform the steps of the statement inference method described in any of the foregoing embodiments, and will not be described again.

[0184] The computer-readable storage medium may include, for example, any suitable type of memory cell, memory device, memory article, memory medium, storage device, storage article, storage medium and / or storage cell, such as memory, removable or non-removable medium, erasable or non-erasable medium, writable or rewritable medium, digital or analog medium, hard disk, floppy disk, optical disc read-only memory (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), optical disc, magnetic medium, magneto-optical medium, removable memory card or disk, various types of digital universal optical disc (DVD), magnetic tape, cassette tape, etc.

[0185] Computer instructions may include any suitable type of code implemented using any appropriate high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, etc.

[0186] While the embodiments disclosed in this specification are as described above, they are not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the embodiments in this specification. Therefore, the scope of protection of the embodiments in this specification should be determined by the scope defined in the claims.

Claims

1. A training method of sentence inference, characterized by, The method comprises the following steps: obtaining a first training corpus and a second training corpus of a target language type respectively, wherein the second training corpus comprises a semantic ambiguity training corpus and a grammatical error training corpus; inputting the first training corpus and the second training corpus into a preset first neural network model, training the first neural network model until the first neural network model reaches a preset first stop condition, and obtaining a first sentence inference initial model; inputting the second training corpus into a preset second neural network model, training the second neural network model until the second neural network model reaches a preset second stop condition, and obtaining a second sentence inference initial model; obtaining a third training corpus, a fourth training corpus and a fifth training corpus containing annotation information respectively, wherein the third training corpus comprises correct corpus, the fourth training corpus comprises semantic ambiguity corpus, and the fifth training corpus comprises grammatical error corpus; inputting the third training corpus, the fourth training corpus and the fifth training corpus into the first sentence inference initial model, fine-tuning the first sentence inference initial model until the first sentence inference initial model reaches a preset third stop condition, and obtaining a first sentence inference model; and inputting the fourth training corpus and the fifth training corpus into the second sentence inference initial model, fine-tuning the second sentence inference initial model until the second sentence inference initial model reaches a preset fourth stop condition, and obtaining a second sentence inference model; outputting the trained sentence inference model, including the first sentence inference model and the second sentence inference model.

2. The training method of claim 1, wherein, The method for obtaining the semantic ambiguity training corpus in the second training corpus comprises the following steps: obtaining a sixth training corpus; inputting source corpus in the sixth training corpus into a preset translation model for translation processing to obtain a translation result of the source corpus in the target language type; performing alignment operation on each segment of the source corpus and the corresponding translation result, and deleting the aligned part of the translation result; inputting the source corpus in the aligned part into the translation model for translation to obtain a corresponding direct translation result; embedding the direct translation result into the translation result after deleting the aligned part to obtain the semantic ambiguity training corpus.

3. The training method of claim 1, wherein, The method for obtaining the semantic ambiguity training corpus in the second training corpus further comprises at least one of the following methods: swapping the word order of the corresponding corpus in the first training corpus; deleting part of the words of the corresponding corpus in the first training corpus; removing the segment at the preset position of each corpus in the first training corpus with the same probability.

4. Training method according to any of claims 1-3, characterized in that, The first neural network model comprises a plurality of encoding layers coupled in sequence and a fully connected layer coupled with the last encoding layer; the method for training the first neural network model until the first neural network model reaches a preset first stop condition to obtain a first sentence inference initial model comprises the following steps: The first training corpus and the second training corpus are sequentially input into a plurality of encoding layers of the first neural network model to obtain corresponding encoding vectors, and a first encoding vector in the encoding vectors is output from the last encoding layer to the fully connected layer to obtain a two-dimensional vector corresponding to the encoding vectors; The cross entropy of the two-dimensional vector and the one-hot vector of the first training corpus and the second training corpus is calculated as a loss function; The loss function is minimized, the parameters of the first neural network model are updated, and when the first neural network model reaches a first stopping condition, the updating of the parameters of the first neural network model is stopped to obtain the first sentence inference initial model.

5. The training method of claim 4, wherein, Also includes: Obtaining a first validation corpus and a second validation corpus for verifying whether the first neural network model is trained, wherein the corpora in the first validation corpus and the second validation corpus are corpora of the target language type; The first neural network model reaches a first stopping condition, the parameters of the first neural network model are stopped, and the first sentence inference initial model is obtained, including: The first validation corpus and the second validation corpus are sequentially input into the first neural network model for a plurality of times to obtain a plurality of two-dimensional vectors corresponding to the first validation corpus and the second validation corpus; If the result obtained in the continuous preset first number of traversal processes meets a first preset condition, the training of the first neural network model is stopped to obtain the first sentence inference initial model.

6. The training method of claim 4, wherein, The first sentence inference initial model is fine-tuned until the first sentence inference initial model reaches a preset third stopping condition to obtain a first sentence inference model, including: The third training corpus, the fourth training corpus and the fifth training corpus are sequentially input into a plurality of encoding layers of the first neural network model to obtain corresponding encoding vectors, and a first encoding vector in the encoding vectors is output from the last encoding layer to the fully connected layer to obtain a two-dimensional vector corresponding to the encoding vectors; The cross entropy of the two-dimensional vector and the one-hot vector of the third training corpus, the fourth training corpus and the fifth training corpus is calculated as a loss function; The loss function is minimized, the parameters of the first sentence inference initial model are updated, and when the first sentence inference initial model reaches a third stopping condition, the updating of the parameters of the first sentence inference initial model is stopped to obtain the first sentence inference model.

7. The training method according to any one of claims 1 to 3, characterized in that, The second neural network model includes a plurality of encoding layers and a fully connected layer coupled with the last encoding layer; The second neural network model is trained until the second neural network model reaches a preset second stopping condition to obtain a second sentence inference initial model, including: inputting the second training corpus into multiple encoding layers of the second neural network model in sequence to obtain corresponding encoding vectors, and outputting a first encoding vector in the encoding vectors from a last encoding layer to the fully connected layer to obtain a two-dimensional vector corresponding to the encoding vectors; calculating cross-entropy of the two-dimensional vector and a one-hot vector of the second training corpus as a loss function; minimizing the loss function, updating parameters of the second neural network model, and stopping updating the parameters of the second neural network model when the second neural network model reaches a second stopping condition to obtain the second sentence inference initial model.

8. The training method of claim 7, wherein, Further comprising: obtaining a third validation corpus for verifying whether the second neural network model is trained, wherein the corpus in the third validation corpus is a corpus of the target language type; stopping updating the parameters of the second neural network model when the second neural network model reaches a second stopping condition to obtain the second sentence inference initial model, comprising: inputting the third validation corpus into the second neural network model for multiple times in sequence to obtain multiple two-dimensional vectors corresponding to the third validation corpus; stopping training the second neural network model when a result obtained in a continuous preset second number of traversal processes meets a second preset condition to obtain the second sentence inference initial model.

9. The training method of claim 7, wherein, fine-tuning the second sentence inference initial model until the second sentence inference initial model reaches a preset fourth stopping condition to obtain a second sentence inference model, comprising: inputting the fourth training corpus and the fifth training corpus into multiple encoding layers of the second sentence inference initial model in sequence to obtain corresponding encoding vectors, and outputting a first encoding vector in the encoding vectors from a last encoding layer to the fully connected layer to obtain a two-dimensional vector corresponding to the encoding vectors; calculating cross-entropy of the two-dimensional vector and a one-hot vector of the fourth training corpus and the fifth training corpus as a loss function; minimizing the loss function, updating parameters of the second sentence inference initial model, and stopping updating the parameters of the second sentence inference initial model when the second sentence inference initial model reaches a fourth stopping condition to obtain the second sentence inference model.

10. A method of sentence inference, the method comprising: comprising: obtaining a sentence to be inferred; inputting the sentence to be inferred into a sentence inference model for inference processing and outputting an inference result; wherein the sentence inference model comprises a first sentence inference model and the second sentence inference model, and is obtained by using the sentence inference training method in any one of claims 1-9; the inference processing and the outputting of the inference result, comprising: inputting the sentence to be inferred into the first sentence inference model to obtain a first inference result, and if the first inference result is of a first type, taking the first inference result as the inference result and outputting; if the first inference result is of a second type, inputting the sentence to be inferred into the second sentence inference model to obtain a second inference result, and taking the second inference result as the inference result and outputting. 11.A sentence inference training apparatus, comprising: a first corpus obtaining unit adapted to obtain a first training corpus and a second training corpus of a target language type respectively, wherein the second training corpus comprises a semantic ambiguous training corpus and a grammatical error training corpus; a first training unit adapted to input the first training corpus and the second training corpus into a preset first neural network model, train the first neural network model until the first neural network model reaches a preset first stop condition, and obtain a first sentence inference initial model; a second training unit adapted to input the second training corpus into a preset second neural network model, train the second neural network model until the second neural network model reaches a preset second stop condition, and obtain a second sentence inference initial model; a second corpus obtaining unit adapted to obtain a third training corpus, a fourth training corpus and a fifth training corpus containing annotation information respectively, wherein the third training corpus comprises correct corpus, the fourth training corpus comprises semantic ambiguous corpus, and the fifth training corpus comprises grammatical error corpus; a first fine-tuning unit adapted to input the third training corpus, the fourth training corpus and the fifth training corpus into the first sentence inference initial model, fine-tune the first sentence inference initial model until the first sentence inference initial model reaches a preset third stop condition, and obtain a first sentence inference model; a second fine-tuning unit adapted to input the fourth training corpus and the fifth training corpus into the second sentence inference initial model, fine-tune the second sentence inference initial model until the second sentence inference initial model reaches a preset fourth stop condition, and obtain a second sentence inference model; a inference model output unit adapted to output a trained sentence inference model comprising the first sentence inference model and the second sentence inference model. 12.A sentence inference apparatus, comprising: a sentence to be inferred obtaining unit adapted to obtain a sentence to be inferred; a processing unit adapted to input the sentence to be inferred into a sentence inference model for inference processing and output an inference result; wherein the sentence inference model comprises a first sentence inference model and a second sentence inference model, and the first sentence inference model and the second sentence inference model are obtained by using the training method of the sentence inference according to any one of claims 1-9; the inference processing and the output of the inference result comprise: inputting the sentence to be inferred into the first sentence inference model to obtain a first inference result, and if the first inference result is of a first type, taking the first inference result as the inference result and outputting the inference result; if the first inference result is of a second type, inputting the sentence to be inferred into the second sentence inference model to obtain a second inference result, and taking the second inference result as the inference result and outputting the inference result.

13. An electronic device comprising a memory and a processor, wherein, The memory is adapted to store one or more computer instructions, wherein the processor, when running the computer instructions, executes the steps of the training method of the sentence inference according to any one of claims 1 to 9, or executes the steps of the sentence inference method according to claim 10.

14. A computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions, when running, execute the steps of the training method of the sentence inference according to any one of claims 1 to 9, or execute the steps of the sentence inference method according to claim 10.

Images