Grammar knowledge prediction method and device, electronic equipment and storage medium

Abstract

The application discloses a syntax knowledge prediction method and device, electronic equipment and a storage medium, and belongs to the technical field of natural language processing, wherein the syntax knowledge prediction method comprises the following steps: acquiring a natural language text and position information of a target word in the natural language text; performing vectorization processing on each sentence in the natural language text to obtain a first feature vector corresponding to each word in the natural language text; performing dependency syntax analysis on each sentence in the natural language text to obtain dependency syntax information corresponding to each word in the natural language text, and performing vectorization processing on the dependency syntax information to obtain a dependency feature vector corresponding to each word in the natural language text; fusing the first feature vector and the dependency feature vector to obtain a fused vector; and performing syntax knowledge classification based on the fused vector and the position information of the target word to obtain syntax knowledge corresponding to the target word.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, electronic device, and storage medium for predicting grammatical knowledge. Background Technology

[0002] In smart education, grammar prediction is a crucial application of text classification, applicable to scenarios such as student learning analysis and related knowledge point recommendations. Existing grammar prediction solutions utilize text feature-based classification methods. This involves acquiring natural language sentences, specifying a word within the sentence, segmenting, encoding, and vectorizing the sentence, and then passing the resulting feature vector through a classification neural network to assign it to a specific grammar category. However, due to the presence of complex sentence structures such as clauses, inversions, and nested sentences, the grammar of complex natural language sentences is often difficult to predict. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, and storage medium for predicting grammatical knowledge, in order to solve the problem of the difficulty in predicting the grammar of complex sentence structures in natural language.

[0004] This invention provides a method for predicting grammatical knowledge, comprising:

[0005] Obtain natural language text and the location information of target words in the natural language text;

[0006] Each sentence in the natural language text is vectorized to obtain the first feature vector corresponding to each word in the natural language text.

[0007] Dependency parsing is performed on each sentence in the natural language text to obtain the dependency parsing information corresponding to each word in the natural language text. The dependency parsing information is then vectorized to obtain the dependency feature vector corresponding to each word in the natural language text.

[0008] The first feature vector and the dependency feature vector are fused to obtain a fused vector. Based on the fused vector and the position information of the target word, grammatical knowledge is classified to obtain the grammatical knowledge corresponding to the target word.

[0009] In some embodiments, the vectorization process of each sentence in the natural language text to obtain a first feature vector corresponding to each word in the natural language text includes:

[0010] Each sentence in the natural language text is vectorized sequentially to obtain the word vector and position encoding vector corresponding to each word in the natural language text.

[0011] In some embodiments, the dependency syntax information includes: the position information of another word that the word grammatically depends on in the sentence, and the relationship between the word and the other word that the word grammatically depends on;

[0012] The vectorization process of the dependency syntax information to obtain the dependency feature vector corresponding to each word in the natural language text includes:

[0013] The position information in the dependency syntax information is vectorized to obtain the dependency position vector corresponding to each word in the natural language text.

[0014] The relations in the dependency syntax information are vectorized to obtain the dependency relation vector corresponding to each word in the natural language text.

[0015] In some embodiments, the step of classifying grammatical knowledge based on the fused vector and the positional information of the target word to obtain the grammatical knowledge corresponding to the target word includes:

[0016] Feature extraction is performed on the fused vector to obtain a second feature vector;

[0017] Based on the second feature vector and the position information of the target word, the feature vector corresponding to the target word is obtained;

[0018] The feature vectors corresponding to the target word are classified according to grammatical knowledge to obtain the grammatical knowledge corresponding to the target word.

[0019] In some embodiments, classifying the feature vector corresponding to the target word to obtain the grammatical knowledge corresponding to the target word includes:

[0020] The feature vector corresponding to the target word is input into a multi-objective classification layer for grammatical knowledge classification to obtain the grammatical knowledge corresponding to the target word.

[0021] The multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data and the grammatical knowledge corresponding to the target training words in the natural language training text as training labels.

[0022] Alternatively, the multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data, and using the grammatical knowledge and part-of-speech information corresponding to the target training words in the natural language training text as training labels, wherein the part-of-speech information is obtained by performing dependency parsing on the natural language training text.

[0023] In some embodiments, the process of determining the multi-target classification layer includes:

[0024] Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0025] Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0026] Based on the feature vector corresponding to the natural language training text and the position information of the target training word, the feature vector corresponding to the target training word is obtained;

[0027] Determine the grammatical knowledge corresponding to the target training words;

[0028] Using the feature vectors corresponding to the target training words as training data and the grammatical knowledge corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: a grammar classification layer, an error type classification layer, and a phrase classification layer.

[0029] After the initial multi-object classification layer is trained, the multi-object classification layer is obtained.

[0030] In some embodiments, the process of determining the multi-target classification layer includes:

[0031] Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0032] Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0033] Based on the feature vector corresponding to the natural language training text and the position information of the target training word, the feature vector corresponding to the target training word is obtained;

[0034] The grammatical knowledge and part-of-speech information corresponding to the target training word are determined, wherein the part-of-speech information corresponding to the target training word is obtained by dependency parsing;

[0035] Using the feature vectors corresponding to the target training words as training data, and the grammatical knowledge and part-of-speech information corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: a grammar classification layer, an error type classification layer, and a phrase classification layer, as well as a part-of-speech classification layer.

[0036] After the initial multi-objective classification layer is trained, at least one of the syntax classification layer, error type classification layer, and phrase classification layer in the initial multi-objective classification layer is retained to obtain the multi-objective classification layer.

[0037] In some embodiments, training the initial multi-objective classification layer includes:

[0038] The feature vector corresponding to the target training word is input into at least one of the grammar classification layer, error type classification layer and phrase classification layer to obtain the grammar knowledge prediction result, which includes at least one of the grammar prediction result, error type prediction result and phrase prediction result.

[0039] Based on the grammar knowledge prediction results and the grammar knowledge corresponding to the target training word, the first loss function value is calculated using the multi-label cross-entropy loss function;

[0040] Based on the first loss function value, update the parameters of the initial multi-target classification layer.

[0041] The present invention also provides a grammar knowledge prediction device, comprising:

[0042] The acquisition unit is used to acquire natural language text and the location information of target words in the natural language text;

[0043] The text feature extraction unit is used to vectorize each sentence in the natural language text to obtain the first feature vector corresponding to each word in the natural language text.

[0044] The dependency parsing unit is used to perform dependency parsing on each sentence in the natural language text to obtain dependency parsing information corresponding to each word in the natural language text, and to perform vectorization processing on the dependency parsing information to obtain dependency feature vectors corresponding to each word in the natural language text.

[0045] The classification unit is used to fuse the first feature vector and the dependency feature vector to obtain a fused vector, and to perform grammatical knowledge classification based on the fused vector and the position information of the target word to obtain the grammatical knowledge corresponding to the target word.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the syntax knowledge prediction method as described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the syntax knowledge prediction method as described above.

[0048] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the syntax knowledge prediction method as described above.

[0049] The grammar knowledge prediction method, device, electronic device, and storage medium provided by this invention vectorize each sentence in natural language text to obtain a first feature vector corresponding to each word, and perform dependency parsing and vectorization on each sentence to obtain a dependency feature vector corresponding to each word. The dependency feature vector is fused with the first feature vector, enabling the model to understand the relationship between sentence structure and words. Based on the fused vector and the position information of the target word, grammar knowledge is classified, which can predict the grammar knowledge corresponding to the target word in complex sentence natural language text. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the existing BERT text classification process;

[0052] Figure 2 This is a schematic diagram of the grammar knowledge prediction method provided in an embodiment of the present invention;

[0053] Figure 3 Example diagram of dependency syntax information provided in embodiments of the present invention;

[0054] Figure 4 A schematic diagram illustrating the process of predicting and classifying grammatical knowledge based on fused vectors, provided in an embodiment of the present invention;

[0055] Figure 5 This is one of the schematic diagrams of the multi-target classification layer model structure provided in the embodiments of the present invention;

[0056] Figure 6 This is one of the flowcharts illustrating the method for determining a multi-target classification layer provided in an embodiment of the present invention;

[0057] Figure 7 This is the second schematic diagram of the model structure of the multi-target classification layer provided in the embodiments of the present invention;

[0058] Figure 8 This is the second schematic diagram of the method for determining a multi-target classification layer provided in an embodiment of the present invention;

[0059] Figure 9 A schematic diagram illustrating the loss calculation using mask loss to control the loss in the grammar knowledge classification model provided in this embodiment of the invention;

[0060] Figure 10 A schematic diagram of the structure of an apparatus for performing a grammatical knowledge prediction method provided in an embodiment of the present invention;

[0061] Figure 11 A schematic diagram of the grammar knowledge prediction device provided by the present invention;

[0062] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0064] The terms "first," "second," etc., used in this specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0065] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0066] Text classification is a fundamental task in NLP and an important module in text processing, often appearing as an upstream task in specific business operations. Text classification has a wide range of applications, such as syntax prediction, sentiment analysis, intent recognition, and spam classification.

[0067] Text classification models typically consist of multiple encoder layers. These encoder layers extract feature representations of the text and output them as representation vectors. In operation, text input to the classification model passes through multiple encoder layers sequentially. The output of one encoder layer serves as the input to the next, and the output of the final encoder layer becomes the corresponding feature information for text classification. Text classification is then performed by the classification layer. Encoder layers closer to the input are considered lower layers, while those closer to the classification layer are considered higher layers. Lower encoder layers have higher resolution, containing more detailed information and reflecting lexical features more semantically. Higher encoder layers have stronger semantic information but limited resolution, reflecting syntactic features more semantically.

[0068] Bidirectional Encoder Representation from Transformers (BERT) is a pre-trained language model. The goal of BERT is to train on a large-scale corpus to obtain a semantic representation of the text, then fine-tune the semantic representation of the text for a specific NLP task, and finally apply it to that NLP task.

[0069] Existing syntax prediction schemes typically employ text classification methods based on pre-trained language models. The text to be predicted is input into the pre-trained model for feature extraction, and the extracted text features are then used for classification. The pre-trained model usually uses BERT. Figure 1 This is a schematic diagram of the existing BERT text classification process. Figure 1As shown, the input text is: "The way referred to to solve the problem sounds reasonable." The second "to" in the specified text is the target word. A special character [CLS] is appended to the beginning of the input text, and a special character [SEP] is appended to the end. The English text with special characters at the beginning and end is first segmented into individual words or sub-words by a word segmenter. Each word, sub-word, or special character is encoded by an encoder and represented by a number. The input English text is converted into a string of numbers: 101 762 10293825 12752 92 92 52712314 823 15278 1019 6524 1021 102. Then, this string of numbers is input into the BERT model. The model converts each number into a vector. Finally, the vector corresponding to [CLS] at the beginning of the sentence is used as the feature of the entire English text. This vector is passed through a classification neural network and finally classified into a certain grammatical category. The output is the English grammar corresponding to the target word "to" in the English text as "the usage of to, infinitive as an attributive, non-finite verb".

[0070] However, existing grammar prediction schemes classify grammar based on single text features, resulting in poor performance for complex sentence structures. In complex sentences, the words to be examined may be in clauses, and these clauses may have corresponding subject-verb-object structures. Existing schemes are prone to mistakenly treating the structure of a word in a clause as the main structure. Furthermore, based solely on the text's surface appearance, existing schemes struggle to distinguish long and complex sentences from other sentences. For some grammar points, inferences about the relationships between words are required to arrive at correct predictions, but existing models struggle to learn this inference ability solely from text data.

[0071] Using a single model for prediction across multiple tasks is a significant development direction for general-purpose language models. Currently, obtaining data with multiple labels during multi-task training typically requires manual annotation, which is quite challenging, while obtaining data with a single label for any sub-task is relatively simple. Furthermore, existing multi-task training methods suffer from weight imbalance across tasks.

[0072] To address this, the present invention provides a method, apparatus, electronic device, and storage medium for predicting grammatical knowledge. By vectorizing the acquired natural language text, a first feature vector is obtained for each word in the text. Dependency parsing is then performed on the text to obtain the dependency feature vector for each word. The first feature vector and the dependency feature vector are fused. Based on the fused vector and the positional information of the target word, the text is simultaneously classified using multiple tasks to obtain the grammatical knowledge corresponding to the target word. By introducing dependency parsing, the model can understand the relationship between sentence structure and words, enabling accurate prediction of grammatical knowledge for natural language texts with complex sentence structures, thus solving the problem of difficulty in predicting the grammar of complex natural language sentences.

[0073] The grammar knowledge prediction method provided by this invention can be applied to a grammar knowledge prediction device, which can be implemented in hardware or software and can be configured on the terminal device side or the server side. This invention does not limit this.

[0074] It should be noted that the grammatical knowledge classification and grammatical knowledge prediction in the embodiments of the present invention have the same meaning.

[0075] Figure 2 A flowchart illustrating the grammar knowledge prediction method provided in this embodiment of the invention. (See attached diagram.) Figure 2 As shown, a grammar knowledge prediction method is provided, which is then applied to... Figure 2 The method is illustrated using a terminal as an example, including the following steps: steps 210, 220, 230, and 240. These steps are merely one possible implementation of the invention.

[0076] Step 210: Obtain the natural language text and the location information of the target words in the natural language text;

[0077] It's important to clarify that natural language typically refers to a language that evolves naturally with culture. Examples include Chinese, English, and Japanese. However, sometimes all languages ​​used by humans (including those that have evolved naturally with culture, as well as artificial languages) are considered "natural" languages ​​in contrast to "artificial" languages ​​designed for computers, such as programming languages.

[0078] Natural language text can be obtained based on grammatical knowledge classification tasks. For example, to classify / predict English grammatical knowledge, English text is obtained.

[0079] In NLP, the finest granularity is the word. Words form sentences, and sentences form paragraphs, chapters, documents, etc. Therefore, in this embodiment of the invention, natural language text includes at least one sentence. A sentence includes at least one word. It should be noted that in this embodiment of the invention, the terms "word," "lexicon," and "vocabulary" have the same meaning and can be used interchangeably.

[0080] It should be noted that the target word is the word to be examined in the natural language text, which may include at least one of the following types of words: verb, noun, adjective, preposition, pronoun, etc.

[0081] For example, one or more verbs in a sentence can be designated as target words; another example is designating a verb and a noun in a sentence as target words; yet another example is designating a verb, a noun, and a preposition in a sentence as target words. Target words can also be words specified by other users for examination, or words with grammatical errors, etc., which will not be listed here.

[0082] The location information of a target word refers to the position of the target word in a sentence or natural language text.

[0083] Optionally, the location information of the target word can be annotated manually or by machine.

[0084] In some embodiments, obtaining natural language text and the location information of target words in the natural language text includes:

[0085] Retrieve text from a question bank or student essay database;

[0086] The text in the question bank or student essay database is processed to obtain natural language text and the location information of target words in the natural language text.

[0087] Text can be obtained from a question bank or a student essay database, and then the obtained text can be processed to obtain natural language text.

[0088] The acquired text can be processed in the following ways:

[0089] Method 1: When the obtained text consists of multiple-choice, cloze, or other similar questions, there may be abnormalities such as incorrect answers or unanswered questions, resulting in errors or incomplete text, which may affect the prediction of grammar knowledge.

[0090] Therefore, in some embodiments, questions such as multiple choice and cloze tests are obtained, the correct answers are filled into the question text to form complete sentences, which are used as natural language text, and the position information of the words tested in the questions is marked, and the position information of the words tested in the questions is used as the position information of the target words.

[0091] When using this natural language text as a training sample, teaching and research experts can provide grammatical knowledge related to the words tested in the questions, which can be used as grammatical knowledge labels for this natural language text.

[0092] Method 2: When the obtained text is a reading comprehension exercise or a student's essay, the sentences written by the student are input into the error detection and correction engine. The engine corrects the grammatical errors in the student's sentences, resulting in grammatically correct sentences, which are then used as natural language text. The location information of the words with grammatical errors is marked and used as the location information of the target words.

[0093] When using this natural language text as a training sample, teaching and research experts can provide grammatical knowledge related to the words tested in the questions, which can be used as grammatical knowledge labels for this natural language text.

[0094] It should be noted that when the acquired natural language text consists of multiple sentences, it is necessary to first segment the acquired natural language text into sentences, and then process and annotate each sentence separately.

[0095] Step 220: Vectorize each sentence in the natural language text to obtain the first feature vector corresponding to each word in the natural language text;

[0096] The first feature vector is a vector that represents the textual features of natural language text.

[0097] Specifically, before vectorizing each sentence in the natural language text, each sentence is first segmented and encoded.

[0098] It should be noted that word segmentation refers to dividing the input natural language text into individual words or sub-words using a word segmenter. For example, the sentence "The way referred to to solve the problem sounds reasonable" will be segmented into "The", "way", "referred", "to", "to", "solve", "the", "problem", "sounds", and "reasonable". By segmenting natural language text, it is easier to distinguish each word in the text.

[0099] Encoding refers to the process of encoding each word, sub-word, or special character by an encoder, thereby converting the input natural language text into a string of numbers. For example, the encodings of “The”, “way”, “referred”, “to”, “to”, “solve”, “the”, “problem”, “sounds”, and “reasonable” are “101”, “762”, “1029”, “3825”, “12752”, “92”, “92”, “5271”, “2314”, “823”, “15278”, “1019”, “6524”, “1021”, and “102”, respectively.

[0100] Vectorization refers to converting a string of encoded numbers into a sequence of feature vectors through a vectorization layer. Based on this sequence of feature vectors, grammatical knowledge prediction can be performed on the acquired natural language text.

[0101] The word segmenter, encoder, and vectorization layer mentioned above can be part of the BERT model or can be implemented as separate network structures. This invention does not impose any restrictions on this.

[0102] In some embodiments, step 220 includes:

[0103] Each sentence in the natural language text is vectorized sequentially to obtain the word vector and position encoding vector corresponding to each word in the natural language text.

[0104] Understandably, the first feature vector includes word vectors and positional encoding vectors. That is, the text features of natural language text can be represented by the word vectors and positional encoding vectors corresponding to each word in the natural language text.

[0105] Among them, word vectors are vectorized representations of words or sub-words, and positional encoding vectors are used to characterize the position of the word in a sentence.

[0106] Positional encoding vectors can be used for text classification. In the text classification process, the positional information of the target word is matched with the positional encoding vector of each word in the acquired natural language text, which facilitates the extraction of the feature vector of the target word.

[0107] In this embodiment of the invention, each sentence in the natural language text is sequentially segmented, encoded, and vectorized to obtain the word vector and positional encoding vector corresponding to each word in the natural language text. The word vector and positional encoding vector reflect the part-of-speech features of the natural language text and the positional features of each word that makes up the text, and can be used for subsequent grammatical knowledge classification of the target word.

[0108] Step 230: Perform dependency parsing on each sentence in the natural language text to obtain the dependency parsing information corresponding to each word in the natural language text. Then, vectorize the dependency parsing information to obtain the dependency feature vector corresponding to each word in the natural language text.

[0109] It is difficult to predict the grammatical knowledge of sentences with complex sentence structures using only a single text feature. Therefore, this embodiment of the invention also performs dependency parsing on each sentence in the natural language text.

[0110] Syntactic analysis is one of the key technologies in natural language processing. It mainly analyzes the syntactic structure of sentences and the dependency relationships between words. Analyzing the syntactic structure of sentences refers to analyzing the subject-verb-object, attributive, adverbial, and complement relationships, while the dependency relationships between words include: coordination, subordination, and progression.

[0111] Common syntactic analysis tasks include:

[0112] 1) Syntactic structure analysis, also known as phrase structure analysis or constituent syntactic analysis, is used to identify the phrase structure in a sentence and the hierarchical syntactic relationships between phrases.

[0113] 2) Dependency syntactic parsing, also known as dependency relation analysis or simply dependency analysis, is a process of analyzing a sentence into a dependency syntactic tree, describing the dependency relationships between words, that is, pointing out the syntactic collocation relationships between words. These collocation relationships are related to semantics, and their function is to identify the interdependencies between words in a sentence.

[0114] 3) Deep grammar and syntactic analysis, which uses deep grammars, such as lexicalized tree adjoining grammar (LTAG), lexical functional grammar (LFG), and combinatory categorical grammar (CCG), to perform deep syntactic and semantic analysis on sentences.

[0115] In this embodiment of the invention, dependency parsing is performed on each sentence in the natural language text to identify the interdependencies between the words in each sentence, so as to use the interdependencies to classify grammatical knowledge.

[0116] Dependency parsing is performed on each sentence in the natural language text to obtain the dependency parsing information corresponding to each word in the natural language text.

[0117] Among them, dependency syntactic information is the information representing the interdependencies between words in a sentence;

[0118] Dependency syntactic information is encoded and vectorized to obtain dependency feature vectors, which are feature vectors that represent the interdependencies between words in a sentence.

[0119] In dependency syntax, the verb in the predicate is considered the core of a sentence. "Dependency" refers to the dominant-subordinate relationship between words, which is not equivalent but directional. Specifically, the dominant element is called the governor (or regent, or head), while the subordinate element is called the modifier (or subordinate, or dependency).

[0120] In some embodiments, dependency syntax information includes:

[0121] 1) The positional information of another word in the sentence on which a word depends grammatically, such as positional information that can be represented by a positional subscript;

[0122] 2) The relationship between a word and another word on which it is grammatically dependent, such as “noun clause modifier” or “noun subject”, which can be expressed as deprel.

[0123] In some embodiments, dependency syntactic information includes not only 1) and 2) above, but also part-of-speech information of words. Part-of-speech, as a basic grammatical attribute of words, is a key feature of words and sentences.

[0124] Further, in step 230, the dependency syntax information is vectorized to obtain the dependency feature vector corresponding to each word in the natural language text, including:

[0125] The positional information in dependency syntax is vectorized to obtain the dependency position vector corresponding to each word in the natural language text.

[0126] The relations in dependency syntax information are vectorized to obtain the dependency relation vector corresponding to each word in the natural language text.

[0127] Before vectorizing the dependency syntax information, the process also includes encoding the dependency syntax information.

[0128] Figure 3This is an example diagram of dependency parsing information provided in an embodiment of the present invention. In this embodiment, dependency parsing is performed on the English text "The way referred to to solve the problem sounds reasonable," and the resulting dependency parsing information is shown in the example diagram. Figure 4 As shown, the word "sounds" is the root of the sentence. The word "to" depends on the word "solve" which has a positional marker. The dependencies between "referred" and "way" are "noun clause modifier" and "solve" and "referred" are "adverbial clause modifier". The dependencies between "way" and "sounds" are "noun subject". The part of speech of "The" is "DT", which stands for "Determiner". The part of speech of "way" is "NN", which stands for "Noun, singular or mass". The part of speech of "referred" is "VBD", which stands for "Verb, pasttense". The parts of speech of other words are not detailed here.

[0129] By introducing dependency parsing and encoding and vectorizing the obtained dependency parsing information, we can obtain the dependency position vector and dependency relation vector corresponding to each word in the natural language text. This improves the richness of the grammatical feature vector information, making it easier for the model to learn the dependency relations between words in a sentence and understand the relationship between sentence structure and words, thereby effectively improving the accuracy of grammatical knowledge prediction.

[0130] It should be noted that step 230 can also be performed before step 220.

[0131] Step 240: Fuse the first feature vector and the dependency feature vector to obtain the fused vector. Based on the fused vector and the position information of the target word, perform grammatical knowledge classification to obtain the grammatical knowledge corresponding to the target word.

[0132] It is difficult to predict grammatical knowledge for complex sentence texts based on a single feature vector. Therefore, this invention integrates dependency feature vectors on the basis of the first feature vector. The resulting fused vector information can more richly represent the feature information of natural language texts and improve the accuracy of predicting grammatical knowledge for complex sentence texts.

[0133] The fusion of the first feature vector and the dependency feature vector refers to adding the first feature vector and the dependency feature vector to obtain the fused vector, which is the set of feature vectors corresponding to each word in each sentence of the natural language text.

[0134] Optionally, the fused vector includes word vectors, positional encoding vectors, dependency position vectors, and dependency relation vectors corresponding to each word in the natural language text. Optionally, the grammatical knowledge classification includes at least one of grammar classification, error type classification, and phrase classification.

[0135] It should be noted that in smart education, grammar classification is an important application of text classification. This task requires inputting any sentence and specifying a word in that sentence, and the model needs to predict the grammar associated with that word. Error type classification is another application of text classification in smart education. In scenarios such as student essay correction, this task requires inputting a sentence containing errors and the location of the errors, and predicting the type of error. Phrase classification predicts which key phrases are contained in the input English sentence.

[0136] Accordingly, grammatical knowledge includes at least one of the following: grammatical information, error type information, and phrase information.

[0137] It should be noted that grammatical knowledge and grammatical knowledge classification correspond to each other. That is, if the grammatical knowledge classification includes grammatical categories, then grammatical knowledge includes grammatical information; if the grammatical knowledge classification includes both grammatical categories and error type classifications, then grammatical knowledge includes both grammatical information and error type information.

[0138] In the case of single-task classification, grammatical knowledge can be classified into grammatical categories, error type categories, or phrase categories; the corresponding grammatical knowledge can be grammatical information, error type information, or phrase information.

[0139] In multi-task classification, grammatical knowledge classification can be at least two of the following: grammar classification, error type classification, and phrase classification. For example, grammar knowledge classification can be grammar classification and error type classification, with the corresponding grammatical knowledge being grammatical information and error type information; grammar knowledge classification can also be grammar classification and phrase classification, with the corresponding grammatical knowledge being grammatical information and phrase information. Furthermore, grammar knowledge classification can also be grammar classification, error type classification, and phrase classification, with the corresponding grammatical knowledge being grammatical information, error type information, and phrase information.

[0140] Since error types, grammar, and phrases are correlated to some extent, a single model can be used to predict at least two of the grammar information, error type information, and phrase information simultaneously. This multi-task learning allows the prediction effects of each to complement each other, thereby improving the accuracy of grammar knowledge prediction.

[0141] Grammar prediction, error type prediction, and phrase prediction can be used in business scenarios such as student learning analysis and related knowledge point recommendation. The grammar knowledge prediction method provided in this invention segmentes, encodes, and vectorizes the acquired natural language text to obtain the first feature vector of each word. It then performs dependency parsing on the text to obtain the dependency feature vector of each word. The first feature vector and the dependency feature vector are fused. Based on the fused vector and the positional information of the target word, the text is simultaneously classified using multi-task methods to obtain the grammar knowledge corresponding to the target word. By introducing dependency parsing, the model can understand the relationship between sentence structure and words, thus enabling accurate grammar knowledge prediction for natural language texts with complex sentence structures.

[0142] It should be noted that each embodiment of the present invention can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0143] Figure 4 This is a flowchart illustrating the pre-classification of grammatical knowledge based on fused vectors provided in an embodiment of the present invention. Specifically, it classifies grammatical knowledge based on the fused vectors and the positional information of the target word to obtain the grammatical knowledge corresponding to the target word, including:

[0144] Step 410: Extract features from the fused vector to obtain the second feature vector;

[0145] The fused vector is a set of feature vectors corresponding to each word in the natural language text, including word vectors, position encoding vectors, dependency position vectors, and dependency relation vectors.

[0146] Optionally, the fused vector can be input into the BERT model for feature extraction to obtain a second feature vector.

[0147] The second feature vector is a vector that represents the text features and dependency features of natural language text. Compared with the text features corresponding to the first feature vector, the second feature vector contains richer feature information, including not only text feature information but also corresponding dependency feature information.

[0148] It is understandable that the second feature vector can reflect the part-of-speech information, positional encoding information, positional information of another word in the sentence that it depends on grammatically, and the relationship between the second word and the other word that it depends on grammatically.

[0149] By extracting features from the fused vector, a second feature vector is obtained, which facilitates the prediction of grammatical knowledge in natural language texts with complex sentence structures.

[0150] Step 420: Based on the second feature vector and the position information of the target word, obtain the feature vector corresponding to the target word;

[0151] Based on the location information of the target word, the feature vector corresponding to the target word can be extracted from the second feature vector.

[0152] Step 430: Perform grammatical knowledge classification on the feature vectors corresponding to the target words to obtain the grammatical knowledge corresponding to the target words.

[0153] In this embodiment of the invention, feature extraction is performed on the fused feature vector, and the feature vector corresponding to the target word is obtained based on the position information of the target word. The feature vector corresponding to the target word is then classified into grammatical knowledge to obtain the grammatical knowledge corresponding to the target word, thus realizing grammatical knowledge prediction.

[0154] In some embodiments, step 430 includes:

[0155] The feature vector corresponding to the target word is input into a multi-objective classification layer for grammatical knowledge classification, thereby obtaining the grammatical knowledge corresponding to the target word.

[0156] The multi-objective classification layer includes at least one of the following: grammar classification layer, error type classification layer, and phrase classification layer. The multi-objective classification layer can be one or two of these, or a sum of all three; details will not be elaborated here. The grammar knowledge classification corresponds one-to-one with the obtained grammar knowledge, enabling single-task or multi-task prediction, and has a wide range of applications.

[0157] Figure 5 This is one of the schematic diagrams of the multi-objective classification layer model structure provided in an embodiment of the present invention. In this embodiment, the multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data and the grammatical knowledge corresponding to the target training words in the natural language training text as training labels.

[0158] Figure 6 This is one of the flowcharts illustrating the method for determining a multi-target classification layer provided in an embodiment of the present invention, such as... Figure 6 As shown, the process of determining the multi-target classification layer includes the following steps: step 610, step 620, step 630, step 640 and step 650.

[0159] Step 610: Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0160] The method for obtaining the natural language training text and the location information of the target training words in the natural language training text can be referred to the description in step 210 above, and will not be repeated here.

[0161] Step 620: Extract features from the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0162] Feature extraction of natural language training text can be performed as described in steps 220 to 240 above.

[0163] Optionally, feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text, including:

[0164] Each sentence in the natural language training text is segmented, encoded, and vectorized to obtain the word vector and position encoding vector corresponding to each word in the natural language training text.

[0165] Dependency parsing is performed on each sentence in the natural language training text to obtain the dependency parsing information corresponding to each word in the natural language training text. The dependency parsing information includes: the position information of another word that the word depends on in the sentence, the relationship between the word and the other word that the word depends on, and the part-of-speech information of the word.

[0166] The position information in the dependency syntax information is encoded and vectorized to obtain the dependency position vector corresponding to each word in the natural language training text. The relations in the dependency syntax information are encoded and vectorized to obtain the dependency relation vector corresponding to each word in the natural language training text.

[0167] The word vector and position encoding vector corresponding to each word in the natural language training text, as well as the dependency position vector and dependency relation vector corresponding to each word in the natural language training text, are fused to obtain the fused vector.

[0168] Feature extraction is performed on the fused vectors to obtain the feature vectors corresponding to the natural language training text.

[0169] Step 630: Based on the feature vectors corresponding to the natural language training text and the position information of the target training words, obtain the feature vectors corresponding to the target training words;

[0170] Based on the location information of the target training words, feature vectors corresponding to the target training words are extracted from the feature vectors corresponding to the natural language training text.

[0171] Step 640: Determine the grammar knowledge corresponding to the target training words;

[0172] You can obtain the grammar knowledge corresponding to the target training words provided by teaching and research experts, or you can obtain the grammar knowledge corresponding to the target training words from a pre-established grammar knowledge base.

[0173] The grammatical knowledge of the target training words can include at least one of the following: grammatical information, error type information, and phrase information.

[0174] It needs to be explained that the grammatical knowledge corresponding to the target training words is used as training labels, that is, real data, to facilitate model training.

[0175] Step 650: Train the initial multi-objective classification layer using the feature vectors corresponding to the target training words as training data and the grammatical knowledge corresponding to the target training words as training labels; after the initial multi-objective classification layer is trained, the multi-objective classification layer is obtained.

[0176] Optionally, the initial multi-objective classification layer includes at least one of a syntax classification layer, an error type classification layer, and a phrase classification layer.

[0177] In some embodiments, the initial multi-objective classification layer includes: a syntax classification layer, an error type classification layer, and a phrase classification layer.

[0178] Using the feature vectors corresponding to the target training words as training data, and the grammatical information, error type information, and phrase information corresponding to the target training words as training labels, the label lists of multiple sub-tasks are merged into one label list, and the classification layers corresponding to multiple sub-tasks are merged into one multi-target classification layer, the dimension of which is equal to the sum of the dimensions of the classification layers corresponding to each sub-task. This transforms multiple classification tasks into a single multi-label classification task, which facilitates alternating training on multi-label text data, simplifies the training process, and improves the efficiency of model training.

[0179] In some embodiments, training the initial multi-objective classification layer in step 650 includes:

[0180] Input the feature vector corresponding to the target training word into at least one of the grammar classification layer, error type classification layer and phrase classification layer to obtain the grammar knowledge prediction result;

[0181] Based on the grammatical knowledge prediction results and the grammatical knowledge corresponding to the target training words, the first loss function value is calculated using the multi-label cross-entropy loss function;

[0182] Based on the first loss function value, update the parameters of the initial multi-object classification layer.

[0183] The grammar knowledge prediction results include at least one of the following: grammar prediction results, error type prediction results, and phrase prediction results.

[0184] In this embodiment of the invention, the grammar knowledge prediction result is used as the prediction data, and the grammar knowledge corresponding to the target training word is used as the real data. The loss function value is calculated using the multi-label cross-entropy loss function. Based on the loss function value, the parameters of the initial multi-objective classification layer are updated. After training, the initial multi-objective classification layer that has been trained is obtained, which is a multi-objective classification layer that can be used for grammar knowledge classification.

[0185] When the initial multi-objective classification layer includes at least two classification layers, the multi-label cross-entropy loss function can be used to achieve alternating training from different data sources and dynamically balance the loss of the classification layers corresponding to each subtask.

[0186] Figure 7 The second schematic diagram of the multi-objective classification layer model structure provided in this embodiment of the invention is as follows: In this embodiment, the multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data, and using the grammatical knowledge and part-of-speech information corresponding to the target training words in the natural language training text as training labels. The part-of-speech information is obtained by performing dependency parsing on the natural language training text.

[0187] Figure 8 This is a second schematic diagram of the method for determining a multi-target classification layer provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the process of determining the multi-target classification layer includes the following steps: step 810, step 820, step 830, step 840 and step 850.

[0188] Step 810: Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0189] Step 820: Extract features from the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0190] Step 830: Based on the feature vectors corresponding to the natural language training text and the position information of the target training words, obtain the feature vectors corresponding to the target training words;

[0191] Step 840: Determine the grammatical knowledge and part-of-speech information corresponding to the target training words; wherein, the part-of-speech information corresponding to the target training words is obtained based on dependency parsing.

[0192] Step 850: Using the feature vectors corresponding to the target training words as training data, and the grammatical knowledge and part-of-speech information corresponding to the target training words as training labels, train an initial multi-objective classification layer; the initial multi-objective classification layer includes at least one of the following: grammar classification layer, error type classification layer, and phrase classification layer, as well as a part-of-speech classification layer;

[0193] Step 860: After the initial multi-objective classification layer is trained, retain at least one of the syntax classification layer, error type classification layer and phrase classification layer in the initial multi-objective classification layer to obtain the multi-objective classification layer.

[0194] Steps 810-860 can be referred to steps 610-650, and will not be described in detail here. The difference is that this embodiment introduces part-of-speech prediction as a subtask on the basis of grammatical knowledge prediction: In step 840, the grammatical knowledge and part-of-speech information corresponding to the target training word are determined; in step 850, the grammatical knowledge and part-of-speech information corresponding to the target training word are used as training labels, wherein the part-of-speech information is obtained during the dependency parsing process of the natural language training text; in step 860, after the initial multi-objective classification layer is trained, at least one of the grammatical classification layer, error type classification layer and phrase classification layer in the initial multi-objective classification layer is retained to obtain the multi-objective classification layer, that is, in the application stage of the multi-objective classification layer, the part-of-speech classification layer is not required.

[0195] Since part-of-speech information and grammatical knowledge are also related, this embodiment introduces part-of-speech prediction as an additional subtask to train a multi-objective classification layer, which can effectively improve the prediction effect of grammatical knowledge and thus solve the problem of difficult prediction of complex sentence grammar.

[0196] In this embodiment of the invention, the feature vectors corresponding to the target training words in the natural language training text are used as training data, and the grammatical knowledge and part-of-speech information corresponding to the target training words are used as training labels for training. On the basis of grammatical knowledge prediction, part-of-speech prediction is introduced as a subtask to train the multi-objective classification layer, which facilitates the optimization of the parameters of the multi-objective classification layer and improves the accuracy of the multi-objective classification layer in predicting grammatical knowledge.

[0197] In some embodiments, training the initial multi-objective classification layer in step 850 includes:

[0198] Input the feature vector corresponding to the target training word into at least one of the grammar classification layer, error type classification layer and phrase classification layer to obtain the grammar knowledge prediction result;

[0199] The feature vectors corresponding to the target training words are input into the part-of-speech classification layer to obtain the part-of-speech prediction results;

[0200] Based on the grammatical knowledge prediction results and the grammatical knowledge corresponding to the target training words, the first loss function value is calculated using the multi-label cross-entropy loss function;

[0201] Based on the part-of-speech prediction results and the part-of-speech information corresponding to the target training words, the value of the second loss function is calculated.

[0202] The parameters of the initial multi-target classification layer are updated based on the first and second loss function values.

[0203] The grammar knowledge prediction results include at least one of the following: grammar prediction results, error type prediction results, and phrase prediction results.

[0204] In this embodiment of the invention, the grammar knowledge prediction result and the part-of-speech prediction result are used as prediction data, and the grammar knowledge and part-of-speech information corresponding to the target training word are used as real data. The loss between the grammar knowledge prediction result and the grammar knowledge corresponding to the target training word is calculated using the multi-label cross-entropy loss function to obtain a first loss function value. The loss between the part-of-speech prediction result and the part-of-speech information corresponding to the target training word is calculated using the single-label cross-entropy loss function, cross-entropy loss function, or multi-label cross-entropy loss function to obtain a second loss function value. Based on the first loss function value and the second loss function value, the parameters of the initial multi-objective classification layer are updated. After training, the initial multi-objective classification layer is obtained. At least one of the grammar classification layer, error type classification layer, and phrase classification layer is retained, or the part-of-speech classification layer is removed, thus obtaining a multi-objective classification layer that can be used for grammar knowledge classification.

[0205] In this embodiment of the invention, the parameters of the initial multi-objective classification layer are updated based on the first loss function value and the second loss function value, which facilitates the optimization of the multi-objective classification layer and further improves the prediction effect of the multi-objective classification layer on grammatical knowledge.

[0206] In some embodiments, the multi-label cross-entropy loss function is:

[0207]

[0208] Among them, s i Let s represent the score corresponding to the i-th category. j Let represent the score corresponding to the j-th category, i represent the i-th category, j represent the j-th category, neg represents the set of non-target categories, and pos represents the set of target categories.

[0209] Because the input data comes from different data sources for different subtasks, the labels on the data may be inconsistent. For example, some natural speech training texts may be labeled with only grammatical information, others with only error type information, and still others with labels that include grammatical, error type, and phrase information. Therefore, during training, a loss mask needs to be used on the loss function to control the range of loss calculation.

[0210] At the same time, because the number of labels for different subtasks is different (for example, there are hundreds of grammar knowledge points and phrase categories, but only dozens of error types), the loss can vary greatly when calculating the loss for data from different sources.

[0211] To address the aforementioned issues, instead of simply using 0 and 1 as mask values ​​when calculating the loss mask, we consider the number of labels contained in each sample and use the average of 0 and the number of labels as the loss mask value. Finally, the loss obtained by gradient descent is the product of the original loss and the loss mask, thus ensuring the stability of the loss value.

[0212] In this context, a loss mask is used to control the range of loss calculation on the multi-label cross-entropy loss function. The loss mask is determined according to the following formula:

[0213]

[0214] Where t represents the t-th task, mask t The 0-1 form mask representing the t-th task, mask′ t The task is a dynamic mask for updating weight values, which is either a syntax classification task, an error type classification task, or a phrase classification task.

[0215] Finally, the dynamic loss function value for gradient descent is the product of the original loss function value and the loss mask. The formula for calculating the dynamic loss function value is:

[0216] Loss′=Loss*mask′ t

[0217] Where Loss is the original loss function value, and mask′ t This is a dynamic mask for updating the weight values.

[0218] Figure 9 A schematic diagram illustrating the loss calculation using masked loss control in the grammar knowledge classification model provided in this embodiment of the invention; as shown. Figure 7 The grammar knowledge classification model includes a word segmentation and encoding module, a dependency parsing module, a BERT model, a multi-objective classification layer, and a part-of-speech classification layer. This grammar knowledge classification model is used to execute steps 810-860, using a loss mask to control the range of loss calculation on the multi-label cross-entropy loss function, obtaining dynamic mask values ​​for different classification tasks in the multi-objective classification layer, and finally performing gradient descent to obtain the dynamic loss function value Loss1, which is the product of the original loss function value and the loss mask, and Loss2, which is the loss function value of the part-of-speech classification layer.

[0219] Table 1 provides examples of dynamic masking loss provided in embodiments of the present invention. As shown in Table 1, for example, the input is: "[CLS]The way referred to to solve the problem. [SEP]The way referred to solving the..." The problem is specified as "to" and "solving". After processing by the grammar knowledge classification model, the output grammar knowledge is "grammar label: infinitive as attributive", "error type label: verb tense error" and "phrase label: none". The original mask values ​​for the grammar classification task are [0,0,0,0,1,…,0,0,0], the original mask values ​​for the error type classification task are [0,1,0,…,0], and the original mask values ​​for the phrase classification task are [0,0,0,0,1,…,0,0,0]. The loss mask is used on the multi-label cross-entropy loss function to control the range of loss calculation. The resulting dynamic mask values ​​are [0.0967,0.0967,…,0.0967], [0.9032,0.9032,…,0.9032], and [0,0,0,0,0,…,0,0,0].

[0220] Table 1 Examples of Dynamic Masking Loss

[0221]

[0222] In this embodiment of the invention, instead of simply using 0 and 1 as mask values ​​when calculating the loss mask, the number of labels contained in each sample is considered, and the average of 0 and the number of labels is used as the value of the loss mask. This can automatically balance the loss functions of each subtask and solve the problem of class imbalance when calculating the loss function value of data from different sources.

[0223] Figure 10 This is a schematic diagram of the apparatus for performing a grammatical knowledge prediction method according to an embodiment of the present invention. In this embodiment, natural language training text is obtained from a database of questions and student essays. The obtained natural language training text is processed by a data generation and training module to obtain training text. The training text is then subjected to dependency parsing by a syntactic analysis module to obtain the dependency parsing information of each sentence. The training text is used as training data to input into a training model for training. During the application model training process, the dependency parsing information of each sentence obtained by the syntactic analysis module is introduced. When in use, a student essay or an English question is input, and the trained application model performs reasoning on the input text to obtain the relevant knowledge points, error types, and phrases contained in the text.

[0224] In this embodiment of the invention, by performing three tasks simultaneously—grammar classification, error type classification, and phrase classification—using a single model, it is possible not only to correct student essays and determine error types, but also to predict relevant grammar knowledge points and phrases in student essays and questions, facilitating grammar learning and student performance analysis.

[0225] The grammar knowledge prediction device provided in the embodiments of the present invention is described below. The grammar knowledge prediction device described below can be referred to in correspondence with the grammar knowledge prediction method described above.

[0226] Figure 11 A schematic diagram of the structure of the grammar knowledge prediction device provided by the present invention is shown below. Figure 11 As shown, the grammar knowledge prediction device 1100 includes:

[0227] Acquisition unit 1110 is used to acquire natural language text and the location information of target words in the natural language text;

[0228] The text feature extraction unit 1120 is used to vectorize each sentence in the natural language text to obtain the first feature vector corresponding to each word in the natural language text.

[0229] The dependency parsing unit 1130 is used to perform dependency parsing on each sentence in the natural language text, obtain the dependency parsing information corresponding to each word in the natural language text, and perform vectorization processing on the dependency parsing information to obtain the dependency feature vector corresponding to each word in the natural language text.

[0230] Classification unit 1140 is used to fuse the first feature vector and the dependency feature vector to obtain the fused vector. Based on the fused vector and the position information of the target word, grammatical knowledge classification is performed to obtain the grammatical knowledge corresponding to the target word.

[0231] Optionally, each sentence in the natural language text is vectorized to obtain the first feature vector corresponding to each word in the natural language text, including:

[0232] Each sentence in the natural language text is vectorized sequentially to obtain the word vector and position encoding vector corresponding to each word in the natural language text.

[0233] Optionally, dependency syntactic information includes: the position of another word that a word depends on grammatically in the sentence, and the relationship between the word and the other word that a word depends on grammatically;

[0234] The dependency syntax information is vectorized to obtain the dependency feature vector corresponding to each word in the natural language text, including:

[0235] The positional information in dependency syntax is vectorized to obtain the dependency position vector corresponding to each word in the natural language text.

[0236] The relations in dependency syntax information are vectorized to obtain the dependency relation vector corresponding to each word in the natural language text.

[0237] Optionally, grammatical knowledge classification is performed based on the fused vector and the positional information of the target word to obtain the grammatical knowledge corresponding to the target word, including:

[0238] Feature extraction is performed on the fused vector to obtain the second feature vector;

[0239] Based on the second feature vector and the position information of the target word, the feature vector corresponding to the target word is obtained;

[0240] The feature vectors corresponding to the target words are classified according to grammatical knowledge to obtain the grammatical knowledge corresponding to the target words.

[0241] Optionally, the feature vectors corresponding to the target word are classified according to grammatical knowledge to obtain the grammatical knowledge corresponding to the target word, including:

[0242] The feature vector corresponding to the target word is input into a multi-objective classification layer to classify grammatical knowledge and obtain the grammatical knowledge corresponding to the target word.

[0243] The multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data and the grammatical knowledge corresponding to the target training words in the natural language training text as training labels.

[0244] Alternatively, the multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data, and using the grammatical knowledge and part-of-speech information corresponding to the target training words in the natural language training text as training labels. The part-of-speech information is obtained by performing dependency parsing on the natural language training text.

[0245] Optionally, the process of determining the multi-target classification layer includes:

[0246] Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0247] Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0248] Based on the feature vectors corresponding to the natural language training text and the position information of the target training words, the feature vectors corresponding to the target training words are obtained.

[0249] Identify the grammar knowledge corresponding to the target words for training;

[0250] Using the feature vectors corresponding to the target training words as training data and the grammatical knowledge corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: grammar classification layer, error type classification layer, and phrase classification layer.

[0251] After the initial multi-object classification layer is trained, the multi-object classification layer is obtained.

[0252] Optionally, the process of determining the multi-target classification layer includes:

[0253] Obtain the natural language training text and the location information of the target training words in the natural language training text;

[0254] Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text;

[0255] Based on the feature vectors corresponding to the natural language training text and the position information of the target training words, the feature vectors corresponding to the target training words are obtained.

[0256] The grammatical knowledge and part-of-speech information corresponding to the target training words are determined, wherein the part-of-speech information corresponding to the target training words is obtained through dependency parsing;

[0257] Using the feature vectors corresponding to the target training words as training data, and the grammatical knowledge and part-of-speech information corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: grammar classification layer, error type classification layer, and phrase classification layer, as well as a part-of-speech classification layer.

[0258] After the initial multi-objective classification layer is trained, at least one of the syntax classification layer, error type classification layer, and phrase classification layer in the initial multi-objective classification layer is retained to obtain the multi-objective classification layer.

[0259] Optionally, the initial multi-object classification layer is trained, including:

[0260] Input the feature vector corresponding to the target training word into at least one of the grammar classification layer, error type classification layer and phrase classification layer to obtain the grammar knowledge prediction result, which includes at least one of the grammar prediction result, error type prediction result and phrase prediction result;

[0261] Based on the grammatical knowledge prediction results and the grammatical knowledge corresponding to the target training words, the first loss function value is calculated using the multi-label cross-entropy loss function;

[0262] Based on the first loss function value, update the parameters of the initial multi-object classification layer.

[0263] It should be noted that the grammar knowledge prediction device provided in this embodiment of the invention can implement all the method steps implemented in the above-described grammar knowledge prediction method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0264] Figure 12 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240. The processor 1210, communications interface 1220, and memory 1230 communicate with each other via the communication bus 1240. The processor 1210 can call logical instructions in the memory 1230 to execute a grammar knowledge prediction method. This method includes: acquiring natural language text and the positional information of target words in the natural language text; vectorizing each sentence in the natural language text to obtain a first feature vector corresponding to each word; performing dependency parsing on each sentence in the natural language text to obtain dependency syntactic information corresponding to each word; vectorizing the dependency syntactic information to obtain a dependency feature vector corresponding to each word; fusing the first feature vector and the dependency feature vector to obtain a fused vector; and classifying grammar knowledge based on the fused vector and the positional information of the target word to obtain the grammar knowledge corresponding to the target word.

[0265] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0266] The electronic device provided in this embodiment of the invention can implement all the method steps implemented in the above-described grammar knowledge prediction method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0267] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the grammar knowledge prediction method provided in the above-described method embodiments. The method includes: acquiring natural language text and the position information of target words in the natural language text; performing vectorization processing on each sentence in the natural language text to obtain a first feature vector corresponding to each word in the natural language text; performing dependency parsing on each sentence in the natural language text to obtain dependency parsing information corresponding to each word in the natural language text; performing vectorization processing on the dependency parsing information to obtain a dependency feature vector corresponding to each word in the natural language text; fusing the first feature vector and the dependency feature vector to obtain a fused vector; and performing grammar knowledge classification based on the fused vector and the position information of the target word to obtain the grammar knowledge corresponding to the target word.

[0268] The computer program product provided in this embodiment of the invention can implement all the method steps implemented in the above-described grammar knowledge prediction method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0269] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the grammar knowledge prediction method provided in the above-described method embodiments. The method includes: acquiring natural language text and the position information of target words in the natural language text; performing vectorization processing on each sentence in the natural language text to obtain a first feature vector corresponding to each word in the natural language text; performing dependency parsing on each sentence in the natural language text to obtain dependency syntactic information corresponding to each word in the natural language text; performing vectorization processing on the dependency syntactic information to obtain a dependency feature vector corresponding to each word in the natural language text; fusing the first feature vector and the dependency feature vector to obtain a fused vector; and performing grammar knowledge classification based on the fused vector and the position information of the target word to obtain the grammar knowledge corresponding to the target word.

[0270] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0271] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0272] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of predicting syntactic knowledge, the method comprising: include: Obtain natural language text and the location information of target words in the natural language text; Each sentence in the natural language text is vectorized to obtain the first feature vector corresponding to each word in the natural language text. Dependency parsing is performed on each sentence in the natural language text to obtain the dependency parsing information corresponding to each word in the natural language text. The dependency parsing information is then vectorized to obtain the dependency feature vector corresponding to each word in the natural language text. The first feature vector and the dependency feature vector are fused to obtain a fused vector; features are extracted from the fused vector to obtain a second feature vector; based on the second feature vector and the position information of the target word, the feature vector corresponding to the target word is obtained; the feature vector corresponding to the target word is input into a multi-objective classification layer for grammatical knowledge classification to obtain the grammatical knowledge corresponding to the target word. The multi-objective classification layer includes at least one of the following: a syntax classification layer, an error type classification layer, and a phrase classification layer.

2. The syntax knowledge prediction method of claim 1, wherein, The process of vectorizing each sentence in the natural language text to obtain the first feature vector corresponding to each word in the natural language text includes: Each sentence in the natural language text is vectorized sequentially to obtain the word vector and position encoding vector corresponding to each word in the natural language text.

3. The syntax knowledge prediction method of claim 1, wherein, The dependency syntactic information includes: the position information of another word that the word depends on grammatically in the sentence, and the relationship between the word and the other word that the word depends on grammatically; The vectorization process of the dependency syntax information to obtain the dependency feature vector corresponding to each word in the natural language text includes: The position information in the dependency syntax information is vectorized to obtain the dependency position vector corresponding to each word in the natural language text. The relations in the dependency syntax information are vectorized to obtain the dependency relation vector corresponding to each word in the natural language text.

4. The syntax knowledge prediction method of claim 1, wherein, The multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data and the grammatical knowledge corresponding to the target training words in the natural language training text as training labels. Alternatively, the multi-objective classification layer is trained by using the feature vectors corresponding to the target training words in the natural language training text as training data, and using the grammatical knowledge and part-of-speech information corresponding to the target training words in the natural language training text as training labels, wherein the part-of-speech information is obtained by performing dependency parsing on the natural language training text.

5. The syntax knowledge prediction method of claim 4, wherein, The process of determining the multi-target classification layer includes: Obtain the natural language training text and the location information of the target training words in the natural language training text; Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text; Based on the feature vector corresponding to the natural language training text and the position information of the target training word, the feature vector corresponding to the target training word is obtained; Determine the grammatical knowledge corresponding to the target training words; Using the feature vectors corresponding to the target training words as training data and the grammatical knowledge corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: a grammar classification layer, an error type classification layer, and a phrase classification layer. After the initial multi-object classification layer is trained, the multi-object classification layer is obtained.

6. The syntax knowledge prediction method of claim 4, wherein, The process of determining the multi-target classification layer includes: Obtain the natural language training text and the location information of the target training words in the natural language training text; Feature extraction is performed on the natural language training text to obtain the feature vector corresponding to the natural language training text; Based on the feature vector corresponding to the natural language training text and the position information of the target training word, the feature vector corresponding to the target training word is obtained; The grammatical knowledge and part-of-speech information corresponding to the target training word are determined, wherein the part-of-speech information corresponding to the target training word is obtained by dependency parsing; Using the feature vectors corresponding to the target training words as training data, and the grammatical knowledge and part-of-speech information corresponding to the target training words as training labels, an initial multi-objective classification layer is trained. The initial multi-objective classification layer includes at least one of the following: a grammar classification layer, an error type classification layer, and a phrase classification layer, as well as a part-of-speech classification layer. After the initial multi-objective classification layer is trained, at least one of the syntax classification layer, error type classification layer, and phrase classification layer in the initial multi-objective classification layer is retained to obtain the multi-objective classification layer.

7. The syntax knowledge prediction method of claim 5, wherein, The initial multi-objective classification layer for training includes: The feature vector corresponding to the target training word is input into at least one of the grammar classification layer, error type classification layer and phrase classification layer to obtain the grammar knowledge prediction result, which includes at least one of the grammar prediction result, error type prediction result and phrase prediction result. Based on the grammar knowledge prediction results and the grammar knowledge corresponding to the target training word, the first loss function value is calculated using the multi-label cross-entropy loss function; Based on the first loss function value, update the parameters of the initial multi-target classification layer.

8. A syntax knowledge prediction apparatus, characterized by comprising: include: The acquisition unit is used to acquire natural language text and the location information of target words in the natural language text; The text feature extraction unit is used to vectorize each sentence in the natural language text to obtain the first feature vector corresponding to each word in the natural language text. The dependency parsing unit is used to perform dependency parsing on each sentence in the natural language text to obtain dependency parsing information corresponding to each word in the natural language text, and to perform vectorization processing on the dependency parsing information to obtain dependency feature vectors corresponding to each word in the natural language text. A classification unit is used to fuse the first feature vector and the dependency feature vector to obtain a fused vector; to extract features from the fused vector to obtain a second feature vector; to obtain a feature vector corresponding to the target word based on the second feature vector and the position information of the target word; and to input the feature vector corresponding to the target word into a multi-objective classification layer for grammatical knowledge classification to obtain the grammatical knowledge corresponding to the target word. The multi-objective classification layer includes at least one of the following: a syntax classification layer, an error type classification layer, and a phrase classification layer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the syntax knowledge prediction method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the syntax knowledge prediction method as described in any one of claims 1 to 7.

Images