Knowledge point annotation method, model training method, computer device and storage medium

Abstract

An embodiment of the present application provides a method for knowledge point annotation, a method for training a model, a computer device, and a storage medium. The annotation method includes: obtaining a target text; inputting the target text into a preset deep language model to obtain a text content representation corresponding to the target text; obtaining a knowledge point representation corresponding to a preset knowledge point label set, where the knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model; performing a matching process on the text content representation and the knowledge point representation to obtain a knowledge point label corresponding to the target text. Based on the deep language model, deeper semantic modeling can be performed on the target text, and the obtained text content representation can also better adapt to the multi-label classification task; the knowledge point representation contains information on the knowledge point system, and combined with the text content representation, the recognition effect of the knowledge point can be improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular, to a method for labeling knowledge points, a method for training a model, a computer device, and a storage medium. Background Art

[0002] The construction of the discipline system plays a crucial role in teachers' daily teaching, students' mastery of discipline capabilities, and active preparation for exams. Among them, the discipline knowledge point system corresponds to the exam syllabus, which can help teachers scientifically formulate teaching plans and reasonably arrange teaching progress. The knowledge point labels corresponding to test questions can facilitate students to quickly locate their weak points for checking and filling in the gaps, and improve exam scores.

[0003] For the existing massive test questions covering various question types and various knowledge points, if it is required that their knowledge point labels be updated in real time to be synchronized with the latest exam syllabus, that is, the knowledge point system, manual labeling requires a lot of energy and cost, and the work is complicated and there is redundancy. Therefore, gradually, using machines to automatically label knowledge points instead of humans has become an important exploration direction.

[0004] Currently, the automatic labeling of knowledge points is usually abstracted as a supervised multi-label multi-class text classification task. The number of knowledge point labels, that is, the number of categories, is generally as many as hundreds, and a test question usually has more than two knowledge point labels. However, because there are many category labels and it is a multi-label classification, it is a challenging task for traditional machine learning models. At the same time, as a supervised task, the problem of sparse training set labels is relatively common, which will significantly affect the model effect. Summary of the Invention

[0005] The embodiments of this application provide a method for labeling knowledge points, a method for training a model, a computer device, and a storage medium, which can more accurately identify the knowledge point labels corresponding to the target text.

[0006] In a first aspect, this application provides a method for labeling knowledge points, and the method includes:

[0007] Obtain a target text;

[0008] Input the target text into a preset deep language model to obtain a text content representation corresponding to the target text;

[0009] Obtain a knowledge point representation corresponding to a preset knowledge point label set, where the knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model;

[0010] Perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point labels corresponding to the target text.

[0011] Second aspect, the present application provides a method for training a knowledge point annotation model, including:

[0012] Obtain the knowledge point representations corresponding to the preset knowledge point label set, where the knowledge point representations are obtained by processing the knowledge point labels in the knowledge point label set based on a deep language model;

[0013] Obtain the target text and the knowledge point annotation of the target text;

[0014] Input the target text into the deep language model of the knowledge point annotation model to obtain the text content representation corresponding to the target text;

[0015] Perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text;

[0016] Based on a preset loss function, determine the model loss value according to the knowledge point annotation of the target text and the knowledge point label corresponding to the target text;

[0017] Adjust the parameters of the knowledge point annotation model according to the model loss value.

[0018] Third aspect, the present application provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of any of the above methods when executing the computer program.

[0019] Fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. If the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0020] The present application discloses a knowledge point annotation method, a method for training a model, a computer device and a storage medium. Based on a deep language model, the text content representation corresponding to the target text is obtained, and a matching process is performed on the text content representation and the knowledge point representation corresponding to the knowledge point label set to obtain the knowledge point label corresponding to the target text; based on the deep language model, deeper semantic modeling can be performed on the target text, and the obtained text content representation can also better adapt to the multi-label classification task; the knowledge point representation contains the information of the knowledge point system, and combined with the text content representation, the recognition effect of the knowledge point can be improved. Description of the Drawings

[0021] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a schematic diagram of a knowledge point annotation process in the prior art;

[0023] Figure 2 It is a schematic diagram of another knowledge point annotation process in the prior art;

[0024] Figure 3 It is a schematic flowchart of the knowledge point annotation method according to an embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of the application scenario of the knowledge point annotation method in an embodiment;

[0026] Figure 5 It is a schematic diagram of performing knowledge point annotation based on a knowledge point annotation model in an embodiment;

[0027] Figure 6 It is a schematic flowchart of the training method of the knowledge point annotation model according to another embodiment of the present application;

[0028] Figure 7 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0029] [[ID=S33]]The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0030] The flowcharts shown in the accompanying drawings are only illustrative examples, and do not necessarily include all contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order may be changed according to the actual situation.

[0031] The embodiments of the present application provide a knowledge point annotation method, a model training method, a computer device, and a storage medium, which can more accurately identify the knowledge point tags corresponding to the target text.

[0032] Such as Figure 1Shown is a solution for knowledge point annotation based on traditional features and machine learning models. By analyzing the knowledge points in the existing subject system, rules and feature extraction methods are formulated for each knowledge point. For example, text keyword expansion and the type of question stem proposition sentence pattern are used to elaborate in detail on the proposition methods of specific knowledge points. Then, multi-dimensional features are extracted from the test question text for each knowledge point in turn, and a machine learning model is used for text classification based on the extracted multi-dimensional features. For example, by analyzing each knowledge point one by one, different feature extraction rules are customized, and the corresponding keyword list and proposition sentence pattern type are expanded. The advantage of this solution is that it can better grasp the characteristics of the knowledge points themselves. However, it requires a relatively large amount of human cost. When the knowledge point system changes, the rules need to be redesigned, and the rules for the discarded knowledge points cannot be effectively reused. When the exam syllabus is frequently revised, the knowledge point system needs to be updated synchronously. Therefore, the subsequent maintenance cost of the system is also an important consideration. Moreover, the prediction effect of this solution is also restricted by the machine learning model, unable to capture deeper semantics, and it is difficult to correctly distinguish some understanding-type knowledge points such as "understanding the meaning of important concepts in the text" and "understanding the meaning of important sentences in the text". At the same time, the machine learning model performs mediocrely on multi-label classification tasks.

[0033] As Figure 2 Shown is a current solution for knowledge point annotation based on an end-to-end neural network model. By concatenating the test question text or hierarchical modeling to capture the semantics of the test questions, the model uses neural network models such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), or pre-trained language models, and after obtaining the hidden layer representation, uses the softmax activation function or the sigmoid activation function for classification. Although the solution based on the end-to-end neural network can perform deeper semantic modeling and is more suitable for multi-label classification tasks, and by adjusting the network structure and using the sigmoid activation function, more knowledge point labels can be predicted, and the change of the knowledge point system does not require much maintenance cost for the system; however, it fails to consider the hierarchical structure of the knowledge point system itself and / or the correlation relationships existing between different knowledge point labels.

[0034] Based on this, the inventors of the present application improved the knowledge point annotation method, which can make full use of the effective information of the correlation relationships existing between different knowledge point labels, making the accuracy of knowledge point recognition higher and the annotation effect better.

[0035] The knowledge point annotation method and the model training method provided in the embodiments of the present application can be applied to a terminal or a server. The terminal can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, or a personal digital assistant; the server can be an independent server or a server cluster. However, for the sake of easy understanding, the following embodiments will be introduced in detail with the method applied to the server.

[0036] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of a knowledge point annotation method provided by an embodiment of the present application.

[0037] In some embodiments, as Figure 4 shown, the server obtains the target text from the terminal, generates a knowledge point label corresponding to the target text according to the knowledge point annotation method, and sends the corresponding knowledge point label to the terminal.

[0038] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0039] As Figure 3 shown, the knowledge point annotation method includes the following steps S110 to S150.

[0040] Step S110: Obtain the target text.

[0041] In some alternative embodiments, the target text is the text stored locally by the device for implementing the knowledge point annotation method, the text obtained by the device from the network, the text obtained by the device from the input device connected thereto, the text obtained by the device from other electronic devices, the text converted by the device according to voice information, etc.

[0042] In some embodiments, the obtaining of the target text includes: obtaining the question stem and the answer of the question; generating the target text according to a preset start character, the question stem, a preset interval character, and the answer of the question.

[0043] Exemplarily, when the question is a multiple-choice question, the question options and the answer can be concatenated to obtain the answer of the question. When the question is a non-multiple-choice question such as a true / false question, a fill-in-the-blank question, or a short-answer question, the content of the answer can be used as the answer of the question.

[0044] The question stem is used as the first part of the target text, and the start character, such as [CLS], is added at the beginning of the sentence; the answer of the question is used as the second part of the target text, and the interval character, such as [SEP], is used to fill in between the first part and the second part.

[0045] By using all the available text in the question, such as the question stem, options, and answer, as the target text for knowledge point annotation, more abundant information can be utilized to improve the accuracy of annotation.

[0046] Step S120: Input the target text into a preset deep language model to obtain a text content representation corresponding to the target text.

[0047] In some embodiments, the deep language model is a BERT model (Bidirectional Encoder Representations from Transformers). Optionally, the deep language model is a pre-trained BERT model. Of course, the deep language model is not limited to the BERT model. For example, it can also be a recurrent neural network model, a convolutional neural network model, or a combination of multiple models / networks. The BERT model is a bidirectional language representation model. Using the Transformer network (a neural network based on self-attention) as the unit module, it is pre-trained on a large-scale corpus using two upstream tasks: masked language modeling (MLM) and next sentence prediction (NSP). Compared with recurrent neural networks and convolutional neural networks, it has stronger semantic modeling capabilities and can achieve better results with only simple fine-tuning on downstream tasks. Using a pre-trained deep language model such as the BERT model can perform deeper semantic modeling and better adapt to the multi-label classification task of knowledge point annotation.

[0048] Among them, next sentence prediction is one of the upstream tasks of the BERT model, which is used to predict whether the second sentence is the continuation of the first sentence. In common QA (question answering) tasks, the question Q and the answer A are often concatenated as the input of the BERT model, and the BERT model predicts whether the answer A can answer the question Q. In the embodiments of the present application, the BERT model is used to perform semantic modeling on the target text to obtain the text content representation corresponding to the target text.

[0049] In some embodiments, the step of inputting the target text into a preset deep language model to obtain the text content representation corresponding to the target text includes: performing word embedding processing on the target text based on the word embedding layer of the deep language model to obtain text vectors; and processing the text vectors based on the multi-head self-attention network of the deep language model to obtain the text content representation corresponding to the target text.

[0050] Exemplarily, for a certain test question, the number of words in the question stem of the test question is m, the number of words in the test answer (such as options plus the answer) is n, and including the start character and the interval character, the number of words in the target text is m + n + 2. First, each word in the target text will be converted into an embedding vector, and the text vector includes m + n + 2 embedding vectors, for example, expressed as: , where represents the embedding vector corresponding to the th character in the target text.

[0051] The BERT model is a deep language model with 12 layers of Transformers (multi-head self-attention networks) without shared parameters. The output of the previous layer of Transformer serves as the input for the next layer, which can be expressed as follows:

[0052]

[0053] The input of the first layer of Transformer is the text vector , is the output of each layer of Transformer, and the output of the last layer contains the hidden layer representations of all characters in the target text; where represents the context semantic information of the th character, and the vector representation of the start character [CLS] can be selected

[0054] as the text content representation (or semantic representation) corresponding to the entire target text.

[0055]

[0056]

[0057]

[0058] Among them, , are model parameters, represents vector concatenation.

[0059] Step S130: Obtain the knowledge point representations corresponding to the preset knowledge point label set, where the knowledge point representations are obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model.

[0060] Exemplarily, the knowledge point label set includes the labels of all knowledge points in the discipline system: . Inputting the knowledge point labels in the knowledge point label set into the deep language model can obtain the semantic representations of all knowledge points, that is, the knowledge point representations. The principle and steps of the deep language model for semantic modeling of knowledge point labels are the same as those for processing the target text described above, and will not be elaborated here. For example, the vector of the start character [CLS] is also taken as the knowledge point representation.

[0061] Exemplarily, for all knowledge points in the subject system , first, we have performed semantic modeling through the BERT model to obtain the knowledge point representations, and the knowledge point representations include the representation vectors corresponding to each knowledge point .

[0062] In some embodiments, please refer to Figure 5 , obtaining the knowledge point representation corresponding to the preset knowledge point label set includes: obtaining the second knowledge point representation corresponding to the preset knowledge point label set, where the second knowledge point representation is obtained by semantically enhancing the first knowledge point representation based on a graph neural network, and the first knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model.

[0063] Exemplarily, based on the graph neural network, according to the relationships between different knowledge point labels in the knowledge point label set, the first knowledge point representation is semantically enhanced to obtain the second knowledge point representation. By introducing the graph neural network to encode the knowledge point system, a more detailed knowledge point representation, that is, the second knowledge point representation, can be obtained. By combining the second knowledge point representation with the semantic representation of the target text, the recognition accuracy of the knowledge points can be improved.

[0064] Exemplarily, please refer to Figure 5 , obtaining the second knowledge point representation corresponding to the preset knowledge point label set includes steps S131 to S133.

[0065] Step S131: Input the knowledge point labels in the preset knowledge point label set into the deep language model to obtain the first knowledge point representation, where the first knowledge point representation includes the representation vectors of each knowledge point label.

[0066] Exemplarily, inputting the knowledge point labels in the knowledge point label set into the deep language model can obtain the semantic representations of all knowledge points, that is, the first knowledge point representation. The principle and steps of the deep language model for semantic modeling of knowledge point labels are the same as those for the aforementioned processing of the target text, and will not be elaborated here. For example, the vector of the starting character [CLS] is also taken as the first knowledge point representation.

[0067] Step S132: Obtain the association information of the knowledge point labels in the knowledge point label set.

[0068] The method of the embodiments of the present application is for the automatic processing and classification of test questions under the Chinese subject system for junior and senior high schools. According to the existing Chinese knowledge point system structure, the test question text is analyzed and processed, and the corresponding knowledge point labels are automatically marked. In some embodiments, the association information is used to describe the tree structure of the knowledge point labels in the knowledge point label set.

[0069] For example, the knowledge points of the Chinese subject are systematic and present a tree structure. Several root nodes contain child nodes, and the child nodes further expand more nodes until the leaf level. The knowledge point system of junior and senior high school Chinese subjects takes question types as the major categories and unfolds in a tree structure. Multiple child nodes are included under the upper-level nodes. There is no explicit successor relationship between knowledge points, and it only distinguishes between junior high school and senior high school, without distinguishing grades, which is significantly different from the knowledge point systems of science subjects such as mathematics. For example, the knowledge points of junior and senior high schools cover five abilities, representing species levels: memorization, which refers to recognition and memory; understanding, which refers to comprehension and the ability to make simple explanations; analysis and synthesis, which refers to decomposition, analysis, and inductive organization; expression and application, which refers to the application of Chinese knowledge and abilities; appreciation and evaluation, which refers to the discrimination, appreciation, and commentary on reading materials. For example, a high school knowledge point system includes five major root nodes: "Language and Writing", "Modern Reading", "Classical Poetry and Prose Reading", "Whole Book Reading", "Writing", etc. The knowledge points under it, such as "Identifying and Correcting Sentence Errors" and "Correctly Using Punctuation Marks", are closely related to the question types of the test questions and have a strong correlation with the proposition methods of the question stems; knowledge points such as "Understanding the Meaning of Important Sentences in the Text" and "Analyzing the Language Features of the Article" have a great relationship with the test angles and question-setting methods of the test questions; and the sibling knowledge points belonging to the same upper-level knowledge point have similar examination methods.

[0070] Exemplarily, the tree structure includes knowledge point labels that are sibling nodes, where sibling nodes refer to different child nodes included under the same upper-level node. For example, under "Identifying and Correcting Sentence Errors", there are multiple knowledge points: "Improper Collocation", "Inverted Word Order", "Fragmented or Redundant Elements", "Structural Confusion", "Illogicality", "Ambiguous Meaning", etc., which are sibling nodes to each other.

[0071] Step S133: Based on the graph neural network, semantically enhance the representation vectors in the first knowledge point representation according to the association information to obtain a second knowledge point representation corresponding to the first knowledge point representation.

[0072] A tree structure is a type of graph, and it can be modeled through a graph neural network. Exemplarily, the graph neural network is a Graph Attention Networks (GAT). At the same time, the structure of the knowledge point tree is modified to model the association relationships between different knowledge points, such as sibling relationships. Of course, it is not limited to this. For example, the association relationships can also include superior-subordinate relationships, etc.

[0073] Exemplarily, after modeling multiple, such as hundreds of knowledge point tags in the knowledge point tag set using a deep language model, the representation vectors of each knowledge point tag obtained are input into the GAT. These knowledge point tags are represented by nodes, and the sibling relationships between these knowledge point tags are described by the edges between the graph nodes. An attention mechanism is used for encoding to obtain a second knowledge point representation with enhanced semantics.

[0074] In some embodiments, for the representation vector of a certain knowledge point , assuming there is a representation vector of a sibling knowledge point , we use the attention mechanism to calculate the importance of as shown in the following formula:

[0075]

[0076]

[0077] Among them, represents the regularization function. Optionally, the specific method of the Attention function is as follows:

[0078]

[0079] Among them, represents the non-linear activation function, represents the parameter of the linear transformation, represents the vector concatenation.

[0080] After using the attention mechanism to calculate the importance of , the representation vector corresponding to the knowledge point in the second knowledge point representation can be determined according to the importance as shown in the following formula:

[0081]

[0082] Among them, represents all the sibling knowledge points of the knowledge point , represents the activation function.

[0083] Similar to the Transformer of the BERT model, the graph attention neural network also has a multi-head mechanism. After independent calculation by different Heads, the representation vectors are concatenated to obtain :

[0084]

[0085] Optionally, at the last layer of the graph attention neural network, the calculation results of different heads can use the mean value instead of concatenation:

[0086]

[0087] After obtaining the semantically enhanced representation vectors corresponding to each knowledge point label in the knowledge point label set the second knowledge point representation corresponding to the knowledge point label set is obtained.

[0088] It can be understood that after obtaining the corresponding second knowledge point representation according to the knowledge point label set, the second knowledge point representation can be stored. When subsequently annotating the target text, the second knowledge representation can be directly obtained. When the knowledge point labels in the knowledge point label set are modified, or the relationships between different knowledge point labels are adjusted, the second knowledge point representation is generated and stored again according to the modified / adjusted knowledge point labels.

[0089] Step S140: Perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point labels corresponding to the target text.

[0090] Exemplarily, the performing a matching process on the text content representation and the knowledge point representation to obtain the knowledge point labels corresponding to the target text includes: performing a matching process on the text content representation and the second knowledge point representation to obtain the knowledge point labels corresponding to the target text.

[0091] In the embodiments of the present application, the text content representation includes the target text, such as the information of the test question stem and the test question answer. The second knowledge point representation integrates the information of the association relationships between different knowledge points and can contain both its own information and the information of sibling nodes, which can make the accuracy of knowledge point recognition higher and the annotation effect better. For example, sibling knowledge points belonging to the same superior knowledge point have similar examination methods. Based on the second knowledge point representation, it can be more accurately determined whether the target text is marked as one or more of the sibling knowledge points.

[0092] In some embodiments, an attention network model can be used to perform a matching process on the text content representation and the second knowledge point representation to obtain the knowledge point labels corresponding to the target text. Of course, this is not limited to this. It should be noted that the matching process of the text content representation and the second knowledge point representation can be semantic matching, or other text matching methods, etc.

[0093] Exemplarily, the matching process of the text content representation and the second knowledge point representation to obtain the knowledge point label corresponding to the target text includes: based on an attention network model, calculating the semantic matching degree between the target text and each knowledge point label according to the text content representation and the second knowledge point representation; and obtaining the knowledge point label corresponding to the target text according to the semantic matching degree between the target text and each knowledge point label.

[0094] Exemplarily, the calculating the semantic matching degree between the target text and each knowledge point label based on the attention network model according to the text content representation and the second knowledge point representation includes: calculating the semantic matching degree between the target text and each knowledge point label based on the attention mechanism sub-model of the attention network model according to the text content representation and the second knowledge point representation.

[0095] The semantic matching degree can be used to indicate the matching degree between the target text and each knowledge point label. Optionally, the knowledge point label with a semantic matching degree greater than a preset threshold can be used as the marking result of the target text, but of course it is not limited to this.

[0096] Exemplarily, a pre-trained BERT model is used to obtain the text content representation , and a GAT is used to obtain the representation vectors of all knowledge points after semantic enhancement , and the attention mechanism can be used again to calculate the semantic matching degree between the target text and each knowledge point label. For example, the semantic matching degree between the target text and the i-th knowledge point label can be expressed as follows:

[0097]

[0098]

[0099] where represents the activation function, , and U are model parameters.

[0100] Exemplarily, the obtaining the knowledge point label corresponding to the target text according to the semantic matching degree between the target text and each knowledge point label includes: performing weighted summation on the representation vectors of each knowledge point label in the second knowledge point representation according to the semantic matching degree of the knowledge point label to obtain the overall knowledge point representation corresponding to the target text; and classifying the overall knowledge point representation and the text content representation based on the classification sub-model of the attention network model to obtain the knowledge point label corresponding to the target text.

[0101] By calculating the semantic matching degree between the target text and each knowledge point label, and performing weighted summation to obtain the overall representation of the knowledge points, it is beneficial for the model to determine several most suitable knowledge point labels for the target text.

[0102] For example, perform weighted summation on the representation vectors of each knowledge point label in the second knowledge point representation to obtain the overall representation of the knowledge points corresponding to the target text , which can be expressed as follows:

[0103]

[0104] Please refer to Figure 5 , splice the overall representation of the knowledge points and the representation of the text content to obtain the final representation vector of the entire model , as expressed as .

[0105] Exemplarily, process the final representation vector based on the classification sub-model of the attention network model , to obtain the knowledge point labels corresponding to the target text. For example, the classification sub-model includes a fully connected layer and a Sigmoid function, and of course it is not limited to this.

[0106] It should be noted that the embodiments of the present application are not limited to processing the representation of the text content and the second knowledge point representation to obtain the knowledge point labels corresponding to the target text. For example, the representation of the text content and the first knowledge point representation can be processed to obtain the knowledge point labels corresponding to the target text, and it can also effectively utilize the correlation relationship existing between different knowledge point labels, making the accuracy of knowledge point recognition higher and the annotation effect better; the principle and steps of the processing are the same as those for processing the representation of the text content and the second knowledge point representation, and will not be elaborated here.

[0107] The knowledge point annotation method provided by the embodiments of the present application obtains the representation of the text content corresponding to the target text based on a deep language model, and performs matching processing on the representation of the text content and the knowledge point representation corresponding to the knowledge point label set to obtain the knowledge point labels corresponding to the target text; based on the deep language model, deeper semantic modeling can be performed on the target text, and the obtained representation of the text content can also better adapt to the multi-label classification task; the knowledge point representation contains the information of the knowledge point system, and combined with the representation of the text content, it can improve the recognition effect of the knowledge points.

[0108] In some embodiments, the embodiments of the present application can implement a knowledge point automatic annotation system that combines a pre-trained language model and a graph neural network. Exemplarily, a graph neural network is introduced to encode the knowledge point system to obtain a better representation of the knowledge points, which is combined with the representation of the text content to improve the recognition effect.

[0109] Exemplarily, when encoding the knowledge point system, the graph neural network can integrate the hierarchical structure of the knowledge point system itself. For example, there are association relationships among different knowledge point tags, especially the effective information that the sibling knowledge points belonging to the same superior knowledge point have similar examination methods, which can improve the recognition effect.

[0110] Please refer to the foregoing embodiments in conjunction with Figure 6 , and the embodiments of the present application further provide a training method for a knowledge point annotation model. Please refer to Figure 5 and Figure 6 , and the training method includes steps S210 to S260.

[0111] Step S210: Obtain the knowledge point representation corresponding to the preset knowledge point tag set, where the knowledge point representation is obtained by processing the knowledge point tags in the knowledge point tag set based on a deep language model.

[0112] Exemplarily, obtain the first knowledge point representation corresponding to the preset knowledge point tag set or the second knowledge point representation corresponding to the knowledge point tag set.

[0113] For example, exemplarily, the obtaining of the knowledge point representation corresponding to the preset knowledge point tag set includes: inputting the knowledge point tags in the preset knowledge point tag set into the deep language model to obtain the first knowledge point representation corresponding to the knowledge point tag set, where the first knowledge point representation includes the representation vectors of each knowledge point tag; obtaining the association information of the knowledge point tags in the knowledge point tag set, where the association information is used to describe the tree structure of the knowledge point tags in the knowledge point tag set, and the tree structure includes knowledge point tags that are sibling nodes; based on the graph neural network of the knowledge point annotation model, semantically enhance the representation vectors in the first knowledge point representation according to the association information to obtain the second knowledge point representation corresponding to the first knowledge point representation.

[0114] Step S220: Obtain the target text and the knowledge point annotation of the target text.

[0115] The knowledge point annotation of the target text is, for example, the knowledge point tags manually annotated, and the knowledge point tags marked for a target text can be one or more.

[0116] Exemplarily, the target text includes a preset start character, a test question stem, a preset interval character, and a test answer.

[0117] Step S230: Input the target text into the deep language model of the knowledge point annotation model to obtain the text content representation corresponding to the target text.

[0118] In some embodiments, the deep language model is a pre-trained BERT model, although it is not limited thereto.

[0119] Step S240: Perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text.

[0120] Exemplarily, the performing a matching process on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text includes: performing a matching process on the text content representation and the second knowledge point representation to obtain the knowledge point label corresponding to the target text.

[0121] In some embodiments, based on the attention network model of the knowledge point annotation model, perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text. Exemplarily, based on the attention network model, calculate the semantic matching degree between the target text and each knowledge point label according to the text content representation and the second knowledge point representation; and obtain the knowledge point label corresponding to the target text according to the semantic matching degree between the target text and each knowledge point label.

[0122] For example, based on the attention mechanism sub-model of the attention network model, calculate the semantic matching degree between the target text and each knowledge point label according to the text content representation and the second knowledge point representation.

[0123] For example, according to the semantic matching degree of the knowledge point label, perform a weighted sum on the representation vectors of each knowledge point label in the second knowledge point representation to obtain the overall knowledge point representation corresponding to the target text; based on the classification sub-model of the attention network model, classify the overall knowledge point representation and the text content representation to obtain the knowledge point label corresponding to the target text.

[0124] Step S250: Based on a preset loss function, determine the model loss value according to the knowledge point annotation of the target text and the knowledge point label corresponding to the target text.

[0125] For example, use the Binary Cross Entropy loss function to determine the model loss value L (loss), as shown in the following formula:

[0126]

[0127] Step S260: Adjust the parameters of the knowledge point annotation model according to the model loss value.

[0128] In some embodiments, the parameters of at least one of the deep language model, the attention network model, and the graph neural network in the knowledge point annotation model can be adjusted; for example, the parameters of the deep language model and the graph neural network are fixed, and the parameters of the attention network model are adjusted, but of course it is not limited to this.

[0129] Exemplarily, when the model loss value converges to a preset range, the training ends, and a trained knowledge point annotation model is obtained. Based on the trained knowledge point annotation model, the steps of the aforementioned knowledge point annotation method can be executed.

[0130] The specific principles and implementation manners of the training method of the knowledge point annotation model provided by the embodiments of the present application are similar to those of the knowledge point annotation method of the foregoing embodiments, and will not be elaborated herein.

[0131] The method of the present application can be used in many general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on.

[0132] Exemplarily, the above method can be implemented in the form of a computer program, and the computer program can run on a computer device as shown in Figure 7 shown.

[0133] Please refer to Figure 7 , Figure 7 , which is a schematic structural diagram of a computer device provided by an embodiment of the present application. The computer device can be a server or a terminal.

[0134] Referring to Figure 7 , the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory can include a non-volatile storage medium and an internal memory.

[0135] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute the steps of any of the foregoing methods.

[0136] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0137] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute the steps of any of the foregoing methods.

[0138] The network interface is used for network communication, such as sending the assigned tasks, etc. Those skilled in the art can understand that the structure of the computer device is only a block diagram of the part of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0139] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] Among them, in one embodiment, the processor is used to run the computer program stored in the memory to implement the following steps:

[0141] Obtain the target text;

[0142] Input the target text into a preset deep language model to obtain the text content representation corresponding to the target text;

[0143] Obtain the knowledge point representation corresponding to the preset knowledge point label set, where the knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model;

[0144] Perform a matching process on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text.

[0145] Among them, in one embodiment, the processor is used to run the computer program stored in the memory to implement the following steps:

[0146] Obtain the knowledge point representation corresponding to the preset knowledge point label set, where the knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model;

[0147] Obtain the target text and the knowledge point annotation of the target text;

[0148] Input the target text into the deep language model of the knowledge point annotation model to obtain the text content representation corresponding to the target text;

[0149] Perform matching processing on the text content representation and the knowledge point representation to obtain the knowledge point label corresponding to the target text;

[0150] Based on a preset loss function, determine the model loss value according to the knowledge point annotation of the target text and the knowledge point label corresponding to the target text;

[0151] Adjust the parameters of the knowledge point annotation model according to the model loss value.

[0152] From the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of this application, such as:

[0153] A computer-readable storage medium stores a computer program, and the computer program includes program instructions. The processor executes the program instructions to implement the steps of any one of the methods provided in the embodiments of this application.

[0154] Among them, the computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device.

[0155] As described above, the above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for knowledge point annotation, characterized in that, Including: Obtain the target text; Perform word embedding processing on the target text based on the word embedding layer of the deep language model to obtain a text vector; Process the text vector based on the multi-head self-attention network of the deep language model to obtain the text content representation corresponding to the target text; Obtain the second knowledge point representation corresponding to the preset knowledge point label set, where the second knowledge point representation is obtained by semantically enhancing the first knowledge point representation based on a graph neural network, and the first knowledge point representation is obtained by processing the knowledge point labels in the knowledge point label set based on the deep language model; Perform matching processing on the text content representation and the second knowledge point representation to obtain the knowledge point label corresponding to the target text; Among them, the obtaining of the second knowledge point representation corresponding to the preset knowledge point label set includes: Input the knowledge point labels in the preset knowledge point label set into the deep language model to obtain the first knowledge point representation, where the first knowledge point representation includes the representation vectors of each knowledge point label; Obtain the association information of the knowledge point labels in the knowledge point label set; Based on the graph neural network, semantically enhance the representation vectors in the first knowledge point representation according to the association information to obtain the second knowledge point representation corresponding to the first knowledge point representation, and the graph neural network models the association relationship between different knowledge points.

2. The knowledge point annotation method according to claim 1, wherein The obtaining of the target text includes: Obtain the test question stem and the test question answer; Generate the target text according to the preset start character, the test question stem, the preset interval character, and the test question answer.

3. The knowledge point annotation method according to claim 1, wherein, The performing of the matching processing on the text content representation and the second knowledge point representation to obtain the knowledge point label corresponding to the target text includes: Based on the attention network model, calculate the semantic matching degree between the target text and each knowledge point label according to the text content representation and the second knowledge point representation; and Obtain the knowledge point label corresponding to the target text according to the semantic matching degree between the target text and each knowledge point label.

4. The knowledge point annotation method according to claim 3, characterized in that The calculating of the semantic matching degree between the target text and each knowledge point label based on the attention network model according to the text content representation and the second knowledge point representation includes: Based on the attention mechanism sub-model of the attention network model, calculate the semantic matching degree between the target text and each knowledge point label according to the text content representation and the second knowledge point representation; The obtaining of the knowledge point label corresponding to the target text according to the semantic matching degree between the target text and each knowledge point label includes: Perform weighted summation on the representation vectors of each knowledge point label in the second knowledge point representation according to the semantic matching degree of the knowledge point label to obtain the overall knowledge point representation corresponding to the target text; Based on the classification sub-model of the attention network model, classify the overall knowledge point representation and the text content representation to obtain the knowledge point label corresponding to the target text.

5. A training method for a knowledge point annotation model, characterized in that Including: Obtain a second knowledge point representation corresponding to a preset set of knowledge point tags, where the second knowledge point representation is obtained by semantically enhancing a first knowledge point representation based on a graph neural network, and the first knowledge point representation is obtained by processing the knowledge point tags in the set of knowledge point tags based on a deep language model; Obtain a target text and the knowledge point annotation of the target text; Input the target text into the deep language model of the knowledge point annotation model to obtain a text content representation corresponding to the target text; Perform a matching process on the text content representation and the second knowledge point representation to obtain the knowledge point tags corresponding to the target text; Based on a preset loss function, determine a model loss value according to the knowledge point annotation of the target text and the knowledge point tags corresponding to the target text; Adjust the parameters of the knowledge point annotation model according to the model loss value; Among them, the obtaining of the second knowledge point representation corresponding to the preset set of knowledge point tags includes: Input the knowledge point tags in the preset set of knowledge point tags into the deep language model to obtain a first knowledge point representation corresponding to the set of knowledge point tags, and the first knowledge point representation includes the representation vectors of each knowledge point tag; Obtain the association information of the knowledge point tags in the set of knowledge point tags; Based on the graph neural network of the knowledge point annotation model, semantically enhance the representation vectors in the first knowledge point representation according to the association information to obtain a second knowledge point representation corresponding to the first knowledge point representation, and the graph neural network models the association relationship between different knowledge points.

6. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store a computer program; The processor is configured to execute the computer program and, when executing the computer program, implement: The steps of the knowledge point annotation method according to any one of claims 1-4; or The steps of the training method of the knowledge point annotation model according to claim 5.

7. A computer-readable storage medium storing a computer program, characterized in that, If the computer program is executed by the processor, it implements: The steps of the knowledge point annotation method according to any one of claims 1-4; or The steps of the training method of the knowledge point annotation model according to claim 5.

Images