Multi-source fusion knowledge point labeling method and device based on hypergraph knowledge enhancement

Abstract

The invention provides a multi-source fusion knowledge point labeling method and device based on hypergraph knowledge enhancement, and relates to the technical field of knowledge point labeling, and the method comprises the steps: inputting a knowledge point text set into a large language model to obtain a knowledge element system matrix and a knowledge point incidence matrix, constructing a hypergraph matrix according to the knowledge point text set, the knowledge element system matrix and the knowledge point incidence matrix, and inputting the hypergraph matrix into a hypergraph convolutional neural network to obtain a hyperedge matrix; inputting the test question text into a large language model to obtain a vocabulary sequence, inputting the vocabulary sequence into a bidirectional semantic encoder to obtain a vocabulary vector matrix, and inputting the vocabulary vector matrix into a bidirectional long-short-term memory network to obtain a vocabulary feature matrix; splicing the hyperedge matrix and the vocabulary feature matrix to obtain a spliced feature matrix, and inputting the spliced feature matrix into a multilayer perception neural network to obtain a knowledge point prediction matrix; and performing knowledge point labeling on the test question text according to the knowledge point prediction matrix to obtain all predicted knowledge point texts of the test question text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge point labeling, and in particular to a multi-source fusion knowledge point labeling method and device based on supergraph knowledge enhancement. BACKGROUND

[0002] Mathematics questions are different from natural language processing in that they have strong subject field properties. Mathematics question texts are highly abstract and condensed in expression, which is manifested as being short and concise, especially in the case of fill-in-the-blank. Existing deep learning models have difficulty extracting all features of mathematics questions, and the internal logic and features are difficult to obtain directly through semantic information. If semantic explanations are added for these question types, the cost is too high in the face of a large amount of question data in the database.

[0003] In addition, mathematical knowledge point information cannot be ignored. The semantic information of knowledge point texts is a direct manifestation of knowledge point features. However, existing technologies cannot comprehensively capture the semantic information of knowledge point texts, resulting in inaccurate labeling results.

[0004] Meanwhile, in actual scenarios, mathematics questions are often related to one or more knowledge points, and there is similarity between knowledge points, and knowledge points under the same knowledge system are more likely to be combined together for examination, so the relationship between knowledge points is not simply one-to-one, but a more complex many-to-many topological structure. The ordinary graph used in existing technologies cannot reflect this many-to-many topological structure, resulting in the inability to represent the many-to-many relationship between knowledge points and the inability to capture comprehensive semantic information between knowledge points, which reduces the accuracy of knowledge point labeling. SUMMARY

[0005] To solve the above problems, the present application provides a multi-source fusion knowledge point labeling method based on supergraph knowledge enhancement, comprising the steps of: obtaining a knowledge point text set, inputting the knowledge point text set into a large language model to obtain a knowledge element system matrix and a knowledge point association matrix, constructing a supergraph matrix according to the knowledge point text set, the knowledge element system matrix and the knowledge point association matrix, and inputting the supergraph matrix into a supergraph convolutional neural network to obtain a superedge matrix; obtaining a question text, inputting the question text into a large language model to obtain a word sequence, inputting the word sequence into a bidirectional semantic encoder to obtain a word vector matrix, and inputting the word vector matrix into a bidirectional long short-term memory network to obtain a word feature matrix; splicing the superedge matrix and the word feature matrix to obtain a spliced feature matrix, and inputting the spliced feature matrix into a multilayer perception neural network to obtain a knowledge point prediction matrix; labeling the question text according to the knowledge point prediction matrix to obtain all predicted knowledge point texts of the question text.

[0006] Optionally, the knowledge point text set is input into a large language model to obtain a knowledge element system matrix and a knowledge point correlation matrix, specifically including: Each knowledge point text in the knowledge point text set is input into the large language model, and multiple knowledge point texts with a similarity greater than a preset value are attributed to the same knowledge element. The knowledge element system matrix is constructed according to the knowledge elements. Each sample test text in the sample test text library is obtained, and the frequency of multiple knowledge point texts co-occurring in a sample test text is obtained through the large language model. The knowledge point correlation matrix is constructed according to the frequency of multiple knowledge point texts co-occurring.

[0007] Optionally, the hypergraph matrix is constructed according to the knowledge point text set, the knowledge element system matrix and the knowledge point correlation matrix, specifically including: The knowledge point texts in the knowledge point text set are taken as nodes in the hypergraph. The relationships between different knowledge point texts in the knowledge point text set are taken as hyperedges in the hypergraph. The hypergraph is constructed according to the nodes and hyperedges. The hypergraph is subjected to high-order information extraction according to the knowledge element system matrix and the knowledge point correlation matrix, and the high-order information between different knowledge point texts is obtained. The hypergraph matrix is composed of the high-order information between different knowledge point texts.

[0008] Optionally, the hypergraph matrix is input into a hypergraph convolutional neural network to obtain a hyperedge matrix, specifically including: The node belonging hyperedge number matrix, the hyperedge importance weight matrix and the hyperedge containing node number matrix are obtained according to the hypergraph. The node belonging hyperedge number matrix, the hypergraph matrix, the hyperedge importance weight matrix and the hyperedge containing node number matrix are multiplied in turn to obtain a normalized correlation matrix. The hypergraph convolutional neural network includes multiple convolutional layers. The hypergraph matrix is input into the first convolutional layer. The output matrix of the first convolutional layer is calculated and obtained according to the normalized correlation matrix and the hypergraph matrix . From the beginning, the output matrix of the first convolutional layer is input into the second convolutional layer. The output matrix of the second convolutional layer is calculated and obtained according to the normalized correlation matrix and the output matrix . The output matrix of the second convolutional layer is input into the third convolutional layer. The output matrix of the third convolutional layer is calculated and obtained according to the normalized correlation matrix and the output matrix . The output matrix of the third convolutional layer is input into the fourth convolutional layer. The output matrix of the fourth convolutional layer is calculated and obtained according to the normalized correlation matrix and the output matrix . The output matrix of the last convolutional layer is taken as the hyperedge matrix.

[0009] Optionally, the test text is input into a large language model to obtain a word sequence. The word sequence is input into a bidirectional semantic encoder to obtain a word vector matrix, specifically including: S11: split the test question text into multiple words by a large language model, obtain a time step of each word, and construct a word sequence according to the words and the time steps; S12: extract the t-th word in the word sequence, and obtain a word embedding vector, a sentence embedding vector and a position embedding vector by respectively processing the t-th word through word embedding processing, sentence embedding processing and position embedding processing, wherein t represents a time step; S13: add the word embedding vector, the sentence embedding vector and the position embedding vector to obtain a total embedding vector of the t-th word, input the total embedding vector into a bidirectional semantic encoder for multi-layer encoding to obtain a word vector of the t-th word; S14: repeat steps S12-S13 until the word vectors of all words are obtained, and a word vector matrix is formed by the word vectors of all words.

[0010] Optionally, input the word vector matrix into a bidirectional long short-term memory network to obtain a word feature matrix, specifically including: S21: the bidirectional long short-term memory network includes a forward long short-term memory network and a backward long short-term memory network; S22: obtain a forward time step state of the t-th word vector and a t-1-th word vector in the word vector matrix, wherein t represents a time step; input the word vector and the forward time step state into the forward long short-term memory network, calculate a forward output feature and a forward candidate memory according to the word vector and the forward time step state , and calculate a forward hidden state of the t-th word vector according to the forward output feature and the forward candidate memory ; S23: obtain a backward time step state of the t-1-th word vector ; input the word vector and the backward time step state into the backward long short-term memory network, calculate a backward output feature and a backward candidate memory according to the word vector and the backward time step state , and calculate a backward hidden state of the t-th word vector according to the backward output feature and the backward candidate memory . backward hidden state ; S24: splicing the forward hidden state and the backward hidden state to obtain the vocabulary feature of the tth vocabulary vector ; ; S25: repeating steps S22-S24 until the vocabulary features of all vocabulary vectors are obtained, and the vocabulary feature matrix is constituted by the vocabulary features of all vocabulary vectors.

[0011] Optionally, after obtaining all the predicted knowledge point texts of the test question text, the method further comprises: obtaining the knowledge point text label of the test question text, calculating the score of each predicted knowledge point text according to the probability of each predicted knowledge point text in the knowledge point prediction matrix and the knowledge point text label, calculating the loss value according to the score of each predicted knowledge point text, and adjusting the parameters of the hypergraph convolutional neural network and the bidirectional long short-term memory network according to the loss value.

[0012] The application further provides a multi-source fusion knowledge point labeling device based on hypergraph knowledge enhancement, which is used to realize the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement. The hyperedge matrix acquisition module is configured to acquire the knowledge point text set, input the knowledge point text set into a large language model to obtain a knowledge element system matrix and a knowledge point correlation matrix, construct a hypergraph matrix according to the knowledge point text set, the knowledge element system matrix and the knowledge point correlation matrix, and input the hypergraph matrix into a hypergraph convolutional neural network to obtain a hyperedge matrix. The vocabulary feature matrix acquisition module is configured to acquire the test question text, input the test question text into a large language model to obtain a vocabulary sequence, input the vocabulary sequence into a bidirectional semantic encoder to obtain a vocabulary vector matrix, and input the vocabulary vector matrix into a bidirectional long short-term memory network to obtain a vocabulary feature matrix. The knowledge point prediction matrix acquisition module is configured to splice the hyperedge matrix and the vocabulary feature matrix to obtain a spliced feature matrix, and input the spliced feature matrix into a multilayer perception neural network to obtain a knowledge point prediction matrix. The knowledge point text prediction module is configured to perform knowledge point labeling on the test question text according to the knowledge point prediction matrix to obtain all the predicted knowledge point texts of the test question text.

[0013] The application further provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement when executing the program.

[0014] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the knowledge point labeling method based on hypergraph knowledge enhancement and multi-source fusion.

[0015] The application has the following beneficial effects: 1. The reasoning ability of a large language model is used to construct a knowledge point text knowledge element system matrix, so as to associate knowledge points under the same knowledge element and capture comprehensive semantic information between the knowledge points; a knowledge point association matrix is constructed by counting the frequency relationship of the common occurrence of knowledge points in test questions, and a hypergraph matrix is constructed based on the knowledge point text set, the knowledge element system matrix and the knowledge point association matrix; a hyperedge matrix is constructed by using a hypergraph convolutional neural network to aggregate high-order information between the knowledge points, and the hyperedge matrix can reflect the many-to-many relationship of the knowledge points, thereby improving the accuracy of knowledge point labeling. 2. A bidirectional semantic encoder is used to encode and embed test question texts to obtain a vocabulary vector matrix, a bidirectional long short-term memory network is used to capture the long dependency relationship of each vocabulary in a sentence in the test question analysis text, and the overall features of the sentence are mined while combining the information flow in two directions, so as to comprehensively capture the vocabulary feature matrix of the test question text; the hyperedge matrix and the vocabulary feature matrix are spliced, and multiple source matrices are fused to jointly predict the knowledge point text, thereby significantly improving the accuracy of the predicted knowledge point text. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The method flowchart of the embodiment of the application is shown in the figure. Figure 2 The structure diagram of the knowledge element is shown in the figure. Figure 3 The structure diagram of the hypergraph is shown in the figure. Figure 4 The execution principle diagram of the hypergraph convolutional neural network is shown in the figure. Figure 5 The structure diagram of the bidirectional long short-term memory network is shown in the figure. Figure 6 The experimental result diagram of different text embedding methods is shown in the figure. Figure 7 The result diagram of the influence of multi-source fusion representation on labeling is shown in the figure. Figure 8 The experimental effect diagram of different models on the TMK-PSS-LLM data set is shown in the figure. The implementation, functional characteristics and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0018] With reference to Figure 1 The present application provides a multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement, comprising the following steps: Obtaining a knowledge point text set, inputting the knowledge point text set into a large language model to obtain a knowledge element system matrix and a knowledge point correlation matrix, constructing a hypergraph matrix according to the knowledge point text set, the knowledge element system matrix and the knowledge point correlation matrix, and inputting the hypergraph matrix into a hypergraph convolution neural network to obtain a hyperedge matrix.

[0019] In some embodiments, the knowledge point text set is input into the large language model to obtain the knowledge element system matrix and the knowledge point correlation matrix, and specifically comprising: Inputting each knowledge point text in the knowledge point text set into the large language model, attributing multiple knowledge point texts with a similarity greater than a preset value to the same knowledge element, and constructing a knowledge element system matrix according to the knowledge element; In some embodiments, the relationship between knowledge points is complex, and the examination of a knowledge point in a mathematical test is often accompanied by other knowledge points. The same knowledge element indicates that the knowledge points have a certain similarity, and the performance of this relationship in the specific case of a test is that they are examined together. Therefore, the different knowledge points can also form a triple of "entity-relation-entity". In order to obtain the relationship between knowledge points and connect these entities together as a feature for model learning.

[0020] The present application performs the following operations: first, all knowledge points are input into a large model as texts, the large model establishes these relationships and constructs corresponding knowledge elements through entity recognition, relationship extraction and the like, the structure of the knowledge element is as shown in Figure 2 The knowledge element "time and direction" includes the knowledge point texts "hour, minute and second", "year, month and day", "angle", "position and direction", "understanding clock face", and then these knowledge elements are connected with the knowledge points (specifically generating knowledge elements such as "integer operation", "problem solving", "figure and geometry", "time and direction", etc.), the horizontal relationship between knowledge points is obtained, that is, the knowledge element system matrix is recorded as .

[0021] Obtain each sample test text in the sample test text library, obtain the frequency of multiple knowledge point texts co-occurring in a sample test text through a large language model, and construct a knowledge point association matrix according to the frequency of multiple knowledge point texts co-occurring.

[0022] In some embodiments, the most direct manifestation of the relationship between knowledge points is the frequency of multiple knowledge points co-occurring in a test in the data set. The knowledge point association matrix is constructed by counting this rule in the data set. If a knowledge point co-occurs with other knowledge points, the operation +1 is performed. The knowledge point association matrix obtained after traversing the entire data set is denoted as , and the matrix shape is , where is the number of knowledge points. It can be known that this matrix is a diagonal matrix, and the dimension of its is taken to constitute one of the hyperedge subsets.

[0023] In some embodiments, the supergraph matrix is constructed according to the knowledge point text set, the knowledge element system matrix, and the knowledge point association matrix, specifically including: The knowledge point texts in the knowledge point text set are taken as nodes in the supergraph, the relationships between different knowledge point texts in the knowledge point text set are taken as hyperedges in the supergraph, and the supergraph is constructed according to the nodes and hyperedges; High-order information extraction is performed on the supergraph according to the knowledge element system matrix and the knowledge point association matrix to obtain high-order information between different knowledge point texts, and the supergraph matrix is constructed by the high-order information between different knowledge point texts.

[0024] In some embodiments, each node in the supergraph is a knowledge point text, and the node set , is the vector of the ith knowledge point text, and c is the number of knowledge point texts. The connection relationship between knowledge points constitutes the hyperedge set of the graph, and the hyperedge set represents the connection between knowledge points. The hyperedge set , is the vector of the ith hyperedge, and D is the number of hyperedges. Assuming that the hyperedge weights are the same, the nodes and hyperedges jointly constitute the supergraph , and the structure of the supergraph is shown in Figure 3 . The knowledge element system matrix and the knowledge point association matrix constructed in the foregoing are combined to capture high-order information between knowledge points from the horizontal and vertical directions, and the finally obtained supergraph matrix is denoted as .

[0025] In some embodiments, the relationship between knowledge points is not a simple one-to-one relationship. The examination of one knowledge point is often accompanied by multiple knowledge points. The Hypergraph Convolutional Neural Network (HGCN) is used to capture the high-order relationship between knowledge points. The node matrix can be propagated to the hyperedge through the association matrix, and finally aggregated to the node, so that the model can learn the hierarchical high-order representation of node features and hyperedges. Finally, the knowledge point annotation is completed by combining the test question representation. The execution principle diagram of the Hypergraph Convolutional Neural Network is shown in the figure below. Figure 4 shown.

[0026] The hypergraph matrix is ​​input into the hypergraph convolutional neural network to obtain the hyperedge matrix, which specifically includes: According to the hypergraph, the hyperedge quantity matrix, the hyperedge importance weight matrix and the hyperedge included node quantity matrix of the node are obtained, and the normalized correlation matrix is ​​obtained by multiplying the hyperedge quantity matrix, the hypergraph matrix, the hyperedge importance weight matrix and the hyperedge included node quantity matrix in sequence. In some embodiments, the formula for the normalized incidence matrix is ​​as follows:

[0027] in is the normalized incidence matrix, is the matrix of the number of hyperedges to which the node belongs, is the matrix of the number of nodes included in the hyperedge, is the hyperedge importance weight matrix, and H is the hypergraph matrix.

[0028] The hypergraph convolutional neural network consists of multiple convolutional layers. The hypergraph matrix is ​​input into the first convolutional layer, and the output matrix of the first convolutional layer is calculated based on the normalized correlation matrix and the hypergraph matrix. ; from Start by The output matrix of the convolutional layer Enter Convolutional layers, based on the normalized correlation matrix and output matrix Calculate the first The output matrix of the convolutional layer , the output matrix of the last convolutional layer is used as the hyperedge matrix.

[0029] In some embodiments, the hyperedge weights are dynamically learned to continuously aggregate the node information connected by the hyperedge. As the labeling task builds the association matrix to eliminate the degree difference between the nodes, the hyperedge size difference is weighted and eliminated, so that the knowledge points can better aggregate the hyperedge information and form the output features of the knowledge points of the next convolutional layer. The output matrix The formula is as follows:

[0030] wherein, is a Laplacian matrix composed of normalized association matrices, is a learnable linear variation parameter.

[0031] Obtain the test question text, input the test question text into the large language model to obtain the vocabulary sequence, input the vocabulary sequence into the bidirectional semantic encoder to obtain the vocabulary vector matrix, and input the vocabulary vector matrix into the bidirectional long short-term memory network to obtain the vocabulary feature matrix; In some embodiments, the large language model has strong language understanding and generation capabilities. It can quickly understand the semantics and knowledge points of the test questions, generate comprehensive and accurate analysis based on the knowledge and patterns learned from massive data. Compared with artificial, the large model is not limited by time and effort, and can process a large number of test questions at the same time, greatly improving work efficiency. And the generated analysis is relatively uniform in style and content structure, which can guarantee a certain standardization.

[0032] The large language model of the application adopts DeepSeek-V3, which greatly exceeds other large language models in Chinese processing capability and mathematical reasoning capability, and has high cost performance. First, the test question text is batch processed and the large model API is called, the corresponding prompt is constructed to obtain the test question analysis information generated by the large model, and real-time monitoring and request management are performed during the generation process to avoid connection drop due to network problems, which may cause the generated content to be missing. In addition, save the generated content log to locate the missing point of the generated content and request the API again. Then, accuracy verification is performed, the prompt is iterated manually according to the generated content to ensure the generation quality.

[0033] In the prompt construction process, a hierarchical prompting strategy is adopted to optimize the generation effect, which is implemented as follows. First, the role of the large model is positioned and set, for example, it is indicated that it is a senior mathematics teacher, please generate detailed analysis for the following questions, and the test question content is the main input context. Secondly, different prompts are generated for test questions with different knowledge points and different types of test questions to control the output requirements of the test question analysis. Specifically, for example, when solving the selection fill-in-the-blank question with the knowledge point of "chickens and rabbits in the same cage", it is required to give the solution process step by step. During the generation process, the output content is evaluated and the prompt is changed.

[0034] Finally, the output content is evaluated after output, a sampling evaluation method is adopted to randomly select test questions under different question types and different knowledge point systems to ensure the reliability of the generated data, and then the generated formula is matched for data cleaning to reduce the noise of feature extraction.

[0035] In some embodiments, for the test text representation, this paper first uses a bidirectional semantic encoder (Bert) to embed word vectors. Bert has obvious advantages over traditional embedding in the process of generating dynamic context-related word vectors and can better capture the semantic information of words.

[0036] Input the test text into the large language model to obtain a vocabulary sequence, and then input the vocabulary sequence into the bidirectional semantic encoder to obtain a vocabulary vector matrix. Specifically, it includes: S11: Use the large language model to segment the test text into multiple words, obtain the time step of each word, and construct a word sequence based on the words and time steps; S12: Extract the t-th word in the word sequence, and process the t-th word through word embedding, sentence embedding, and position embedding to obtain a word embedding vector, a sentence embedding vector, and a position embedding vector, where t represents the time step; S13: Add the word embedding vector, sentence embedding vector and position embedding vector to obtain the total embedding vector of the t-th word, input the total embedding vector into the bidirectional semantic encoder for multi-layer encoding, and obtain the vocabulary vector of the t-th word; In some embodiments, Bert embedding mainly includes word embedding, sentence embedding and position embedding. After each input is word embedded, each word will be mapped into a word vector, that is, the Token part. At the same time, after word segmentation, Segment information and Position information will be added to mark the location of the sentence and the absolute position encoding of each token in the sequence, respectively. The word sequence is recorded as , represents the tth word, Represents the position code of the t-th word;

[0037] Where m is the number of words, [CLS] represents the tag of the sequence start task, [SEP] represents the separation tag, and the word sequence By using the WordPiece algorithm to further obtain word embedding for each input sample, the total embedding vector of the t-th word The expression is:

[0038] in Respectively represent word embedding processing, sentence embedding processing and position embedding processing; The vocabulary vector of the t-th word is the hidden state of the last layer of Bert; S14: Repeat steps S12-S13 until the vocabulary vectors of all words are obtained, and the vocabulary vector matrix is ​​formed by the vocabulary vectors of all words.

[0039] In some embodiments, BERT is composed of multiple layers of bidirectional Transformers. When using BERT to obtain the representation of the test text, [CLS] is generally marked at the beginning to represent the semantic vector of the entire sentence. This has a good effect on general text classification, but it also loses the semantic information of keywords. A single [CLS] vector is not sufficient to represent the semantics of the entire sentence, so the hidden state of the last layer of BERT output is obtained to obtain the vocabulary vector of each word in the sentence.

[0040] In some embodiments, the vocabulary vector matrix is ​​input into a bidirectional long short-term memory network to obtain a vocabulary feature matrix, specifically including: S21: Bidirectional long short-term memory network includes forward long short-term memory network and backward long short-term memory network; In some embodiments, the structure of the BiLSTM network is as follows: Figure 5 As shown in Figure 1, the forward LSTM network and the backward LSTM network are composed of multiple LSTM networks with opposite directions.

[0041] BiLSTM can capture the dependencies between words in the context and simultaneously capture both preceding and following context for each word in the test text. This integration of information from both directions allows for a more comprehensive understanding of the text's contextual semantics. For math test annotation, each word has a deep connection to its context. The advantages of BiLSTM are particularly evident when the incorporation of parsing features lengthens the sentence. By combining forward and backward long short-term memory networks, BiLSTM simultaneously captures information from both preceding and following contexts in the sequence, resulting in a representation of the entire sentence.

[0042] S22: Get the tth word vector in the word vector matrix and the t-1th word vector The forward time step state , where t represents the time step; The vocabulary vector and the forward time step state Input forward long short-term memory network, according to the vocabulary vector and the forward time step state Calculate the forward output features and forward candidate memory , according to the forward output feature and forward candidate memory Calculate the tth word vector The forward hidden state ; In some embodiments, for the tth vocabulary vector The calculation process is as follows:

[0043]

[0044]

[0045] in is the forward input of the input gate, is the weight matrix of the forward input, is the bias term, is the sigmoid activation function, is the forward output of the forget gate, is the forward output feature of the output gate, is the forward candidate memory, is the weight matrix of the forward candidate memory. During the training process, the memory information will be gradually aggregated, the cell state will be updated, and the forward candidate memory will be and forward output features The formula for performing the previous selective forgetting update is as follows:

[0046]

[0047] Merge the output gate and the updated merged state to get the information state of the entire sentence. The specific formula is as follows:

[0048] in Represents vector dot product; S23: Get the t-1th vocabulary vector The backward time step state ; The vocabulary vector and the backward time step state Input the backward long short-term memory network, according to the vocabulary vector and the backward time step state Calculate the backward output features and backward candidate memory , according to the backward output feature and backward candidate memory Calculate the tth word vector The backward hidden state ; In some embodiments, the backward hidden state The calculation process of is consistent with the forward hidden state; S24: forward hidden state and the backward hidden state concatenation to obtain the t-th word vector word feature ; In some embodiments, the word feature is calculated as follows:

[0049] where cat() represents the concatenation operation. S25: repeating steps S22-S24 until the word features of all word vectors are obtained, and the word feature matrix is composed of the word features of all word vectors.

[0050] The super-edge matrix and the word feature matrix are concatenated to obtain a concatenated feature matrix, and the concatenated feature matrix is input into a multi-layer perception neural network to obtain a knowledge point prediction matrix. In some embodiments, the concatenated feature matrix is expressed as:

[0051] wherein, is the word feature matrix, is the super-edge matrix, L is the number of convolutional layers of the supergraph convolutional neural network, cat() represents the concatenation operation, and pool() represents the pooling operation. The expression of the knowledge point prediction matrix is as follows:

[0052] wherein, MLP() represents the multi-layer perception neural network.

[0053] According to the knowledge point prediction matrix, the knowledge point labels of the test question text are obtained.

[0054] In some embodiments, after obtaining the predicted knowledge point text of the test question text, the method further comprises: obtaining the knowledge point text label of the test question text, calculating the score of each predicted knowledge point text according to the probability of each predicted knowledge point text in the knowledge point prediction matrix and the knowledge point text label, calculating the loss value according to the score of each predicted knowledge point text, and adjusting the parameters of the supergraph convolutional neural network and the bidirectional long short-term memory network according to the loss value.

[0055] In some embodiments, when constructing the knowledge point association matrix, it is found that there is a class imbalance problem. If the prediction probability deviates from the true value gradient, it is beneficial to convergence. Based on this, the softmax function and the cross-entropy loss function are used to extend the multi-classification problem to the multi-label classification problem to solve the class imbalance problem. Specifically, 0 is set as the threshold, and the score of each target class needs to be greater than 0, and the score of each non-target class needs to be less than 0. That is, 0 minus the maximum value of each target class score and the maximum value of each non-target class score minus 0 needs to be greater than 0. Therefore, the calculation formula of the loss value loss based on multi-task learning is:

[0056] wherein, and denote the target class and non-target class sample sets, respectively, and z i and z j are the scores of the i-th predicted knowledge point text and the j-th predicted knowledge point text, respectively.

[0057] In some embodiments, in order to explore the labeling effect of the Hypergraph-based Knowledge-Enhanced Concept Tagging Method (HKECT) compared with other benchmark models, the following benchmark models are built and comparative experiments are conducted, which are specifically introduced as follows: (1) TextCNN: This model mainly uses multiple different convolution kernels to scan the context window to extract the local context features of the text, and completes the test question labeling task.

[0058] (2) LSTM-ATT: This model combines long short-term memory network and attention mechanism, captures long-term dependencies in test questions, and focuses on important parts of test text to complete knowledge point labeling.

[0059] (3) BERT: A model based on Transformer architecture proposed by Google, which learns context information through a bidirectional encoder to complete knowledge point labeling.

[0060] (5) SOMPT: A model that considers label prediction as a generated sequence, which completes the knowledge point labeling task based on UniLM.

[0061] (6) MTCQT: A knowledge point labeling method based on text comparison.

[0062] After feeding the text and answers of the test questions into the large model to generate analysis, the dataset formed by splicing the test question text and the analysis is denoted as TMK-PSS-LLM. Experiments of F1 score, precision, recall and Hamming loss are performed on the dataset of TMK-PSS-LLM, and the specific experimental results are shown in Table 1: Table 1 Comparison of annotation performance

[0063] From Table 1, it can be seen that the F1 score and HL of HKECT are significantly better than those of the other five models. A high F1 score value represents a good balance between recall and precision, indicating that HKECT performs better in overall performance, followed by SOMPT. SOMPT is a model based on UniLM, which is a modified model based on Bert. From the F1 and HL of SOMPT and HKECT, it is found that the pre-training model based on Bert is superior to other models in word embedding coding. In terms of accuracy, the performance of HKECT is much higher than that of other models, indicating that the confidence of the positive class in the prediction result is higher, followed by SOMPT. In terms of recall, the value of HKECT is only 0.2% lower than the highest SOMPT, but much higher than other models. In addition, the value of TextCNN is much lower than other models. Due to the limitation of local perception of traditional CNN, models that are good at capturing local context are difficult to capture the association of long test questions, resulting in the lowest performance of the model.

[0064] To explore whether the extraction of high-order information of knowledge points is effective for completing the annotation task, an ablation experiment of the knowledge point high-order information extraction module is designed to verify it. Among them, HKECT without using hypergraph convolutional neural network is denoted as KECT; HKECT that retains all modules but does not use text information when inputting nodes and only uses one-hot encoding as knowledge point representation is denoted as HKECT_O, and the annotation effect of knowledge point information ablation is shown in Table 2: Table 2 Annotation effect of knowledge point information ablation

[0065] From the data in Table 2, it can be found that HKECT improves the recall by 2.1% compared with KECT without adding hypergraph aggregation, because after adding the high-order information of knowledge points, the model has fewer positive examples of missed detection in the test; the precision is not obviously improved, only increased by 0.2%; the hamming loss is reduced by 0.1%, and the F1 score is increased by 1.7%, all indicators are slightly improved compared with the hypergraph convolutional neural network without using, which proves that the hypergraph has certain rationality in aggregating the relationship between knowledge points. In addition, when using the association matrix and one-hot vector as node information, HKECT-O has certain improvement in HL and F1 score compared with KECT, and the improvement in recall is also obvious, which indirectly proves that the hypergraph improves the model, because when using the semantic information of knowledge points, combining the text information of the test questions and the pool operation can effectively pay attention to the text information of the knowledge points. The indicators of HKECT are higher than those of other methods, which also proves that the method of using the hypergraph convolutional neural network for knowledge point labeling in the present application has rationality.

[0066] In order to explore the influence of Bert text embedding and test question text representation method on labeling effect in the comparative experiment, an ablation experiment is conducted on the text feature extraction of HKECT, wherein HKECT(-) represents that the text representation method of embedding is used for labeling knowledge points, and the experimental result graph of different text embedding methods is as shown in Figure 6 It can be found from Figure 6 that the F1 score, precision and recall of HKECT are all higher than those of Emb-BiLSTM. Since the dynamic encoding of Bert can capture the semantic changes of words in specific context, and can also capture the complex relationship of syntax, syntax and semantics, Bert can capture both local context and global context, while traditional embedding performs local co-occurrence statistics, lacking modeling of global semantics. In addition, Bert can parallelly compute global dependency relationship, and is more suitable for knowledge point association of long text, especially in the analysis of the characteristics of the supplementary test questions, which shows that the encoding of Bert can better help the model to complete the knowledge point labeling.

[0067] In order to explore the influence of multi-source fusion of test question features on labeling effect, the language large model is taken as a control variable to analyze the results of HKECT, TextCNN, LSTM+ATT and Bert. First, the influence of HKECT on labeling in multi-source fusion representation is discussed, wherein HKECT(-) represents that the features generated by the language large model are not used, and the influence result graph of multi-source fusion representation on labeling is as shown in Figure 7 .

[0068] From Figure 7As can be seen, the four indicators are all better than the analysis generated without using the language large model, among which the F1Score is 1.7% higher, the Precision is 1.4% higher, the Recall is 2.2% higher, and the Hamming loss is 0.04% lower, and all the indicators have a small part of improvement. This proves that using the language large model to supplement the semantic information of the test questions and enrich the input features can indeed help the model prediction, and in addition, it also indirectly proves the reliability of using the language large model to generate test question analysis samples.

[0069] The experimental effect diagram of different models on the TMK-PSS-LLM data set is as shown in Figure 8 It can be found that when using Bert and LSTM combined with attention mechanism to analyze the test questions, the overall prediction result is obviously better than that without using test question analysis data. Using the language large model to generate analysis content can make the characteristics of the test questions more rich and more suitable for short and concise mathematical test questions. While Bert and LSTM have more obvious advantages in extracting long text features of test questions. On the contrary, after observing the TextCNN using test question analysis data, all indicators have decreased, and the model performance has not increased but decreased, indicating that TextCNN has insufficient performance in the context of long sentences due to its characteristics of locality, position insensitivity and fixed structure.

[0070] The application also provides a multi-source fusion knowledge point labeling device based on hypergraph knowledge enhancement, which is used to realize the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement. The super edge matrix acquisition module is configured to acquire a knowledge point text set, input the knowledge point text set into a large language model to obtain a knowledge element system matrix and a knowledge point correlation matrix, construct a hypergraph matrix according to the knowledge point text set, the knowledge element system matrix and the knowledge point correlation matrix, and input the hypergraph matrix into a hypergraph convolutional neural network to obtain a super edge matrix. The vocabulary feature matrix acquisition module is configured to acquire a test question text, input the test question text into a large language model to obtain a vocabulary sequence, input the vocabulary sequence into a bidirectional semantic encoder to obtain a vocabulary vector matrix, and input the vocabulary vector matrix into a bidirectional long short-term memory network to obtain a vocabulary feature matrix. The knowledge point prediction matrix acquisition module is configured to splice the super edge matrix and the vocabulary feature matrix to obtain a spliced feature matrix, and input the spliced feature matrix into a multi-layer perception neural network to obtain a knowledge point prediction matrix. The knowledge point text prediction module is configured to label knowledge points of the test question text according to the knowledge point prediction matrix to obtain all predicted knowledge point texts of the test question text.

[0071] The embodiment of the present application provides an electronic device, including a processor and a memory; the memory has a computer program stored therein, wherein the computer program implements the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement of any one of the above solutions when executed by the processor.

[0072] Specifically, the processor may, for example, include a general-purpose microprocessor, an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), and the like. The processor can also include on-board memory for cache use. The processor can be a single processing unit for performing different actions of the method process according to the embodiment of the present application or a plurality of processing units.

[0073] The memory may, for example, be any medium capable of containing, storing, communicating, propagating or transmitting instructions. For example, the memory may, for example, include but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of the memory include: magnetic storage devices such as magnetic tapes or hard disk drives (HDD); optical storage devices such as compact discs (CD-ROM); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0074] The present application also provides a computer readable medium having a computer program stored thereon, the program being executed by a processor to implement the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement of any one of the above solutions. The computer readable medium can be included in the device / apparatus / system described in the above embodiments; or can exist separately and not be assembled into the device / apparatus / system. The above computer readable medium carries one or more programs, and when the one or more programs are executed, the method as in the embodiment of the present application is implemented.

[0075] According to the embodiments of the present application, the computer readable medium can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer readable program code is contained. Such propagated data signal can take many forms, including but not limited to, electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to, wireless, wired, optical fiber cable, radio frequency (RF) signals, and the like, or any suitable combination thereof.

[0076] Those skilled in the art will appreciate that features recited in the various embodiments and / or claims of the present application can be combined and / or interchanged, even if this is not explicitly stated in the present application. In particular, the features recited in the various embodiments and / or claims of the present application can be combined and / or interchanged, without departing from the spirit and teachings of the present application. All such combinations and / or interchanges are intended to fall within the scope of the present application. Accordingly, the scope of the present application should not be limited to the above-described embodiments, but should be defined in accordance with the following claims and their equivalents. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement, characterized in that: Including steps: Obtain a knowledge point text set, input the knowledge point text set into a large language model to obtain a knowledge meta-system matrix and a knowledge point association matrix, construct a hypergraph matrix based on the knowledge point text set, the knowledge meta-system matrix, and the knowledge point association matrix, and input the hypergraph matrix into a hypergraph convolutional neural network to obtain a hyperedge matrix; Obtain the test text, input the test text into the large language model to obtain a vocabulary sequence, input the vocabulary sequence into the bidirectional semantic encoder to obtain a vocabulary vector matrix, and input the vocabulary vector matrix into the bidirectional long short-term memory network to obtain a vocabulary feature matrix; The hyperedge matrix and the vocabulary feature matrix are concatenated to obtain a concatenated feature matrix, and the concatenated feature matrix is ​​input into a multi-layer perceptron neural network to obtain a knowledge point prediction matrix; The test text is annotated with knowledge points according to the knowledge point prediction matrix to obtain all the predicted knowledge point texts of the test text.

2. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 1 is characterized in that: Input the knowledge point text set into the large language model to obtain the knowledge meta-system matrix and the knowledge point association matrix, specifically including: Input each knowledge point text in the knowledge point text set into the large language model, assign multiple knowledge point texts with similarity greater than a preset value to the same knowledge element, and construct a knowledge element system matrix based on the knowledge elements; The sample test texts in the sample test text library are obtained, the frequency of co-occurrence of multiple knowledge point texts in a sample test text is obtained through the large language model, and a knowledge point association matrix is ​​constructed based on the frequency of co-occurrence of multiple knowledge point texts.

3. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 1 is characterized in that: Construct a hypergraph matrix based on the knowledge point text set, knowledge meta-system matrix and knowledge point association matrix, specifically including: The knowledge point texts in the knowledge point text set are regarded as nodes in a hypergraph; the relationships between different knowledge point texts in the knowledge point text set are regarded as hyperedges in the hypergraph, and a hypergraph is constructed based on the nodes and hyperedges; According to the knowledge element system matrix and the knowledge point association matrix, high-order information is extracted from the hypergraph to obtain high-order information between different knowledge point texts, and the hypergraph matrix is ​​constructed by the high-order information between different knowledge point texts.

4. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 3 is characterized in that: The hypergraph matrix is ​​input into the hypergraph convolutional neural network to obtain the hyperedge matrix, which specifically includes: According to the hypergraph, the hyperedge quantity matrix, the hyperedge importance weight matrix and the hyperedge included node quantity matrix of the node are obtained, and the normalized correlation matrix is ​​obtained by multiplying the hyperedge quantity matrix, the hypergraph matrix, the hyperedge importance weight matrix and the hyperedge included node quantity matrix in sequence. The hypergraph convolutional neural network consists of multiple convolutional layers. The hypergraph matrix is ​​input into the first convolutional layer, and the output matrix of the first convolutional layer is calculated based on the normalized correlation matrix and the hypergraph matrix. ; from Start by The output matrix of the convolutional layer Enter Convolutional layers, based on the normalized correlation matrix and output matrix Calculate the first The output matrix of the convolutional layer , the output matrix of the last convolutional layer is used as the hyperedge matrix.

5. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 1 is characterized in that: Input the test text into the large language model to obtain a vocabulary sequence, and then input the vocabulary sequence into the bidirectional semantic encoder to obtain a vocabulary vector matrix. Specifically, it includes: S11: Use the large language model to segment the test text into multiple words, obtain the time step of each word, and construct a word sequence based on the words and time steps; S12: Extract the t-th word in the word sequence, and process the t-th word through word embedding, sentence embedding, and position embedding to obtain a word embedding vector, a sentence embedding vector, and a position embedding vector, where t represents the time step; S13: Add the word embedding vector, sentence embedding vector and position embedding vector to obtain the total embedding vector of the t-th word, input the total embedding vector into the bidirectional semantic encoder for multi-layer encoding, and obtain the vocabulary vector of the t-th word; S14: Repeat steps S12-S13 until the vocabulary vectors of all words are obtained, and the vocabulary vector matrix is ​​formed by the vocabulary vectors of all words.

6. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 1 is characterized in that: The vocabulary vector matrix is ​​input into the bidirectional long short-term memory network to obtain the vocabulary feature matrix, which includes: S21: Bidirectional long short-term memory network includes forward long short-term memory network and backward long short-term memory network; S22: Get the tth word vector in the word vector matrix and the t-1th word vector The forward time step state , where t represents the time step; The vocabulary vector and the forward time step state Input forward long short-term memory network, according to the vocabulary vector and the forward time step state Calculate the forward output features and forward candidate memory , according to the forward output feature and forward candidate memory Calculate the tth word vector The forward hidden state of ; S23: Get the t-1th vocabulary vector The backward time step state ; The vocabulary vector and the backward time step state Input the backward long short-term memory network, according to the vocabulary vector and the backward time step state Calculate the backward output features and backward candidate memory , according to the backward output feature and backward candidate memory Calculate the tth word vector The backward hidden state ; S24: forward hidden state and the backward hidden state Perform concatenation to obtain the tth vocabulary vector Lexical features of ; S25: Repeat steps S22-S24 until the vocabulary features of all vocabulary vectors are obtained, and the vocabulary feature matrix is ​​formed by the vocabulary features of all vocabulary vectors.

7. The multi-source fusion knowledge point annotation method based on hypergraph knowledge enhancement according to claim 1 is characterized in that: After obtaining all the predicted knowledge point texts of the test text, it also includes: Obtain the knowledge point text labels of the test text, calculate the score of each predicted knowledge point text based on the probability of each predicted knowledge point text in the knowledge point prediction matrix and the knowledge point text label, calculate the loss value based on the score of each predicted knowledge point text, and adjust the parameters of the hypergraph convolutional neural network and the bidirectional long short-term memory network based on the loss value.

8. A multi-source fusion knowledge point labeling device based on hypergraph knowledge enhancement, used to implement the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement according to any one of claims 1 to 7, characterized in that: The device comprises: A hyperedge matrix acquisition module is used to obtain a knowledge point text set, input the knowledge point text set into a large language model to obtain a knowledge meta-system matrix and a knowledge point association matrix, construct a hypergraph matrix based on the knowledge point text set, the knowledge meta-system matrix, and the knowledge point association matrix, and input the hypergraph matrix into a hypergraph convolutional neural network to obtain a hyperedge matrix; The vocabulary feature matrix acquisition module is used to obtain the test text, input the test text into the large language model to obtain a vocabulary sequence, input the vocabulary sequence into the bidirectional semantic encoder to obtain a vocabulary vector matrix, and input the vocabulary vector matrix into the bidirectional long short-term memory network to obtain a vocabulary feature matrix; A knowledge point prediction matrix acquisition module is used to concatenate the hyperedge matrix and the vocabulary feature matrix to obtain a concatenated feature matrix, and input the concatenated feature matrix into a multi-layer perceptron neural network to obtain a knowledge point prediction matrix; The knowledge point text prediction module is used to mark the test text with knowledge points according to the knowledge point prediction matrix and obtain all the predicted knowledge point texts of the test text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-source fusion knowledge point labeling method based on hypergraph knowledge enhancement as described in any one of claims 1 to 7 is implemented.

Images