Mathematics course knowledge graph construction method and system

Abstract

The present invention provides a method and system for constructing a knowledge graph for mathematics courses, and relates to the technical field of knowledge graphs. The present invention solves the technical problem that the current relationship extraction method cannot accurately mine the entity relationship of mathematics courses, and realizes a relationship classification method by integrating sparse biaffine attention and adaptive threshold loss, overcoming the difficulties of extracting complex dependencies and overlapping relationships. At the same time, a hyperbolic space word embedding sparse projection model is used to predict the upper and lower relationships. The hyperbolic space has the ability to express hierarchical structures, can better predict the upper and lower relationships, and reduce the computational cost.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and specifically to a method and system for constructing a knowledge graph for mathematics courses. Background Art

[0002] A knowledge graph is a structured semantic knowledge base that is used to describe concepts in the physical world and their relationships in symbolic form. Its basic components are "entity-relationship-entity" triples, as well as entities and their related attribute-value pairs. Entities are connected to each other through relationships, forming a network-like knowledge structure.

[0003] The construction of knowledge graphs for mathematics courses mainly involves the construction of the instance layer. The main processes in building the instance layer include entity recognition and relationship extraction.

[0004] However, there is currently no method for constructing a mathematics course knowledge graph for mathematics courses (such as discrete mathematics knowledge points). If a mathematics course knowledge graph is constructed using the existing mathematics course knowledge graph construction method, due to the inclusion relationship of mathematics course entities, that is, a hypernym contains multiple hyponyms; the transformation relationship, that is, a sentence usually contains multiple entity pairs, and an entity pair may be associated with multiple possible relationships, the current relationship extraction method cannot achieve accurate mining. Therefore, it is urgent to construct a knowledge graph of the inclusion relationship and transformation relationship between knowledge points for mathematics courses. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies in the prior art, the present invention provides a method and system for constructing a knowledge graph for mathematics courses, which solves the problem that current relationship extraction methods cannot accurately mine entity relationships in mathematics courses.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, the present invention provides a method for constructing a mathematics course knowledge graph, comprising:

[0010] S1. Obtain mathematical knowledge point data and perform preprocessing to obtain training data and a database to be automatically extracted, and construct a training corpus for entity recognition and acquisition of hyponymy based on the training data;

[0011] S2. Train the entity recognition model through the training corpus, and use the trained entity recognition model to identify entities in the database to be automatically extracted;

[0012] S3, training a hyperbolic space word embedding sparse projection model through a training corpus, and identifying the hyponymy relationship of entities through the trained hyperbolic space word embedding sparse projection model;

[0013] S4. According to the entity recognition results and the entity hyponym and hyponym relationships, the entity masking technology is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted; the transformation relationship between entities in the sentence is extracted through the BERT model based on sparse biaffine attention and the classification model based on adaptive threshold loss.

[0014] Preferably, the S1 specifically includes:

[0015] Scan paper textbooks into images and convert the content information in the images into text;

[0016] The acquired text is divided into two parts: after-class knowledge summary and main text. The after-class knowledge summary is used as training data to build a training corpus; the main text is used as a database to be automatically extracted;

[0017] The process of building a training corpus is as follows:

[0018] Use ChatGPT4 to identify phrases from the training data, remove low-frequency phrases, and remove formulas containing only letters and punctuation marks to form knowledge point entities, and use the BIESO format to annotate entities in the original sentences;

[0019] All entities are mined in units of sentences to obtain sentence patterns, and ChatGPT4 is used to find words that represent logical reasoning. According to word frequency, those without intersections are classified into syntactic patterns, and then the relationships are marked with rules to obtain the training corpus.

[0020] Preferably, the structure of the entity recognition model includes word-level representation, BERT and multi-task cascade;

[0021] Among them, word-level representation is used to add word-based representation to the original character-level sequence tagging task;

[0022] BERT is used to pre-train deep bidirectional representations by jointly conditioning on the left and right contexts in all layers;

[0023] Multi-task cascading means that after obtaining the results of entity recognition, we return to the BERT output layer to find the representation vector of each entity word, and then input the entity representation vector into the fully connected layer for classification to determine the entity type.

[0024] Preferably, the entity recognition model measures and optimizes performance through a weighted loss function;

[0025] If the model does not recognize an entity that should have been recognized, the weighted loss function is increased to increase the penalty on the model; if the model recognizes an entity that should not have been recognized, the corresponding weighted loss function is reduced.

[0026] Preferably, the step of training the hyperbolic space word embedding sparse projection model by training the corpus comprises:

[0027] Among them, the hyperbolic space word embedding sparse projection model includes a normalized embedding mapping unit, a projection feature generation unit and a classification unit;

[0028] 301. Using the hyponymous and hyponymous relationship data in the training corpus as positive sample data, and generating negative sample data based on the positive sample data;

[0029] S302, performing normalized embedding processing on both positive sample data and negative sample data through a normalized embedding mapping unit and mapping them to a hyperbolic space to obtain word vector sample data, wherein the word vector sample data includes positive word vector sample data with a set of hyponymy and hyponymy relationships and negative word vector sample data without a set of hyponymy and hyponymy relationships;

[0030] S303, clustering the word vector sample data to obtain the cluster center, and obtaining the weight of the hyponymy word in the hyponymy set and the weight of the non-hypernymy word in the non-hypernymy set through the cluster center and the hyponymy pair;

[0031] 304. Based on the weights of the hyponymy words in the hyponymy set and the positive word vector sample data, a sparse projection vector is calculated to obtain a sparse positive projection matrix; based on the weights of the non-hypernymy words in the non-hypernymy set and the negative word vector sample data, a sparse double projection vector is calculated to obtain a sparse negative projection matrix;

[0032] 305. Generate positive sample projection features and negative sample projection features in a projection feature generation unit according to the positive projection matrix, the negative projection matrix and the word vector sample data;

[0033] 306. Train the classification unit based on the positive sample projection features and the negative sample projection features.

[0034] Preferably, the identifying the hyponymy relationship of the entity of S2 by using the trained hyperbolic space word embedding sparse projection model includes:

[0035] The entity (x, y) is normalized and embedded by the normalized embedding mapping unit and then mapped to the hyperbolic space to obtain Will The projection features are input into the projection feature generation unit, the projection features are calculated, the projection features are input into the classification unit, and 0 or 1 is output, where 0 indicates no hierarchical relationship and 1 indicates a hierarchical relationship.

[0036] Preferably, the extraction of transformation relations between entities in a sentence by using a BERT model based on sparse biaffine attention and a classification model based on adaptive threshold loss includes:

[0037] The weight matrix is ​​solved through the sparse matrix; the formula is as follows:

[0038] W (0) =S

[0039] Among them, W (0) Represents the initial weight matrix, S represents the sparse matrix; use the sparse matrix to approximate the weight matrix inside the BERT model; given a labeled sentence s, obtain the context representation h of each word through the weight-sparse BERT i :

[0040] {h1,...h |s|}=BERT{x1,...x |s|}

[0041] Among them, x i represents the input embedding of each word;

[0042] Cross-sentence context tracking is used to expand the sentence to a fixed window size W. In order to better obtain the position information of the word, a dual affine attention mechanism is used, which includes:

[0043] Two-dimensional dimensionality reduction MLPs are used, one head MLP and one tail MLP:

[0044]

[0045] and It is the mapping representation of the head and tail entities;

[0046] Calculate the score for each word:

[0047]

[0048] where U1∈R |y|*d*d and U2∈R |y|*2d is the weight parameter, b is the bias, Indicates series connection;

[0049] The score g for each word i,j As the output of the classifier, the range is [0, 1], and the score needs to be thresholded to be converted into a relation label;

[0050] Use the threshold class TH to learn the dynamic threshold for multi-label classification as follows:

[0051] For each entity pair Divide the relation set R into two parts: the positive set P containing relation r exists in Between, and the negative set N = RP, an adaptive threshold loss is applied to learn the relationship classifier, a threshold class TH is introduced, and a standard classification cross entropy loss function based on adaptive threshold loss is adopted, as shown below:

[0052] L=L1+L2

[0053]

[0054] Among them, logit represents the value of the relationship type, P T Represents a set of entities that have relationships, logit r′ Indicates that in P T The value of the relationship type in the set that is also in the threshold class TH, logit r Indicates that in P T The value of the relationship type in the set, logit TH Indicates that in P T The values ​​of the relation type in the set that are also in the threshold class TH.

[0055] In a second aspect, the present invention provides a mathematics course knowledge graph construction system, comprising:

[0056] The data module is used to obtain and preprocess mathematical knowledge point data to obtain training data and a database to be automatically extracted, and to build a training corpus for entity recognition and acquisition of hyponymy based on the training data;

[0057] An entity recognition module is used to train an entity recognition model through a training corpus, and use the trained entity recognition model to identify entities in the database to be automatically extracted;

[0058] The relationship recognition module is used to train the hyperbolic space word embedding sparse projection model through the training corpus, and identify the upper and lower relationships of entities through the trained hyperbolic space word embedding sparse projection model;

[0059] The transformation relationship extraction module is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted based on the entity recognition results and the entity hyponym relationship, using entity masking technology; the transformation relationship between entities in the sentence is extracted through the BERT model based on sparse biaffine attention and the classification model based on adaptive threshold loss.

[0060] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program for constructing a knowledge graph for a mathematics course, wherein the computer program enables a computer to execute the method for constructing a knowledge graph for a mathematics course as described above.

[0061] According to a fourth aspect, an electronic device is provided, comprising:

[0062] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the mathematics course knowledge graph construction method as described above.

[0063] (III) Beneficial effects

[0064] The present invention provides a method and system for constructing a knowledge graph for mathematics courses. Compared with the prior art, it has the following beneficial effects:

[0065] The embodiment of the present invention first obtains mathematical knowledge point data and performs preprocessing to obtain training data and a database to be automatically extracted, and constructs a training corpus for entity recognition and upper and lower relationship acquisition based on the training data; then trains an entity recognition model through the training corpus, and uses the trained entity recognition model to recognize entities in the database to be automatically extracted; trains a hyperbolic space word embedding sparse projection model through the training corpus, and uses the trained hyperbolic space word embedding sparse projection model to identify the upper and lower relationship of the entity; finally, according to the entity recognition result and the upper and lower relationship of the entity, the entity masking technology is used to mark the beginning and end of each entity mentioned in the sentence in the database to be automatically extracted; and the conversion relationship between entities in the sentence is extracted by the BERT model based on sparse biaffine attention and the classification model of adaptive threshold loss. The present invention solves the technical problem that the current relationship extraction method cannot accurately mine the entity relationship of the mathematical program class, and realizes the relationship classification method by integrating sparse biaffine attention and adaptive threshold loss, overcoming the difficulty of complex dependency and overlapping relationship extraction. At the same time, the hyperbolic space word embedding sparse projection model is used to predict the hierarchical relationship. The hyperbolic space has the ability to express hierarchical structure, can better predict the hierarchical relationship, and reduce the computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A block diagram of a method for constructing a mathematics course knowledge graph according to an embodiment of the present invention;

[0068] Figure 2 This is the structural diagram of the entity recognition model;

[0069] Figure 3 Flowchart of the training process for the entity recognition model;

[0070] Figure 4 A schematic diagram of a method for calculating a weighted loss function for an entity recognition model;

[0071] Figure 5 This is a specific flow chart of step S4. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] The embodiments of the present application provide a method and system for constructing a knowledge graph for mathematics courses, thereby solving the technical problem that current relationship extraction methods cannot accurately mine entity relationships in mathematics courses. This method achieves a relationship classification method that combines sparse bi-affine attention with adaptive threshold loss, thereby overcoming the difficulties in extracting complex dependencies and overlapping relationships.

[0074] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0075] The process of constructing a knowledge graph for mathematics courses in the mathematics field includes named entity recognition and relationship extraction. Named entity recognition includes BERT, BiLSTM-CRF, etc. Existing upper-lower relationship prediction models include single projection model, segmented projection model, etc., and relationship classification models include BERT, etc. However, the traditional named entity recognition model, upper-lower relationship prediction model, and relationship classification model have the following problems in the construction of knowledge graphs for mathematics courses:

[0076] (1) Entities in mathematics courses are professional terms. Traditional entity recognition methods easily split entities, that is, incorrect entity boundary recognition; and the precision and recall rate cannot reach high values.

[0077] (2) The inclusion relationship in mathematics courses is usually a 1:N relationship, that is, a hypernym contains multiple hyponyms, and the number of hyponyms contained in each hypernym is generally quite different. Currently, most of them are obtained by word embedding projection model. The conventional matrix linear transformation method in traditional Euclidean space cannot well represent the situation where there are many and uneven hyponyms.

[0078] (3) In mathematics course materials, a sentence usually contains multiple entity pairs, and an entity pair may be associated with multiple possible relationships. Therefore, the extraction of transformation relationships between knowledge points needs to capture the dependency and overlapping relationships between multiple entities at the same time. Current relationship extraction methods cannot achieve both of these points at the same time.

[0079] To solve the above problems, an embodiment of the present invention proposes a method and system for constructing a knowledge graph for mathematics courses.

[0080] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0081] The present invention provides a method for constructing a mathematics course knowledge graph. Figure 1 As shown, the method includes:

[0082] S1. Obtain mathematical knowledge point data and perform preprocessing to obtain training data and a database to be automatically extracted, and construct a training corpus for entity recognition and acquisition of hyponymy based on the training data;

[0083] S2. Train the entity recognition model through the training corpus, and use the trained entity recognition model to identify entities in the database to be automatically extracted;

[0084] S3, training a hyperbolic space word embedding sparse projection model through a training corpus, and identifying the hyponymy relationship of entities through the trained hyperbolic space word embedding sparse projection model;

[0085] S4. According to the entity recognition results and the entity hyponym and hyponym relationships, the entity masking technology is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted; the transformation relationship between entities in the sentence is extracted through the BERT model based on sparse biaffine attention and the classification model based on adaptive threshold loss.

[0086] The embodiment of the present invention solves the technical problem that the current relationship extraction method cannot accurately mine the entity relationship of mathematical programs, and realizes the relationship classification method by integrating sparse biaffine attention and adaptive threshold loss, overcoming the difficulties of complex dependency and overlapping relationship extraction. At the same time, the hyperbolic space word embedding sparse projection model is used to predict the upper and lower relationship. The hyperbolic space has the ability to express hierarchical structure, can better predict the upper and lower relationship, and reduce the computational cost.

[0087] The following is a detailed description of each step:

[0088] In step S1, mathematical knowledge point data is obtained and preprocessed to obtain training data and a database to be automatically extracted, and a training corpus for entity recognition and acquisition of hyponymy and hyponymy is constructed based on the training data. The specific implementation process is as follows:

[0089] In the embodiment of the present invention, a detailed description is given by taking the construction of a knowledge graph of discrete mathematics knowledge points as an example.

[0090] The purpose of data preprocessing is to convert paper textbooks into machine-readable data and build a training corpus for training models. The main methods are as follows:

[0091] First, extract text in the form of words and formulas from paper textbooks. Scan the paper textbooks into images, and use the Yolov8 model to identify content information such as words, formulas, pictures, and tables. For the text part, use the PDF parsing tool to extract the words to form text; for the formula, use the Image2LaTeX tool to convert the picture into latex format text. The obtained text is divided into two parts: after-class knowledge summary and main text. The after-class knowledge summary is used as training data to build a training corpus; the main text is used as a database to be automatically extracted for automatic extraction of knowledge point entities and their relationships.

[0092] Then, a training corpus for entity recognition and acquisition of hyponymy and hyponymy is constructed. ChatGPT4 is used to identify phrases from the after-class knowledge summary text, remove low-frequency phrases, and remove formulas containing only letters and punctuation marks to form knowledge point entities. The entities are annotated in the original sentences using the BIESO format, where B-tages indicates the beginning of the entity word, I-tages indicates the middle of the entity word, E-tages indicates the end of the entity word, S-tages indicates the entity of a single word, and O indicates a word without practical meaning. Finally, all entities are mined in sentences to obtain sentence patterns, and ChatGPT4 is used to find words that represent logical reasoning. According to word frequency, those without intersections are classified as syntactic patterns, and then the relationships are annotated using rules.

[0093] In step S2, the entity recognition model is trained by training the corpus, and the trained entity recognition model is used to identify entities in the database to be automatically extracted. The specific implementation process is as follows:

[0094] The structure of the entity recognition model of the embodiment of the present invention is as follows Figure 2 As shown in the figure, it includes multi-task cascade (Cascade), BERT, word-level representation (WLF, Word Level Feature), and when training the entity recognition model, the weighted loss function (WOL, Weight of Loss) is used to measure whether the model training is successful. Among them, BERT is a language representation model that aims to pre-train deep bidirectional representation by jointly adjusting the left and right contexts in all layers; multi-task cascade (Cascade) is to return to the BERT output layer after obtaining the result of entity recognition, find the representation vector of each entity word, and then input the entity representation vector into the fully connected layer for classification to determine the entity type; word-level representation is to add word-based representation to the original character-level sequence labeling task; weighted loss function is to balance precision and recall by setting weights in the loss function to achieve the purpose of improving F1.

[0095] The entity recognition model process is as follows Figure 3 As shown in the figure, after obtaining the result of entity recognition, the multi-task cascade returns to the BERT output layer to find the representation vector of each entity word (the representation vector is obtained by averaging the vectors of each entity word), and then inputs the entity representation vector into a layer of full connection for classification to determine the entity type. That is, during training, each word, whether it is an entity word or not, goes through the full connection, performs entity type classification and calculates the loss, and then masks the loss corresponding to the non-entity word; during prediction, the classification result corresponding to the last word of the entity is taken as the entity type.

[0096] The word-level representation represents characters and words separately through embedding matrices, and concatenates them according to the corresponding relationship.

[0097] like Figure 4 As shown in the figure, the weighted loss function is performed through the mask. If the model does not recognize an entity that should have been recognized, the corresponding loss is increased to increase the penalty for the model; if the model recognizes an entity that should not have been recognized, the corresponding loss is reduced.

[0098] Identify BIE and S structured text as required knowledge point entities

[0099] The trained entity recognition model is used to identify the knowledge point entities in the database to be automatically extracted.

[0100] In step S3, the hyperbolic space word embedding sparse projection model is trained by the training corpus, and the entities of S2 are identified by the trained hyperbolic space word embedding sparse projection model. The specific implementation process is as follows:

[0101] The hyperbolic space word embedding sparse projection model includes a normalized embedding mapping unit, a projection feature generation unit and a classification unit.

[0102] The training process of the hyperbolic space word embedding sparse projection model is as follows:

[0103] 301. Take the hyponymy data in the training corpus as positive sample data, and generate negative sample data based on the positive sample data. The specific implementation process is as follows:

[0104] The sample data is divided into a positive sample test set and a positive sample training set in a ratio of 2:8. A random sampling method is used to randomly select a word from its non-hypernym as its hypernym for a hyponym to generate a negative sample, which is divided into a negative sample test set and a negative sample training set in a ratio of 2:8.

[0105] S302, normalizing and embedding the positive sample data and the negative sample data and mapping them to the hyperbolic space through the normalized embedding mapping unit to obtain word vector sample data, wherein the word vector sample data includes positive word vector sample data with a set of upper and lower relationships and negative word vector sample data without a set of upper and lower relationships. The specific implementation process is as follows:

[0106] After obtaining the knowledge point entity, it is necessary to obtain the inclusion relationship and transformation relationship between the knowledge points. The inclusion relationship is obtained by using the hyperbolic space word embedding sparse projection model.

[0107] The Word2Vec word embedding model is used to normalize and embed the entities obtained in step 2. In this embodiment of the present invention, 300 dimensions are selected as the feature vector dimension of the word, and then the obtained embedding is projected into the hyperbolic space. Represents entity x i Normalized embedding via Word2Vec, express By mapping to the embedding representation of the hyperbolic space. The mapping formula is as follows:

[0108]

[0109] and Represents entity x i The hypernyms and non-hypernyms are normalized and embedded by Word2Vec and then mapped to the embedding representation of the hyperbolic space to establish a set of hypernyms and hyponyms. and no hierarchical relationship set After normalized embedding and mapping to hyperbolic space, the set of upper and lower relations No hierarchical relationship set

[0110] S303, cluster the word vector sample data to obtain the cluster center, and obtain the weight of the hyponymy word in the hyponymy set and the weight of the non-hypernymy word in the non-hypernymy set through the cluster center and the hyponymy pair. The specific implementation process is as follows:

[0111] Let K be the number of clusters, where each cluster corresponds to one epistasis component. Apply K-means to D (+) Using Vector Offset As a feature. express and The inner product between express and The Euclidean distance between them. The cluster center is recorded as Will Defined as In the dataset D (+) Weight on:

[0112]

[0113] 304. Based on the weights of the hyponymy words in the hyponymy set and the sample data of the positive word vector, a sparse projection vector is calculated to obtain a sparse positive projection matrix; based on the weights of the non-hypernymy words in the non-hypernymy set and the sample data of the negative word vector, a sparse double projection vector is calculated to obtain a sparse negative projection matrix. The specific implementation process is as follows:

[0114] Minimize the weighted projection error on the jth cluster, is a d*d projection matrix. Class j:

[0115]

[0116] When calculating, we use the random sparse method to Fill in the sparse matrix and multiply the matrices to obtain a sparse projection matrix let is the set of all K projection parameters. Learning the hyper-valued projection is equivalent to minimizing the following objective function:

[0117]

[0118] Learning non-hypernymy projections In the semantic relation classification task, the non-hypernyms of a term can be co-hyponyms, synonyms, etc. It is also applicable to semantic modeling of different non-hypernymy relations. The definition of this objective is as follows:

[0119] With The strings are used as non-hyperstatic cluster centers, and is the projection parameter. The cluster center is K-means in D (-) Generate, use As a feature. Defined as In the dataset D (-) The weight on .

[0120]

[0121] 305. In the projection feature generation unit, positive sample projection features and negative sample projection features are generated according to the positive projection matrix, the negative projection matrix and the word vector sample data. The specific implementation process is as follows:

[0122] Given a pair (x i ,y i )∈D (+) ∪D (-) Generate two sets of features:

[0123]

[0124] in is the vector concatenation operator. represents a positive feature, Indicates negative features. is the projection feature.

[0125] If a group of words has a hyponym relationship Very small, Very big; the opposite is true if there is no superior-subordinate relationship.

[0126] In an embodiment of the present invention, positive sample projection features are generated based on positive word vector sample data and positive double projection matrix and negative double projection matrix in word vector sample data. Negative sample projection features are generated based on negative word vector sample data and positive double projection matrix and negative double projection matrix.

[0127] 306. Train the classification unit based on the positive sample projection features and the negative sample projection features. The specific implementation process is as follows:

[0128] The classification unit package includes MLP (Multilayer Perceptron), which is a basic feedforward neural network consisting of an input layer, multiple hidden layers, and an output layer. Each layer consists of multiple neurons (or nodes), and the neurons between adjacent layers are all connected and have weights.

[0129] joint As the projection feature, the classification unit is trained, in D (+) and D (-) Calculation Input into the classification unit for training; select linear activation function, set parameter range for parameters such as learning rate, use grid search to select the optimal parameters, and use Adam algorithm to update weights.

[0130] After the hyperbolic space word embedding sparse projection model is trained, the entity of S2 is identified by the hyponym relationship, including:

[0131] The entity (x, y) is normalized and embedded by the normalized embedding mapping unit (corresponding to the operation of step S302) and then mapped to the hyperbolic space to obtain Will Input to the projection feature generation unit to calculate the projection feature (corresponding to the operation of step S305), the projection feature is input into the classification unit, and 0 or 1 is output, where 0 indicates no hierarchical relationship and 1 indicates a hierarchical relationship. A hierarchical relationship is an inclusion relationship.

[0132] In step S4, based on the entity recognition results and entity hyponymy, the entity masking technology is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted; the transformation relationship between entities in the sentence is extracted through the BERT model based on sparse biaffine attention and the classification model based on adaptive threshold loss. The specific implementation process is as follows:

[0133] like Figure 5 As shown, the embodiment of the present invention uses the BERT pre-trained model to classify the transformation relationship, and uses the entity masking technology ENTITY MARKERS as the input, and uses four tags [E1], [ / E1end], [E2] and [ / E2] to mark the start and end of each entity mentioned in the sentence. By processing the positions of the head entity and the tail entity, the corresponding entity vector representation is obtained. Then, the classification model of sparse biaffine attention and adaptive threshold loss is used to extract the transformation relationship between entities in the sentence. The specific implementation process is as follows:

[0134] The weight matrix is ​​approximated by using a sparse matrix. The formula is as follows:

[0135] W (0) =S

[0136] Among them, W (0)Denotes the initial weight matrix, and S denotes the sparse matrix. In this way, the sparse matrix is ​​used to approximate the weight matrix inside the BERT model. Given a labeled sentence s, the context representation h of each word is obtained through the weight-sparse BERT. i :

[0137] {h1,...h |s|}=BERT{x1,...x |s|}

[0138] Among them, x i represents the input embedding of each word.

[0139] In order to capture long-range dependencies, the embodiment of the present invention also uses cross-sentence context tracking to expand the sentence to a fixed window size W. In order to better obtain the position information of the word, a dual affine attention mechanism is used. Specifically, it includes:

[0140] Two-dimensional dimensionality reduction MLPs are used, one head MLP and one tail MLP:

[0141]

[0142] and It is the mapping representation of the head and tail entities.

[0143] Then calculate the score of each word:

[0144]

[0145]

[0146] where U1∈R |y|*d*d and U2∈R |y|*2d is the weight parameter, b is the bias, Indicates series connection.

[0147] The score g for each word i,jAs the output of the classifier, in the range [0, 1], this score needs to be thresholded to be converted into a relation label. Thresholding refers to the process of converting data or model output into a binary (or multivariate) form according to some threshold. Since thresholding has neither a closed form solution nor is it differentiable, and the model may have different confidences for different entity pairs or classes, a global threshold is not enough. A global threshold refers to a single threshold that is universal across the entire dataset or model output and applied to all samples or predictions. The number of relations varies (multi-label problem) and the model may not be globally calibrated, so the same probability does not mean the same for all entity pairs. This problem motivates us to replace the global threshold with a learnable adaptive threshold, which can reduce decision errors during inference. An adaptive threshold is a threshold that is dynamically adjusted based on specific rules, other indicators of the model output, or the characteristics of the data itself.

[0148] The embodiment of the present invention uses the threshold class TH to learn the dynamic threshold of multi-label classification, as follows:

[0149] For each entity pair Divide the relation set R into two parts: the positive set P containing relation r exists in Here, an adaptive threshold loss is applied to learn the relation classifier, a threshold class TH is introduced, and a standard classification cross entropy loss function based on adaptive threshold loss is adopted, as shown below:

[0150] L=L1+L2

[0151]

[0152] Among them, logit represents the value of the relationship type, P T A set of entities that represent relationships. r′ Indicates that in P T The value of the relationship type in the set that is also in the threshold class TH, logit r Indicates that in P T The value of the relationship type in the set, logit TH Indicates that in P T The values ​​of the relation type in the set that are also in the threshold class TH.

[0153] The transformation relationship between entities in all sentences in the data is extracted through the classification model.

[0154] Through steps S2-S4, we obtain entities, hierarchical relationships, and relationship classification to obtain a large number of triples between entities and build a knowledge graph.

[0155] The embodiment of the present invention also provides a mathematics course knowledge graph construction system, the system comprising:

[0156] The data module is used to obtain and preprocess mathematical knowledge point data to obtain training data and a database to be automatically extracted, and to build a training corpus for entity recognition and acquisition of hyponymy based on the training data;

[0157] An entity recognition module is used to train an entity recognition model through a training corpus, and use the trained entity recognition model to identify entities in the database to be automatically extracted;

[0158] The relationship recognition module is used to train the hyperbolic space word embedding sparse projection model through the training corpus, and identify the upper and lower relationships of the entities obtained by the entity recognition module through the trained hyperbolic space word embedding sparse projection model;

[0159] The transformation relationship extraction module is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted based on the entity recognition results and the entity hyponym relationship, using entity masking technology; the transformation relationship between entities in the sentence is extracted through the BERT model based on sparse biaffine attention and the classification model based on adaptive threshold loss.

[0160] It can be understood that the mathematics course knowledge graph construction system provided by the embodiment of the present invention corresponds to the above-mentioned mathematics course knowledge graph construction method. The explanations, examples, beneficial effects and other parts of the relevant contents can refer to the corresponding contents in the mathematics course knowledge graph construction method, and will not be repeated here.

[0161] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for constructing a knowledge graph for mathematics courses, wherein the computer program enables a computer to execute the method for constructing a knowledge graph for mathematics courses as described above.

[0162] An embodiment of the present invention further provides an electronic device, including:

[0163] one or more processors;

[0164] Memory; and

[0165] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include programs for executing the method for constructing a knowledge graph for a mathematics course as described above.

[0166] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0167] 1. The embodiment of the present invention solves the technical problem that the current relationship extraction method cannot accurately mine the entity relationship of mathematical programs, and realizes a relationship classification method by integrating sparse bi-affine attention and adaptive threshold loss, thereby overcoming the difficulties in extracting complex dependencies and overlapping relationships.

[0168] 2. The embodiment of the present invention uses a hyperbolic space word embedding sparse projection model to predict the hypernymy relationship. The hyperbolic space has the ability to express hierarchical structures, can better predict the hypernymy relationship, and reduce the computational cost.

[0169] 3. The embodiment of the present invention adopts an entity recognition model that combines multi-task cascade, BERT, word-level representation, and weighted loss function, splitting traditional named entity recognition into two tasks, one task for simply extracting entities and the other task for determining entity types, and incorporating word-level representation to improve the accuracy of named entity recognition.

[0170] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0171] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a knowledge graph for mathematics courses, characterized in that: include: S1. Obtain mathematical knowledge point data and preprocess it to obtain training data and a database to be automatically extracted. Based on the training data, a training corpus for entity recognition and hyponym acquisition is constructed; S2. Train the entity recognition model using the training corpus, and use the trained entity recognition model to identify entities in the database to be automatically extracted; S3. Train a hyperbolic space word embedding sparse projection model using the training corpus, and use the trained hyperbolic space word embedding sparse projection model to identify the hyponym relationship of entities; S4. Based on the entity recognition results and the entity hyponym relationship, use entity masking technology to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted; Extract the transformation relationship between entities in sentences through the BERT model based on sparse biaffine attention and the classification model with adaptive threshold loss; Among them, the mathematics knowledge point data refers to the knowledge points in the paper version of mathematics textbooks; The method of obtaining mathematical knowledge point data and preprocessing it to obtain training data and a database to be automatically extracted includes: Scan paper mathematics textbooks into images and convert the content information in the images into text; divide the obtained text into two parts: after-class knowledge summary and main text. The after-class knowledge summary is used as training data to build a training corpus; the main text is used as the database to be automatically extracted; The method of training a hyperbolic space word embedding sparse projection model using a training corpus includes: The hyperbolic space word embedding sparse projection model includes a normalized embedding mapping unit, a projection feature generation unit and a classification unit; 301. Using the hyponym-hypernym relationship data in the training corpus as positive sample data, and generating negative sample data based on the positive sample data; S302, performing normalized embedding processing on both the positive sample data and the negative sample data by a normalized embedding mapping unit and mapping the resultant data into a hyperbolic space to obtain word vector sample data, wherein the word vector sample data includes positive word vector sample data with a hyponymy set and negative word vector sample data without a hyponymy set; S303, clustering the word vector sample data to obtain cluster centers, and obtaining the weights of the hyponymy words in the hyponymy set and the weights of the non-hypernymy words in the non-hypernymy set based on the cluster centers and the hyponymy pairs; 304. Based on the weights of the hyponymy words in the hyponymy set and the sample data of the positive word vectors, a sparse projection vector is calculated to obtain a sparse positive projection matrix; based on the weights of the non-hypernymy words in the non-hypernymy set and the sample data of the negative word vectors, a sparse double projection vector is calculated to obtain a sparse negative projection matrix; 305. Generate positive sample projection features and negative sample projection features in the projection feature generation unit according to the positive projection matrix, the negative projection matrix, and the word vector sample data; 306. Training a classification unit based on the positive sample projection features and the negative sample projection features; The BERT model based on sparse bi-affine attention uses entity masking technology ENTITY MARKERS as input, using four tags [E1], [ / E1end], [E2], and [ / E2] to mark the start and end of each entity mentioned in the sentence; by processing the positions of the head and tail entities, the corresponding entity vector representations are obtained; and a classification model with sparse bi-affine attention and adaptive threshold loss is used to extract the transformation relationship between entities in the sentence. The method of extracting the transformation relationship between entities in a sentence by using a BERT model based on sparse biaffine attention and a classification model based on adaptive threshold loss includes: The weight matrix is ​​solved through the sparse matrix; the formula is as follows: W (0) =S Among them, W (0) Represents the initial weight matrix, S represents the sparse matrix; use the sparse matrix to approximate the weight matrix inside the BERT model; given a labeled sentence s, obtain the context representation h of each word through the weight-sparse BERT i , i=1,2,3,...,|s|: {h1,...,h |s| }=BERT{x1,...,x |s| } Among them, x i represents the input embedding of each word, i=1,2,3,...,|s|; Cross-sentence context tracking is used to expand the sentence to a fixed window size W. To better obtain word position information, a dual affine attention mechanism is used, which includes: Two-dimensional dimensionality reduction MLPs are used, one head MLP and one tail MLP: and It is the mapping representation of the head and tail entities; Calculate the score for each word: where U1∈R |y|*d*d and U2∈R |y|*2d is the weight parameter, b is the bias, Indicates concatenation; The score g for each word i,j As the output of the classifier, the range is [0, 1]. The score needs to be thresholded to convert it into a relation label; Use the threshold class TH to learn the dynamic threshold for multi-label classification as follows: For each entity pair Divide the relation set R into two parts: the positive set P containing relation r exists in Between, and the negative set N = RP; apply adaptive threshold loss to learn the relationship classifier, use the threshold class TH, and adopt the standard classification cross entropy loss function based on adaptive threshold loss, as shown below: L=L1+L2 Among them, logit represents the value of the relationship type, P T Represents a set of entities that have relationships, logit r′ Indicates that in P T The value of the relationship type in the set that is also in the threshold class TH, logit r Indicates that in P T The value of the relationship type in the set, logit TH Indicates that in P T The values ​​of the relation type in the set that are also in the threshold class TH.

2. The method for constructing a mathematics course knowledge graph according to claim 1, wherein: The training corpus for entity recognition and hyponym relationship acquisition based on training data includes: Use ChatGPT4 to identify phrases from the training data, remove low-frequency phrases, and remove formulas containing only letters and punctuation marks to form knowledge point entities. Use the BIESO format to annotate entities in the original sentences. All entities are mined in sentence units to obtain sentence patterns, and ChatGPT4 is used to find words that represent logical reasoning. According to word frequency, those without intersection are classified as syntactic patterns, and then the relationships are marked with rules to obtain the training corpus.

3. The method for constructing a mathematics course knowledge graph according to claim 1, wherein: The structure of the entity recognition model includes word-level representation, BERT and multi-task cascade; Among them, word-level representation is used to add word-based representation to the original character-level sequence tagging task; BERT is used to pre-train deep bidirectional representations by jointly conditioning on left and right contexts in all layers; Multi-task cascade means that after obtaining the results of entity recognition, we return to the BERT output layer, find the representation vector of each entity word, and then input the entity representation vector into the fully connected layer for classification to determine the entity type.

4. The method for constructing a mathematics course knowledge graph according to claim 3, wherein: The entity recognition model measures and optimizes performance through a weighted loss function; If the model does not recognize an entity that it should have recognized, the weighted loss function is increased to increase the penalty on the model; if the model recognizes an entity that it should not have recognized, the corresponding weighted loss function is reduced.

5. The method for constructing a mathematics course knowledge graph according to claim 1, wherein: The method of identifying the hyponymy relationship of entities by using the trained hyperbolic space word embedding sparse projection model includes: The entity (x, y) is normalized and embedded by the normalized embedding mapping unit and then mapped to the hyperbolic space to obtain Will The projection feature is input into the projection feature generation unit, the projection feature is calculated, the projection feature is input into the classification unit, and 0 or 1 is output, where 0 indicates no hierarchical relationship and 1 indicates a hierarchical relationship.

6. A mathematics course knowledge graph construction system, characterized by: include: The data module is used to obtain mathematical knowledge point data and perform preprocessing to obtain training data and a database to be automatically extracted. Based on the training data, a training corpus for entity recognition and hyponym acquisition is constructed; The entity recognition module is used to train the entity recognition model through the training corpus and use the trained entity recognition model to identify entities in the database to be automatically extracted; The relationship recognition module is used to train the hyperbolic space word embedding sparse projection model through the training corpus, and identify the hyponym relationship of entities through the trained hyperbolic space word embedding sparse projection model; The transformation relationship extraction module is used to mark the start and end of each entity mentioned in the sentence in the database to be automatically extracted based on the entity recognition results and the entity hyponym relationship, using entity masking technology; Extract the transformation relationship between entities in sentences through the BERT model based on sparse biaffine attention and the classification model with adaptive threshold loss; Among them, the mathematics knowledge point data refers to the knowledge points in the paper version of mathematics textbooks; The method of obtaining mathematical knowledge point data and preprocessing it to obtain training data and a database to be automatically extracted includes: Scan paper mathematics textbooks into images and convert the content information in the images into text; divide the obtained text into two parts: after-class knowledge summary and main text. The after-class knowledge summary is used as training data to build a training corpus; the main text is used as the database to be automatically extracted; The method of training a hyperbolic space word embedding sparse projection model using a training corpus includes: The hyperbolic space word embedding sparse projection model includes a normalized embedding mapping unit, a projection feature generation unit and a classification unit; 301. Using the hyponym-hypernym relationship data in the training corpus as positive sample data, and generating negative sample data based on the positive sample data; S302, performing normalized embedding processing on both the positive sample data and the negative sample data by a normalized embedding mapping unit and mapping the resultant data into a hyperbolic space to obtain word vector sample data, wherein the word vector sample data includes positive word vector sample data with a hyponymy set and negative word vector sample data without a hyponymy set; S303, clustering the word vector sample data to obtain cluster centers, and obtaining the weights of the hyponymy words in the hyponymy set and the weights of the non-hypernymy words in the non-hypernymy set based on the cluster centers and the hyponymy pairs; 304. Based on the weights of the hyponymy words in the hyponymy set and the sample data of the positive word vectors, a sparse projection vector is calculated to obtain a sparse positive projection matrix; based on the weights of the non-hypernymy words in the non-hypernymy set and the sample data of the negative word vectors, a sparse double projection vector is calculated to obtain a sparse negative projection matrix; 305. Generate positive sample projection features and negative sample projection features in the projection feature generation unit according to the positive projection matrix, the negative projection matrix, and the word vector sample data; 306. Training a classification unit based on the positive sample projection features and the negative sample projection features; The BERT model based on sparse bi-affine attention uses entity masking technology ENTITY MARKERS as input, using four tags [E1], [ / E1end], [E2], and [ / E2] to mark the start and end of each entity mentioned in the sentence; by processing the positions of the head and tail entities, the corresponding entity vector representations are obtained; and a classification model with sparse bi-affine attention and adaptive threshold loss is used to extract the transformation relationship between entities in the sentence. The method of extracting the transformation relationship between entities in a sentence by using a BERT model based on sparse biaffine attention and a classification model based on adaptive threshold loss includes: The weight matrix is ​​solved through the sparse matrix; the formula is as follows: W (0) =S Among them, W (0) Represents the initial weight matrix, S represents the sparse matrix; use the sparse matrix to approximate the weight matrix inside the BERT model; given a labeled sentence s, obtain the context representation h of each word through the weight-sparse BERT i , i=1,2,3,...,|s|: {h1,...,h |s| }=BERT{x1,...,x |s| } Among them, x i represents the input embedding of each word, i=1,2,3,...,|s|; Cross-sentence context tracking is used to expand the sentence to a fixed window size W. To better obtain word position information, a dual affine attention mechanism is used, which includes: Two-dimensional dimensionality reduction MLPs are used, one head MLP and one tail MLP: and It is the mapping representation of the head and tail entities; Calculate the score for each word: where U1∈R |y|*d*d and U2∈R |y|*2d is the weight parameter, b is the bias, Indicates concatenation; The score g for each word i,j As the output of the classifier, the range is [0, 1]. The score needs to be thresholded to convert it into a relation label; Use the threshold class TH to learn the dynamic threshold for multi-label classification as follows: For each entity pair Divide the relation set R into two parts: the positive set P containing relation r exists in Between, and the negative set N = RP; apply adaptive threshold loss to learn the relationship classifier, use the threshold class TH, and adopt the standard classification cross entropy loss function based on adaptive threshold loss, as shown below: L=L1+L2 Among them, logit represents the value of the relationship type, P T Represents a set of entities that have relationships, logit r′ Indicates that in P T The value of the relationship type in the set that is also in the threshold class TH, logit r Indicates that in P T The value of the relationship type in the set, logit TH Indicates that in P T The values ​​of the relation type in the set that are also in the threshold class TH.

7. A computer-readable storage medium, characterized in that It stores a computer program for constructing a mathematics course knowledge graph, wherein the computer program enables a computer to execute the mathematics course knowledge graph construction method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the mathematics course knowledge graph construction method as described in any one of claims 1 to 5.

Images