A new engineering major Chinese knowledge concept extraction method based on a dictionary

Abstract

The application discloses a new engineering major Chinese knowledge concept extraction method based on a dictionary, and comprises the following steps: 1) obtaining a new engineering related subdivided major, and converting all course teaching materials and syllabuses into text data; 2) using relevant text data to obtain corresponding words through word segmentation processing, and training a word2vec word vector model and an original word vector on the basis; 3) obtaining a large number of relevant course major vocabulary sets through a crawler technology, selecting corresponding major keywords as seeds, inputting the seeds into the trained word2vec model, obtaining words with a similarity above a threshold, and jointly forming a new engineering knowledge concept dictionary with the segmented words; and 4) constructing an NECE model, recognizing knowledge concepts of original course materials, and storing a concept set. The application can use the word2vec model to construct corresponding course major vocabulary sets and word vectors, and use the NECE model to realize extraction of major course concepts, thereby laying a data foundation for construction of an education system knowledge graph.

Description

Technical Field

[0001] This invention belongs to the field of knowledge concept extraction, specifically involving a dictionary-based method for extracting Chinese knowledge concepts for new engineering majors. Background Technology

[0002] In recent years, MOOCs (Massive Open Online Courses) and online education have been hot topics in the field of smart education. Smart education platforms, with the influx of users, have accumulated massive amounts of teaching behavior data and knowledge resources. The analysis and mining of these two important types of educational data have injected new momentum into the development of smart education. Currently, most online education platforms are organized around courses, categorized by subject area, university, and year. However, course concepts are implicit within the courses themselves, requiring learners to organize, classify, and process them during the learning process. How to automatically extract course concepts from a course using big data methods is one of the hot topics and challenges in current smart education research.

[0003] Although scholars both domestically and internationally have conducted relevant research on knowledge concept extraction, firstly, due to the inherent advantages of the English language, most current research is based on the extraction of English knowledge concepts, while the extraction of Chinese knowledge concepts requires further study. Secondly, most extraction methods are based on character-level knowledge concept extraction, which can only use relevant information from characters and cannot fully utilize word-level information in the original text. This results in many technical terms not being fully extracted during concept extraction. Therefore, this invention proposes a method for extracting knowledge concepts in emerging engineering disciplines. By incorporating dictionary information relevant to emerging engineering disciplines, the accuracy of concept extraction is improved. Furthermore, by adding new word information and combining it with an embedding method, the speed of the concept extraction process is greatly improved. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to overcome the shortcomings of existing concept extraction methods and provide a dictionary-based method for extracting Chinese knowledge concepts for new engineering disciplines. This method uses the NECE (New Engineering Concept Extraction) model to extract concepts from the original text, while also utilizing word information within the sentences and incorporating a new vector fusion method, thereby improving both accuracy and speed.

[0005] Technical Solution: This invention proposes a dictionary-based method for extracting Chinese knowledge concepts for new engineering majors, comprising the following steps:

[0006] (1) Obtain relevant sub-specialty courses of new engineering disciplines and convert all course textbooks and teaching outlines into raw text data;

[0007] (2) Obtain the corresponding words by segmenting the text data, and train the word2vec word vector model and the original word vectors based on this.

[0008] (3) Using Python technology, obtain a set of professional vocabulary related to the relevant courses, select the keywords of the corresponding major as seeds, input them into the trained word2vec model, obtain the Top K similarity words, and form a new engineering knowledge concept dictionary together with the segmented words.

[0009] (4) Construct the New Engineering Knowledge Concept Recognition (NECE) model to identify knowledge concepts in the original course textbooks and store the concept set.

[0010] Further, step (1) includes the following steps:

[0011] (11) For professional courses in new engineering disciplines, including Python-based computer vision processing, natural language processing and deep learning, the corresponding textbooks and teaching outlines were collected as text data.

[0012] (12) Obtain the BMES sequence corresponding to the text data by manually annotating its knowledge concepts.

[0013] Furthermore, step (2) includes the following steps:

[0014] (21) The original text data obtained in step (1) is preprocessed, including traditional Chinese conversion, removal of XML symbols, removal of stop words and interference words, processing of word entries into single-line data, and then obtaining the word segmentation file corresponding to the original text data through jieba word segmentation or LTP word segmentation technology.

[0015] (22) Use the gensim package in Python to train the word2vec model using word segmentation file data, and train the vocabulary sets of different professional courses separately;

[0016] (23) Feature processing: convert text data into data that can be recognized by computers. Use word2vec's skip-gram model to convert words in the word segmentation file into word vectors of a specified dimension.

[0017] Furthermore, step (3) includes the following steps:

[0018] (31) Using Python-related technologies, obtain a dictionary set for the relevant field and combine them to form a candidate vocabulary set;

[0019] (32) Manually select important concepts related to the major as seed words, use the wod2vec model trained in step (2) to perform word vector similarity calculation; select the skip-gram model to perform word vector similarity calculation, calculate the similarity with different seed words respectively, take the Top K as the external vocabulary, and convert it into a specified dimension word vector suitable for the NECE model, and form a professional dictionary of new engineering related courses with the words in the word segmentation file.

[0020] Furthermore, step (4) includes the following steps:

[0021] (41) Construct a new engineering knowledge concept recognition (NECE) model and pre-train the NECE model using existing labeled text data; the NECE model consists of a character embedding layer, a dictionary matching layer, a sequence modeling layer, and a CRF layer from bottom to top.

[0022] (42) In the NECE model, the original text data is represented by character vector embedding by the bottom character embedding layer. For each character, the dictionary is matched to find the word in the dictionary. Then, the corresponding word information in the dictionary is concatenated with the character vector of the original text and input into the sequence modeling layer. This layer models the dependency relationship between characters. The general architecture of the sequence modeling layer includes bidirectional long short-term memory network and convolutional neural network. Above the sequence modeling layer, the label inference of the entire character sequence is performed at once based on the conditional random field layer, so as to label each character, complete the recognition of knowledge concepts, output the new engineering related course concepts, and store them.

[0023] Further, step (42) includes the following steps:

[0024] (421) For the NECE model, the original input text is considered as:

[0025] S = {c1, c2, c3, ..., c n}

[0026] Here, e represents each character in the input, and each input character can be represented using a dense matrix vector, where e c This represents a word table lookup operation that converts characters into vectors.

[0027]

[0028] (422) Add word information vectors to the character embedding vectors. For each Chinese character c in any input original text data i Using the professional dictionaries of new engineering courses, all matching words were found and divided into four sets: "B", "M", "E", and "S". The construction of the four sets is described as follows:

[0029]

[0030] Where L is the dictionary used here, w i,j Represents a subsequence {c} in S i ,c i+1 ,…,c j If a set is empty, meaning no matching word was found for that character, then the special character "empty" is added to that empty word set.

[0031] (423) Transform the four sets into vectors of a specified dimension, and use the frequency of each word as the weight in the vector compression representation. Specifically, assuming z(w) represents the frequency of the selected word in a fixed set of words, the weighted word set S is represented as:

[0032]

[0033] in:

[0034]

[0035] Here, w represents the selected knowledge concept vocabulary. All words in the four word sets "B", "M", "E" and "S" are weighted and normalized. Furthermore, if word w is found to be contained in a subsequence of another matching word during the statistical analysis, the word frequency of w will not increase.

[0036] (424) Combine the four word sets into a fixed-dimensional feature and add it to the representation of each character. Concatenate the representations of the four word sets. The final representation of each character is:

[0037] e s (B,M,E,S)=[v s (B); v s (M); v s (E); v s (S)](7)

[0038] x c ←[x c ;e s (B,M,E,S)]((8)

[0039] Among them, v s This represents the weighting function mentioned above;

[0040] (425) After adding the dictionary information related to new engineering disciplines, the generated vector x c The input is fed into the sequence modeling layer, which is implemented using a single-layer BiLSTM. The forward LSTM network is defined as follows:

[0041]

[0042] Where σ is the sigmoid function of the corresponding matrix element-wise multiplication, and ⊙ represents the product of the elements at the corresponding positions of the matrix; W and b represent the trainable parameters in the model; backward LSTM has the same definition as forward LSTM, but models sequences in reverse order; the hidden state of the i-th step connection of forward LSTMs and backward LSTMs. This constitutes the contextual information-related representation of ci;

[0043] (426) At the top of the sequence modeling layer, the sequential conditional random field layer is applied to perform label inference on the entire character sequence at once, as defined below:

[0044]

[0045] Among them, y s Let represent all possible label sequences in s, and K be a fixed hyperparameter in the model; and

[0046]

[0047] Among them, w y',y and b y',y These are all trainable parameters for the label pair (y', y); for label prediction, given the input sequence s, we search for the label sequence y* with the highest conditional probability, which is solved using the Viterbi algorithm, specifically defined as follows:

[0048] y * =arg y maxp(y|s;θ) (12).

[0049] Beneficial Effects: Compared with the prior art, the beneficial effects of this invention are as follows: 1. By adding a relevant dictionary of new engineering professional terms, this invention solves the problem that traditional concept extraction models cannot fully utilize word information, thus improving the accuracy of professional term phrase extraction; 2. In the part where word vectors and character vectors are fused, a new fusion method is adopted. By statistically analyzing the frequency of occurrence of different professional terms, they are fused with different weights, thereby reducing information loss when using word vectors later; 3. The application of BiLSTM and CRF in the sequence modeling layer can learn the interrelationships between tags, thereby improving the extraction accuracy. Attached Figure Description

[0050] Figure 1 This is a flowchart of the present invention;

[0051] Figure 2 A schematic diagram of the NECE model structure for identifying new engineering knowledge concepts.

[0052] Figure 3This is a schematic diagram of the NECE model structure after adding word information in this invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings.

[0054] like Figure 1 As shown, this invention proposes a dictionary-based method for extracting Chinese knowledge concepts for new engineering majors, including the following steps:

[0055] Step 1: Obtain relevant sub-specialties in emerging engineering disciplines, including courses such as Python-based computer vision processing, natural language processing, and deep learning, and convert all course textbooks and syllabi into text data.

[0056] For emerging engineering courses, including Python-based computer vision processing, natural language processing, and deep learning, their corresponding textbooks and syllabi were collected as text data. By manually annotating the knowledge concepts, the corresponding BMES sequences were obtained from the text data, fulfilling the data type requirements for NECE (New Engineering Concept Extraction) model input and performance testing.

[0057] Step 2: Use relevant text data to obtain corresponding words through word segmentation, and train the word2vec word vector model and the original word vectors based on this.

[0058] The original course text data is preprocessed, including converting to traditional Chinese characters, removing XML symbols, removing stop words and interference words, and processing the word entries into single-line data. Then, the word segmentation file corresponding to the original text data is obtained by using jieba or LTP word segmentation technology.

[0059] We use the gensim package in Python to train a word2vec model using segmented word data. Word2vec is a static word vector construction method. The training principle of word2vec is based on word co-occurrence to train the semantic relationships between words. Vocabulary sets from different professional courses need to be trained separately.

[0060] Feature processing, also known as word vector encoding, converts text data into data that computers can recognize, facilitating computation. This is typically done by converting the data into numerical data. Using the skip-gram model of word2vec, words in the word segmentation file are transformed into word vectors of a specified dimension suitable for the NECE (New Engineering Concept Extraction) model.

[0061] Step 3: Using Python technology, obtain a large set of relevant course professional vocabulary. Select keywords of the corresponding major as seeds and input them into the trained word2vec model to obtain the Top K similarity words. Together with the segmented words, they form a new engineering knowledge concept dictionary.

[0062] Using Python-related technologies, we obtained relevant professional vocabulary sets from Baidu Encyclopedia and relevant domain dictionaries from Sogou Input Method, which together formed a candidate vocabulary set.

[0063] Important concepts from relevant fields are manually selected as seed words, and the wod2vec model trained in step 2.2) is used to calculate word vector similarity. wod2vec includes two models: CBOW and skip-gram.

[0064] The word similarity described by CBOW is measured by cosine similarity, as shown in formula (1):

[0065]

[0066] Where x and y represent the two words entered.

[0067] Skip-gram describes word similarity, meaning that two words frequently appear together. This can be measured using mutual information, as shown in formula (2):

[0068]

[0069] Similarly, x and y here also represent the two words that were entered.

[0070] Mutual information has the following mathematical properties:

[0071] • Nonnegativity: I(x,y)≥0

[0072] • Symmetry: I(c,y)=I(y,x)

[0073] • If I(x,y)=0, it means that variables x and y are completely uncorrelated, that is, the greater the mutual information, the more correlated they are.

[0074] Since the amount of course data in the New Engineering discipline is relatively small, the skip-gram model was chosen to calculate word vector similarity. The similarity with different seed words was calculated separately, and the Top K words were taken as the external vocabulary. The vocabulary was then transformed into word vectors of a specified dimension suitable for the NECE (New Engineering Concept Extraction) model. These vectors, together with the words in the word segmentation file, formed a professional dictionary for New Engineering discipline related courses.

[0075] Step 4: Construct a New Engineering Knowledge Concept Recognition (NECE) model, such as... Figure 2As shown, the original course materials are used to identify knowledge concepts and store the concept set.

[0076] The labeled text data was divided into training, validation, and test sets in a 6:2:2 ratio. The New Engineering Concept Extraction (NECE) model was then used for pre-training on the existing labeled text data. The NECE model, from bottom to top, consists of a character embedding layer, a dictionary matching layer, a sequence modeling layer, and a CRF layer.

[0077] In the NECE (New Engineering Concept Extraction) model, the underlying character embedding layer represents the original text data using character vector embeddings. Then, for each character, a match is performed in a dictionary to find its corresponding word. The word information from the dictionary is then concatenated with the original text's character vectors and input into the sequence modeling layer. This layer models the dependencies between characters, such as... Figure 3 As shown. The general architecture of this layer includes bidirectional long short-term memory networks, convolutional neural networks, etc. Here, a single-layer Bi-LSTM is used to implement dependency modeling. Above the sequence modeling layer, a Conditional Random Field (CRF) layer typically performs label inference on the entire character sequence at once, thereby labeling each character (BMES), thus completing the recognition of knowledge concepts, outputting new engineering-related course concepts, and storing them.

[0078] For the NECE (New Engineering Concept Extraction) model, the input raw text is considered as:

[0079] S = {c1, c2, c3, ..., c n},

[0080] Here, e represents each character in the input, and each input character can be represented using a dense matrix vector, where e c This represents a word table lookup operation that converts characters into vectors.

[0081]

[0082] After converting characters into corresponding character embedding vectors, word information vectors need to be added to the character embedding vectors to utilize the information between words. For any input original text data S, for each Chinese character c iUsing the professional dictionaries of new engineering courses, all matching words were found and divided into four sets: "B", "M", "E", and "S". The construction of the four sets is described as follows:

[0083]

[0084] Where L is the dictionary used here, w i,j Represents a subsequence {c} in S i ,c i+1 ,…,c j If a set is empty (i.e., no matching word is found for that character), a special character "empty" is added to that empty word set. This method allows for a precise joint representation of the matching results at character positions and word information, reducing information loss.

[0085] The resulting four sets are transformed into vectors of a specified dimension. Here, the frequency of each word is calculated as the weight in the vector compression representation. Specifically, assuming z(w) represents the frequency of the selected word in a fixed set of words, the weighted word set S can be represented as:

[0086]

[0087] in:

[0088]

[0089] Here, 'w' represents the selected knowledge concept set. The weights of all words in the four word sets "B", "M", "E", and "S" are normalized to facilitate subsequent overall comparisons. Furthermore, when calculating the frequency of word 'w', to prevent shorter words from consistently having a lower frequency than longer words containing them, if word 'w' is found to be contained in a subsequence of another matching word, the frequency of 'w' will not increase.

[0090] The final step in vector operations is to combine the four word sets into a fixed-dimensional feature and add it to the representation of each character. To retain as much information as possible, the representations of the four word sets are concatenated, and the final representation of each character is:

[0091] e s (B,M,E,S)=[v s (B); v s (M); v s (E); v s (S)](7)

[0092] x c ←[x c ;e s (B,M,E,S)](8)

[0093] Among them, v s This represents the weighting function mentioned above.

[0094] After incorporating relevant dictionary information from emerging engineering disciplines, the final generated vector x will be... c The input is fed into the sequence modeling layer, which models the dependencies between characters. Common architectures for this layer include Bidirectional Long Short-Term Memory (BiLSTM) networks, Convolutional Neural Networks (CNNs), etc. Here, a single-layer BiLSTM is used to implement this layer. The forward LSTM network is defined as follows:

[0095]

[0096] Where σ is the sigmoid function of the corresponding matrix element-wise multiplication, and ⊙ represents the product of the elements at the corresponding positions of the matrix. W and b represent the trainable parameters in the model. Backward LSTM has the same definition as forward LSTM, but models sequences in reverse order. The hidden states of the i-th connection in both forward and backward LSTMs are shown. This constitutes the context-related representation of ci.

[0097] At the top of the sequence modeling layer, a sequential conditional random field (CRF) layer is typically applied to perform label inference on the entire character sequence at once, as defined below:

[0098]

[0099] Among them, y s Represents all possible label sequences in s, and

[0100]

[0101] Among them, w y',y and = y',y These are all trainable parameters for the label pair (y', y). θ is a model hyperparameter that, for label prediction, searches for the label sequence y* with the highest conditional probability given the input sequence s. This can be solved using the Viterbi algorithm, specifically defined as follows:

[0102] y * =arg y maxp(y|s;θ) (12)

[0103] This invention addresses the shortcomings of traditional concept extraction models in fully utilizing word information by adding a relevant dictionary of new engineering professional terms during the model input stage, thereby improving the accuracy of professional term phrase extraction. In the word vector and character vector fusion section, a novel fusion method is employed. By statistically analyzing the frequency of different professional terms, they are fused with different weights, thus reducing information loss when using word vectors later. The sequence modeling layer applies BiLSTM and CRF to learn the relationships between labels, further improving extraction accuracy. This invention overcomes the deficiencies of existing technologies, accurately identifies knowledge concepts related to new engineering professional courses, provides a solid foundation for personalized education in the field of new engineering, and has significant application value.

[0104] This invention is not limited to the above embodiments. Based on the technical solutions disclosed in this invention, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

[0105] There are many methods and approaches to implement this technical solution, and the above are only preferred embodiments of this invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technology.

Claims

1. A dictionary-based method for extracting Chinese knowledge concepts for new engineering majors, characterized in that, Includes the following steps: (1) Obtain the courses of the new engineering disciplines and convert the textbooks and syllabi of all courses into raw text data; (2) Obtain the corresponding words by segmenting the original text data, and train the word2vec word vector model and the original word vectors on this basis; (3) Obtain relevant domain dictionary sets through Python web crawling technology, and form a candidate vocabulary set together; manually select important concepts of relevant majors as seed words, use the word2vec model trained in step (2) to perform word vector similarity calculation; select the skip-gram model to perform word vector similarity calculation, calculate the similarity with different seed words respectively, take the Top K as the external vocabulary, and convert it into a specified dimension word vector suitable for the NECE model, and form a professional dictionary of new engineering related courses with the words in the word segmentation file; (4) Construct the New Engineering Knowledge Concept Recognition (NECE) model, perform knowledge concept recognition on the original text data, and store the concept set; Step (4) includes the following steps: (41) Construct a new engineering knowledge concept recognition (NECE) model and pre-train the NECE model using existing labeled text data; the NECE model consists of a character embedding layer, a dictionary matching layer, a sequence modeling layer, and a CRF layer from bottom to top. (42) In the NECE model, the original text data is represented by character vector embedding by the bottom character embedding layer. For each character, the dictionary is matched to find the word in the dictionary. Then, the corresponding word information in the dictionary is concatenated with the character vector of the original text and input into the sequence modeling layer. This layer models the dependency relationship between characters. The sequence modeling layer adopts a bidirectional long short-term memory network and a convolutional neural network. Above the sequence modeling layer, the label inference of the entire character sequence is performed at once based on the conditional random field layer, so as to label each character, complete the recognition of knowledge concepts, output the new engineering related course concepts, and store them.

2. The dictionary-based method for extracting Chinese knowledge concepts for new engineering majors according to claim 1, characterized in that, Step (1) includes the following steps: (11) The courses of the new engineering disciplines, including Python-based computer vision processing, natural language processing and deep learning, were collected as text data, including their corresponding textbooks and teaching outlines. (12) Obtain the BMES sequence corresponding to the text data through manual annotation.

3. The dictionary-based method for extracting Chinese knowledge concepts for new engineering majors according to claim 1, characterized in that, Step (2) includes the following steps: (21) Perform data preprocessing on the original text data obtained in step (1), including traditional Chinese conversion, removal of XML symbols, removal of stop words and interference words, processing the word entry content into single-line data, and then obtaining the word segmentation file corresponding to the original text data through jieba word segmentation or LTP word segmentation technology. (22) Use the gensim package in Python to train the word2vec model using word segmentation file data, and train the vocabulary sets of courses in different disciplines separately; (23) Feature processing: convert text data into data that can be recognized by computers. Use word2vec's skip-gram model to convert words in the word segmentation file into word vectors of a specified dimension.

4. The dictionary-based method for extracting Chinese knowledge concepts for new engineering majors according to claim 1, characterized in that, Step (42) includes the following steps: (421) For the NECE model, the original input text is considered as: ； in, Each input character is represented by a dense matrix vector. This represents a word table lookup operation that converts characters into vectors. ； (422) Add word information vectors to the character embedding vectors. For each Chinese character in any input original text data... Using the professional dictionaries of new engineering courses, find all matching words and divide all matching words into four sets "B", "M", "E", and "S". The construction description of the four sets is as follows: ； ； ； （4）； Among them, L is the dictionary used here, represent subsequence in ; if a set is empty, that is, no matching word is found for the character, a special character "empty" is added to the empty word set; (423) Transform the four sets into vectors of a specified dimension, and use the frequency of each word as the weight in the vector compression representation. Specifically, assuming z(w) represents the frequency of the selected word in a fixed corpus, the weighted word set S is represented as: ； in: ； Here, w represents the selected knowledge concept vocabulary. All words in the four word sets "B", "M", "E" and "S" are weighted and normalized. Furthermore, if word w is found to be contained in a subsequence of another matching word during the statistical analysis, the word frequency of w will not increase. (424) Combine the four word sets into a fixed-dimensional feature and add it to the representation of each character. Concatenate the representations of the four word sets. The final representation of each character is: ； ； in, This represents the weighting function mentioned above; (425) After adding the dictionary information of new engineering disciplines, the generated vector The input is fed into the sequence modeling layer, which is implemented using a single-layer BiLSTM. The forward LSTM network is defined as follows: ； ； ； Where σ is the sigmoid function of the corresponding matrix element-wise multiplication. The matrix represents the product of elements at corresponding positions; W and b represent the trainable parameters in the model; backward LSTM has the same definition as forward LSTM, but models sequences in reverse order; the hidden state of the i-th step connection in forward LSTM and backward LSTM. This constitutes the context-related representation of ci; (426) At the top of the sequence modeling layer, the sequential conditional random field layer is applied to perform label inference on the entire character sequence at once, as specifically defined below: (10) in, represent All possible label sequences, These are hyperparameters that are fixed in the model; and （11） in and These are all trainable parameters for the label pair (y', y); for label prediction, given the input sequence... Under the premise of searching for the label sequence with the highest conditional probability. The Viterbi algorithm is used to solve the problem. .

Images