Knowledge element extraction method and device and medium
Patent Information
- Application Number
- CN202411204544.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2025-01-03
Smart Images
Figure CN119250174A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of information extraction, and in particular to a knowledge element extraction method, device and medium. Background Art
[0002] With the continuous development of information technology, the demand for knowledge acquisition is also increasing. How to effectively and quickly acquire accurate knowledge has become the research focus of many researchers. The knowledge element in the literature is the basic unit that reveals the knowledge content of the literature. It is composed of multiple sentences, including research background, research object, research method, research conclusion, etc. according to different contents.
[0003] Traditional knowledge element extraction methods are mostly based on rules and learning. With the advent of the big data era, the amount of knowledge is large and the presentation is complex. The requirements for extraction technology and tools are getting higher and higher, and research based on traditional machine learning and deep learning has gradually begun. At present, knowledge element extraction methods mainly include rule-based methods, traditional machine learning-based methods, and deep learning-based methods. However, rule-based methods require a lot of manual construction of rules, which is inefficient. Traditional machine learning-based methods use relatively simple features and their accuracy needs to be improved. Deep learning models mostly use machine features and lack consideration of multi-dimensional features, so the extraction results are not accurate enough.
[0004] Therefore, there is an urgent need for a knowledge element extraction method to solve the problem of insufficient accuracy of knowledge element extraction in the prior art. Summary of the invention
[0005] In view of this, the present invention provides a knowledge element extraction method, device, medium and equipment, the main purpose of which is to solve the current problem of inaccurate knowledge element extraction.
[0006] To solve the above problems, the present application provides a knowledge element extraction method, comprising:
[0007] Sentence processing is performed on the target document object to obtain a number of initial sentences;
[0008] Extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0009] Extracting semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0010] Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed respectively to obtain the target fusion features corresponding to each initial sentence;
[0011] Based on each of the target fusion features, knowledge element extraction is performed using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object, thereby obtaining the target knowledge element.
[0012] Optionally, the extracting statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence specifically includes:
[0013] Perform stop word removal processing on each initial sentence to obtain a word set corresponding to each initial sentence and including a number of non-stop words;
[0014] For each word set of the initial sentence and the predetermined target word set, respectively determine the chi-square value features corresponding to the word set to obtain the chi-square value features corresponding to the initial sentence;
[0015] At least a chi-square value feature based on each initial sentence is used as a statistical class feature corresponding to the initial sentence.
[0016] Optionally, the knowledge element extraction method further includes:
[0017] Based on each initial sentence, sentence-level features are extracted to obtain sentence-level features corresponding to each sentence;
[0018] Extracting chapter-level features for the target document object to obtain chapter-level features corresponding to the target document object;
[0019] Extract word-level features for the target document object to obtain word-level features corresponding to the target document object;
[0020] The chi-square value feature at least based on each initial sentence, as a statistical feature corresponding to the initial sentence, specifically includes:
[0021] Based on any one or more of the sentence-level features, the paragraph-level features, and the word-level features, and the chi-square value features of each initial sentence, as statistical features corresponding to the initial sentence.
[0022] Optionally, the sentence-level features include any one or more of the following: the number of conclusion feature words, the number of Chinese characters, the Chinese ratio, the number of non-Chinese characters, the ratio of Chinese conclusions to the total Chinese content, sentence length, citation information, the number of quotation marks, the number of semicolons, the number of commas, the number of periods, and the number of periods;
[0023] The chapter-level features include any one or more of the following: sentence order, sentence order ratio, total number of sentences, number of pages, number of Chinese and English authors, and impact factor;
[0024] The word-level features include any one or more of the following: the number of co-occurrences of words whose chi-square values are greater than a predetermined chi-square value threshold, the number of co-occurrences with article nouns, and the number of co-occurrences with keywords.
[0025] Optionally, the extracting semantic features from each initial sentence to obtain the semantic features specifically includes:
[0026] Using the pre-trained target feature extraction model, respectively extracting semantic features from each of the initial sentences to obtain semantic features corresponding to each of the initial sentences;
[0027] The target feature extraction model includes: a language representation model or a word embedding model.
[0028] Optionally, before performing sentence segmentation processing on the target document object, the method further includes:
[0029] Acquire several sample documents in advance and create a vocabulary based on each sample document;
[0030] A predetermined number of words are obtained based on the word library to construct the target word set.
[0031] Optionally, the statistical features and semantic features corresponding to the same initial sentence are respectively subjected to feature fusion processing to obtain target fusion features corresponding to each initial sentence, specifically including:
[0032] Performing horizontal feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a first fused feature;
[0033] Performing vertical feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a second fused feature;
[0034] The target fusion feature is obtained based on the first fusion feature and the second fusion feature corresponding to the same initial sentence.
[0035] Optionally, the target knowledge element extraction model includes an embedding layer, a long short-term memory network layer, a convolutional layer, a maximum pooling layer and an output layer connected in sequence.
[0036] In order to solve the above problems, the present application provides a knowledge element extraction device, comprising:
[0037] The sentence segmentation module is used to segment the target document object into sentences and obtain a number of initial sentences;
[0038] A first feature extraction module is used to extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0039] A second feature extraction module is used to extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0040] A fusion module is used to perform feature fusion processing based on the statistical features and semantic features corresponding to the same initial sentence, so as to obtain target fusion features corresponding to each initial sentence;
[0041] The knowledge element extraction module is used to extract knowledge elements based on the target fusion features using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object to obtain the target knowledge element.
[0042] In order to solve the above problems, the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned knowledge element extraction methods are implemented.
[0043] The knowledge element extraction method, device and medium in the present application increase the dimension of features by acquiring statistical features and semantic features corresponding to each initial sentence. Subsequently, statistical features and semantic features can be combined simultaneously to comprehensively extract knowledge elements, making the final extraction results more accurate and reasonable, improving the accuracy of knowledge element extraction, and at the same time improving the efficiency of knowledge element extraction.
[0044] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0046] Figure 1 This is a flow chart of a knowledge element extraction method according to an embodiment of the present application;
[0047] Figure 2 This is a structural block diagram of a knowledge element extraction device according to another embodiment of the present application;
[0048] Figure 3 This is a structural block diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION
[0049] Various aspects and features of the present application are described herein with reference to the accompanying drawings.
[0050] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.
[0051] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0052] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.
[0053] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.
[0054] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
[0055] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.
[0056] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.
[0057] The present application embodiment provides a knowledge element extraction method, which can be specifically applied to electronic devices such as terminals and servers, such as Figure 1 As shown, the method in this embodiment includes the following steps:
[0058] Step S101, performing sentence processing on the target document object to obtain a number of initial sentences;
[0059] In this step, the target document object may specifically be a scientific document. Before the target document object is subjected to sentence segmentation processing, the target document object may be cleaned first, and then the cleaned target document object is subjected to sentence segmentation processing. When performing sentence segmentation processing, the target document object may be divided into a plurality of initial sentences according to a predetermined rule. The predetermined rule may be a sentence segmentation rule based on predetermined sentence segmentation punctuation marks, that is, the sentence between the predetermined sentence segmentation punctuation marks and the sentence segmentation punctuation marks is taken as the initial sentence.
[0060] Step S102, extracting statistical features from each initial sentence to obtain statistical features corresponding to each initial sentence;
[0061] In this step, the statistical features specifically refer to the features obtained by statistics, induction, and calculation of the initial sentence or document object. Specifically, it may include the chi-square value CHI feature corresponding to the initial sentence, and may also include sentence-level features at the sentence level, chapter-level features at the chapter level, and word-level features at the word level, etc. In this step, by obtaining statistical features from various levels, the statistical features can be made more comprehensive, laying the foundation for subsequent feature fusion based on statistical features to obtain comprehensive target fusion features.
[0062] Step S103, extracting semantic features from each initial sentence to obtain semantic features corresponding to each initial sentence;
[0063] In this step, semantic features refer to features related to contextual semantics obtained through natural language processing, which can also be understood as text features obtained through deep learning.
[0064] In this step, when performing semantic feature extraction, a pre-trained target feature extraction model can be used to extract semantic features from each of the initial sentences to obtain semantic features corresponding to each of the initial sentences; the target feature extraction model includes: a language representation model BERT or a word embedding model Embedding. Other existing language models can also be used.
[0065] Step S104, based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed respectively to obtain target fusion features corresponding to each initial sentence;
[0066] In this step, when performing feature fusion, the feature vectors of semantic features and the feature vectors of statistical features may be subjected to horizontal feature fusion processing and vertical feature fusion processing respectively, so as to obtain target fusion features.
[0067] Step S105, based on each of the target fusion features, knowledge element extraction is performed using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object, so as to obtain the target knowledge element.
[0068] In this step, after obtaining the target fusion features corresponding to each initial sentence, the target fusion features can be input into the target knowledge element extraction model obtained by pre-training, and the target knowledge element extraction model is used to determine whether the initial sentence corresponding to the target fusion features is a knowledge element, that is, to determine a number of target sentences from each initial sentence, thereby completing the extraction of knowledge elements. In this embodiment, the target knowledge element extraction model may include an embedding layer, a long short-term memory network layer, a convolution layer, a maximum pooling layer, and an output layer connected in sequence.
[0069] The knowledge element extraction method in this embodiment increases the dimension of features by acquiring statistical features and semantic features corresponding to each initial sentence. Subsequently, the statistical features and semantic features can be combined simultaneously to comprehensively extract knowledge elements, making the final extraction result more accurate and reasonable, improving the accuracy of knowledge element extraction, and at the same time improving the efficiency of knowledge element extraction.
[0070] On the basis of the above embodiment, the present application provides another embodiment to provide a knowledge element extraction method. In this embodiment, when performing statistical feature extraction to obtain statistical features corresponding to each initial sentence, that is, when executing step S102, the specific process is as follows:
[0071] Step S1021, performing stop word removal processing on each initial sentence to obtain a word set corresponding to each initial sentence and including a number of non-stop words;
[0072] In this step, the initial sentence S can be segmented to obtain the initial word set W corresponding to each initial sentence. s ; Then, based on the predetermined stop word set, the initial word set W s Perform stop word removal to obtain the word set W of the initial sentence S s ′. Among them, the set of stop words stword is: stword = ['a kind of', 'from', 'to', 'of', 'and', 'and', 'among', 'one', 'and', 'in', 'of'...].
[0073] Step S1022, for each word set of the initial sentence and the predetermined target word set, respectively determine the chi-square value features corresponding to the word set to obtain the chi-square value features corresponding to the initial sentence;
[0074] In this step, taking the document object as a scientific document as an example, a predetermined number of scientific and technological words can be obtained in advance to pre-construct a target word set. For example, the target word set can contain 200 scientific and technological words. Specifically, several sample documents can be obtained in advance, and a vocabulary can be created based on each sample document; then, based on the vocabulary, the top 200 common high CHI words are manually screened and used as the target word set for calculating the CHI value vector of sentence S, which is recorded as CHI_W.
[0075] In this embodiment, after obtaining the target word set CHI_W, it can be determined in turn whether each word in CHI_W exists in the word set W. s ′; if it exists, the value of the corresponding position of the word in CHI_W is set to 1, if it does not exist, it is set to 0, so as to obtain the word set W s ′, that is, to obtain the vector corresponding to the word set W s ′, and then obtain the chi-square value feature corresponding to the initial sentence.
[0076] For example, the word set W s ′ is: W s ′=['a','b','c'],CHI_W=['b','o','a','c'],then the word set W s ′ corresponds to a vector of [1,0,1,1]. Therefore, the chi-square value feature / chi-square value vector corresponding to the initial sentence is [1,0,1,1]. In this embodiment, the actual reference CHI_W can be 200 words; the word set W s If the number of words in ′ is less than 200, it can be padded with zeros to ensure that the length of the word set is 200.
[0077] Step S1023, at least based on the chi-square value feature of each initial sentence, as the statistical feature corresponding to the initial sentence.
[0078] In this step, after obtaining the chi-square value feature corresponding to each initial sentence, the chi-square value feature can be used as the statistical feature corresponding to each initial sentence.
[0079] In this embodiment, by adopting the above method to obtain the chi-square value features of each initial sentence, the acquisition of the chi-square value features can be made more reasonable and accurate, thereby making the statistical features more reasonable and accurate, laying the foundation for the subsequent fusion based on the accuracy of statistical features to obtain the target fusion features.
[0080] In the specific implementation process of this embodiment, in order to make the statistical features more comprehensive, after obtaining the chi-square value features, it is further possible to combine any one or more of the sentence-level features, chapter-level features, word-level features, and conclusion paragraph features to comprehensively obtain the statistical features. That is, sentence-level features can be extracted based on each initial sentence to obtain the sentence-level features corresponding to each sentence; chapter-level features can be extracted for the target document object to obtain the chapter-level features corresponding to the target document object; word-level features can be extracted for the target document object to obtain the word-level features corresponding to the target document object; conclusion paragraph features can be extracted based on the conclusion paragraph to obtain the conclusion paragraph features; finally, based on any one or more of the sentence-level features, chapter-level features, word-level features, and conclusion paragraph features, and the chi-square value features of each initial sentence, the statistical features corresponding to the initial sentence can be obtained.
[0081] In this embodiment, the sentence-level features include any one or more of the following: the number of conclusion feature words (num_top1_ci), the number of Chinese characters (chinese_word_len), the Chinese ratio (word_rate), the number of non-Chinese characters (other_lange_len), the ratio of Chinese conclusions to the total Chinese text (word_rate_jielun), sentence length (sen_len), citation information (Isyin, i.e., whether to quote), the number of quotation marks (num_quotes), the number of semicolons (num_semic), the number of commas (num_comma), the number of colons (num_caesura), and the number of periods (num_colon).
[0082] The chapter-level features include any one or more of the following: sentence order (sen_order), sentence order ratio (sense_rate), total number of sentences (paper_ct), number of pages (yeshu), number of Chinese and English authors (zzgs), and impact factor (yxyz). In this embodiment, the impact factor refers to the impact factor of the journal in which the article is located. Specifically, the corresponding impact factor can be pre-configured for each journal according to the level and scale of the journal.
[0083] Word-level features include any one or more of the following: the number of co-occurrences of words with chi-square values greater than a predetermined chi-square value threshold (len_kword_same, i.e., the number of co-occurrences of high CHI words), the number of co-occurrences with article nouns (len_tword_same), and the number of co-occurrences with keywords (len_key_word_zwgjc_same).
[0084] The features of the conclusion paragraph include any one or more of the following: the first judgment method of the conclusion sentence in the conclusion paragraph (jielunci_first), the second judgment method of the conclusion sentence in the conclusion paragraph (jielunci_in), whether it is located in the conclusion paragraph (is_jielunduan), and the position in the conclusion paragraph (order_jielun). Regarding "whether it is located in the conclusion paragraph (is_jielunduan)", "located in the conclusion paragraph" can be preset to "1" and "not located in the conclusion paragraph" to "0".
[0085] In this embodiment, taking the above-mentioned chi-square value features (top200), sentence-level features, paragraph-level features, word-level features and conclusion paragraph features as an example, each initial sentence corresponds to 225 feature values, thereby obtaining statistical features corresponding to the initial sentence.
[0086] In this embodiment, by obtaining the chi-square value feature and further combining the sentence-level features, the chapter-level features, the word-level features, and the conclusion paragraph features, the statistical features can be made more comprehensive, laying the foundation for the subsequent multi-dimensional feature fusion based on statistical features, which is conducive to improving the accuracy of subsequent knowledge element extraction. At the same time, by combining the basic text features of the document with the CHI value features, an artificial feature vector / statistical feature is constructed, so that in the downstream knowledge extraction task, the text's own characteristics are fully considered, without relying on rules or traditional machine learning models, which facilitates the subsequent feature fusion.
[0087] Another embodiment of the present application provides a knowledge element extraction method. In the specific implementation process of this embodiment, when performing feature processing, that is, when executing step S103, the specific process is as follows:
[0088] Step S1031, performing horizontal feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a first fused feature;
[0089] In this step, since the statistical features are numerical features, a feature value sequence can be constructed based on each feature value in the statistical features to obtain a feature vector. The feature vector of the statistical features of the initial sentence S is w i , the feature vector of the semantic feature of the initial sentence S is M_vec_embedding as an example, when performing horizontal feature fusion, specifically along the row direction, the feature vector is w i Fusion with the feature vector M_vec_embedding. That is, before fusion:
[0090] The feature vector M_vec_embedding of the semantic features of sentence S is expressed as: M_vec_embedding = [w1w2...w j ].
[0091] The feature vector H_vec of the statistical class of sentence S is expressed as: H_vec = [h1,h2,h3,h4,…,h v ].
[0092] After horizontal fusion, the feature vector HM_h of the first fusion feature vec It is expressed as:
[0093]
[0094] Step S1032, performing vertical feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a second fused feature;
[0095] In this step, since the statistical features are numerical features, a feature value sequence can be constructed based on each feature value in the statistical features to obtain a feature vector. The feature vector of the statistical features of the initial sentence S is w i , the feature vector of the semantic feature of the initial sentence S is M_vec_embedding as an example, when performing horizontal feature fusion, specifically along the row direction, the feature vector is w i Fusion with the feature vector M_vec_embedding. That is, before fusion:
[0096] The feature vector M_vec_embedding of the semantic features of sentence S is expressed as: M_vec_embedding = [w1w2...w j ].
[0097] The feature vector H_vec of the statistical class of sentence S is expressed as: H_vec = [h1,h2,h3,h4,…,h v ].
[0098] After vertical fusion, the feature vector HM_v of the second fusion feature vec It is expressed as:
[0099] Step S1033: obtaining the target fusion feature based on the first fusion feature and the second fusion feature corresponding to the same initial sentence.
[0100] In this step, after obtaining the first fusion feature and the second fusion feature, the first fusion feature and the second fusion feature can be used as target fusion features.
[0101] In this embodiment, by adopting the above method to perform feature fusion, the feature fusion result can be made more reasonable and accurate, laying a foundation for the subsequent accurate knowledge element extraction based on the target fusion features obtained by fusion. In addition, by realizing horizontal and vertical fusion of statistical features and semantic features in the form of vectors, instead of only considering the features of one dimension, the efficiency and accuracy of knowledge element extraction are greatly improved.
[0102] Based on the above embodiment, another embodiment of the present application provides a knowledge element extraction method. Taking knowledge element extraction from scientific and technological literature as an example, the method in this embodiment specifically includes the following steps:
[0103] Step 1: Collect some sample scientific and technological literature;
[0104] In this step, the basic document metadata in the document, including the full text of the document, can be processed to generate a paper library. ; Word2Vector (W2V) is used to generate word vectors for all words and save them in a pkl file.
[0105] Step 2: Sentence processing is performed on each sample scientific document to obtain a number of initial sample sentences;
[0106] In this step, multiple sample documents can be segmented into sentences respectively, and word segmentation can be completed based on jieba (stammering), and finally the sentence and word segmentation results are stored in the sentence library.
[0107] Step 3: extract statistical features and semantic features from each initial sample sentence to obtain sample statistical features and sample semantic features corresponding to each initial sample sentence;
[0108] In this step, when the statistical features are extracted from the initial sample sentence, the extraction process is the same as the principle of the statistical features extracted from the initial sentence in the above embodiment, which is also based on the initial sample sentence to obtain a sample word set containing several non-stop words, and then based on the sample words in the sample word set and each word in the target word set, the CHI value is determined to obtain the CHI sequence / CHI vector corresponding to each sample word set, thereby obtaining the CHI value feature corresponding to the initial sample sentence; similarly, the sentence level feature, chapter level feature, and word level feature corresponding to the initial sample sentence are obtained. Finally, based on any one or more of the sentence level feature, chapter level feature, and word level feature corresponding to the same initial sample sentence, as well as the CHI value feature, the statistical features of the initial sample sentence are obtained.
[0109] In this step, when the semantic features are extracted from the initial sample sentence, the extraction process is the same as the principle of extracting the semantic features from the initial sentence in the above embodiment, and the semantic features are extracted based on the target feature extraction model. In this embodiment, the target feature extraction model can be a pre-trained language representation model BERT or a word embedding model Embedding, etc., and other existing language models can also be used.
[0110] Step 4: Based on the statistical features and semantic features corresponding to the same initial sample sentence, feature fusion processing is performed to obtain sample fusion features corresponding to each initial sample sentence;
[0111] In this step, when performing feature fusion, two fusion methods, horizontal fusion and vertical fusion, are also adopted to obtain the first sample fusion feature and the second sample fusion feature corresponding to the same initial sample sentence, thereby obtaining the sample fusion feature.
[0112] Step 5: Based on the sample fusion features of each initial sample sentence and the knowledge meta-label of each initial sample sentence, the initial knowledge meta-extraction model is trained to obtain a target knowledge meta-extraction model;
[0113] In this step, it can be determined in advance whether each initial sample sentence is a knowledge element, so as to configure a knowledge element label for each initial sample sentence in advance. Then, the sample fusion feature is input into the initial knowledge element extraction model, and the initial knowledge element extraction model is used to determine whether the sample fusion feature is a knowledge element, and whether each initial sample sentence is an initial extraction result of the knowledge element; then, the extraction result of each initial sample sentence is compared with the knowledge element label to determine whether the predetermined training stop condition is met; when the predetermined training stop condition is not met, the model parameters in the initial knowledge element extraction model are adjusted to obtain the current knowledge element extraction model, and based on the current knowledge element extraction model, it is re-determined whether each sample fusion feature is a knowledge element to obtain the current extraction result of each initial sample sentence, until it is determined based on the current extraction result that the predetermined training stop condition is met, or the number of training rounds reaches the predetermined number of training rounds, the current knowledge element extraction model is used as the target knowledge element extraction model, thereby completing the training of the target knowledge element extraction model.
[0114] In this step, after completing the model training, you can also perform a step based on the fusion feature vector in the test set to complete the test evaluation, and then complete the model saving. Among them, the model test and evaluation use P value, R value and F1 value (PRF value). Among them, P value represents the accuracy; R value represents the recall rate; F value is F1 value, F1_score. The results show that feature fusion has a significant impact on the improvement of knowledge element extraction, among which the vertical fusion method has a particularly outstanding improvement effect.
[0115] Step 6: Sentence the target document object to obtain several initial sentences;
[0116] In this step, the process of performing sentence processing is the same as the process of performing sentence processing in the above embodiment, and will not be repeated here.
[0117] Step 7: extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0118] In this step, the statistical feature extraction process is the same as the statistical feature extraction process in the above embodiment, which will not be repeated here.
[0119] Step 8: extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0120] In this step, the process of extracting semantic features is the same as that in the above embodiment, and will not be repeated here.
[0121] Step nine: Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed respectively to obtain target fusion features corresponding to each initial sentence;
[0122] In this step, the process of performing feature fusion is the same as the process of performing feature fusion in the above embodiment, and will not be repeated here.
[0123] Step 10: Based on the target fusion features, a pre-trained target knowledge element extraction model is used to extract knowledge elements, and a target sentence corresponding to the target document object is obtained to obtain the target knowledge element.
[0124] In this step, after the feature fusion is completed, the target fusion feature can be directly used as the input of the target knowledge element extraction model to carry out subsequent sentence judgment, thereby completing the knowledge element extraction. That is, the target knowledge element extraction model is used to judge whether the initial sentence corresponding to the target fusion feature is a knowledge element according to the target fusion feature. When it is determined to be a knowledge element, the initial sentence is used as the target sentence. This application adopts the deep learning model LSTM_CNN integrated with the long short-term memory artificial neural network LSTM and the convolutional neural network CNN as the model architecture of the target knowledge element extraction model, that is, the output of LSTM is used as the input of CNN; and the input of the entire integrated model is the fusion of artificial features and machine features. That is, the target knowledge element extraction model in this embodiment includes an embedding layer, a long short-term memory network layer, a convolution layer, a maximum pooling layer and an output layer connected in sequence. In this embodiment, the integrated deep learning model is a binary classification model in the final output layer. The output of the model has two results, which are 0 and 1 respectively. 1 indicates that the sentence represented by the fusion feature is the target knowledge element, and 0 indicates that the sentence is not a target knowledge element.
[0125] The knowledge element providing method in this application constructs multiple artificial feature indicators / statistical features based on the characteristics of scientific and technological literature, and realizes feature fusion with machine features / semantic features from the horizontal and vertical directions, and completes the extraction of scientific and technological literature knowledge elements through a deep learning model, which significantly improves the extraction effect and accuracy, thereby providing basic support for the construction of the knowledge base and the accurate interpretation of literature knowledge, and can extract scientific and technological literature knowledge elements conveniently, quickly and accurately.
[0126] Another embodiment of the present application provides a knowledge element extraction device, such as Figure 2 As shown, including:
[0127] Sentence segmentation module 11, used for performing sentence segmentation processing on the target document object to obtain a number of initial sentences;
[0128] A first feature extraction module 12 is used to extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0129] The second feature extraction module 13 is used to extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0130] A fusion module 14 is used to perform feature fusion processing based on the statistical features and semantic features corresponding to the same initial sentence, respectively, to obtain target fusion features corresponding to each initial sentence;
[0131] The knowledge element extraction module 15 is used to extract knowledge elements based on each of the target fusion features using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object to obtain the target knowledge element.
[0132] In the specific implementation process of this embodiment, the first feature extraction module is specifically used to: perform stop word removal processing on each initial sentence to obtain a word set corresponding to each initial sentence and containing a number of non-stop words; determine the chi-square value features corresponding to the word set of each initial sentence and a predetermined target word set to obtain the chi-square value features corresponding to the initial sentence; at least based on the chi-square value features of each initial sentence, use them as statistical features corresponding to the initial sentence.
[0133] In the specific implementation process of this embodiment, the first feature extraction module is also used to: perform sentence-level feature extraction based on each initial sentence to obtain sentence-level features corresponding to each sentence; perform chapter-level feature extraction for the target document object to obtain chapter-level features corresponding to the target document object; perform word-level feature extraction for the target document object to obtain word-level features corresponding to the target document object; and use any one or more of the sentence-level features, chapter-level features, and word-level features, as well as the chi-square value features of each initial sentence as statistical features corresponding to the initial sentence.
[0134] In the specific implementation process of this embodiment, the sentence-level features include any one or more of the following: the number of conclusion feature words, the number of Chinese characters, the Chinese ratio, the number of non-Chinese characters, the ratio of Chinese conclusions to the total Chinese, the sentence length, the citation information, the number of quotation marks, the number of semicolons, the number of commas, the number of periods, and the number of periods;
[0135] The chapter-level features include any one or more of the following: sentence order, sentence order ratio, total number of sentences, number of pages, number of Chinese and English authors, and impact factor;
[0136] The word-level features include any one or more of the following: the number of co-occurrences of words whose chi-square values are greater than a predetermined chi-square value threshold, the number of co-occurrences with article nouns, and the number of co-occurrences with keywords.
[0137] In the specific implementation process of this embodiment, the second feature extraction module is specifically used to: use the pre-trained target feature extraction model to extract semantic features for each of the initial sentences to obtain semantic features corresponding to each initial sentence; the target feature extraction model includes: the language representation model BERT or the word embedding model Embedding.
[0138] In the specific implementation process of this embodiment, the knowledge element extraction device also includes a construction module, which is used to: pre-acquire a number of sample documents and create a vocabulary based on each sample document; filter a predetermined number of words based on the vocabulary to construct the target word set.
[0139] During the specific implementation of this embodiment, the fusion module is specifically used to: perform horizontal feature fusion processing on the feature vectors of statistical features and the feature vectors of semantic features corresponding to the same initial sentence to obtain a first fusion feature; perform vertical feature fusion processing on the feature vectors of statistical features and the feature vectors of semantic features corresponding to the same initial sentence to obtain a second fusion feature; obtain the target fusion feature based on the first fusion feature and the second fusion feature corresponding to the same initial sentence.
[0140] In the specific implementation process of this embodiment, the target knowledge element extraction model includes an embedding layer, a long short-term memory network layer, a convolutional layer, a maximum pooling layer and an output layer connected in sequence.
[0141] The knowledge element extraction device in this embodiment increases the dimension of features by acquiring statistical features and semantic features corresponding to each initial sentence. Subsequently, the statistical features and semantic features can be combined simultaneously to comprehensively extract knowledge elements, making the final extraction result more accurate and reasonable, thereby improving the accuracy of knowledge element extraction.
[0142] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:
[0143] Step 1: Sentence the target document object to obtain several initial sentences;
[0144] Step 2: extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0145] Step 3: extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0146] Step 4: Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed to obtain target fusion features corresponding to each initial sentence;
[0147] Step 5: Based on the target fusion features, a pre-trained target knowledge element extraction model is used to extract knowledge elements, and a target sentence corresponding to the target document object is obtained to obtain the target knowledge element.
[0148] The specific implementation process of the above method steps can refer to the embodiments of any of the above-mentioned knowledge element providing methods, and this embodiment will not be repeated here.
[0149] The storage medium in the present application increases the dimension of features by acquiring statistical features and semantic features corresponding to each initial sentence. Subsequently, the statistical features and semantic features can be combined simultaneously to comprehensively extract knowledge elements, making the final extraction results more accurate and reasonable, thereby improving the accuracy of knowledge element extraction.
[0150] Another embodiment of the present application provides an electronic device, such as Figure 3 As shown, at least a memory 1 and a processor 2 are included, the memory 1 stores a computer program, and the processor 2 implements the following method steps when executing the computer program on the memory 1:
[0151] Step 1: Sentence the target document object to obtain several initial sentences;
[0152] Step 2: extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence;
[0153] Step 3: extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence;
[0154] Step 4: Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed to obtain target fusion features corresponding to each initial sentence;
[0155] Step 5: Based on the target fusion features, a pre-trained target knowledge element extraction model is used to extract knowledge elements, and a target sentence corresponding to the target document object is obtained to obtain the target knowledge element.
[0156] The specific implementation process of the above method steps can refer to the embodiments of any of the above-mentioned knowledge element providing methods, and this embodiment will not be repeated here.
[0157] The electronic device in the present application increases the dimension of features by acquiring statistical features and semantic features corresponding to each initial sentence. Subsequently, the statistical features and semantic features can be combined simultaneously to comprehensively extract knowledge elements, making the final extraction results more accurate and reasonable, thereby improving the accuracy of knowledge element extraction.
[0158] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A knowledge element extraction method, characterized in that: include: Sentence processing is performed on the target document object to obtain a number of initial sentences; Extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence; Extracting semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence; Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed respectively to obtain the target fusion features corresponding to each initial sentence; Based on each of the target fusion features, knowledge element extraction is performed using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object, thereby obtaining the target knowledge element.
2. The method according to claim 1, characterized in that The extracting of statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence specifically includes: Perform stop word removal processing on each initial sentence to obtain a word set corresponding to each initial sentence and including a number of non-stop words; For each word set of the initial sentence and the predetermined target word set, respectively determine the chi-square value features corresponding to the word set to obtain the chi-square value features corresponding to the initial sentence; At least a chi-square value feature based on each initial sentence is used as a statistical class feature corresponding to the initial sentence.
3. The method according to claim 2, characterized in that The method further comprises: Based on each initial sentence, sentence-level features are extracted to obtain sentence-level features corresponding to each sentence; Extracting chapter-level features for the target document object to obtain chapter-level features corresponding to the target document object; Extract word-level features for the target document object to obtain word-level features corresponding to the target document object; The chi-square value feature at least based on each initial sentence, as a statistical feature corresponding to the initial sentence, specifically includes: Based on any one or more of the sentence-level features, the passage-level features, and the word-level features, and the chi-square value features of each initial sentence, as statistical features corresponding to the initial sentence.
4. The method according to claim 3, characterized in that The sentence-level features include any one or more of the following: the number of conclusion feature words, the number of Chinese characters, the Chinese ratio, the number of non-Chinese characters, the ratio of Chinese conclusions to the total Chinese, sentence length, citation information, the number of quotation marks, the number of semicolons, the number of commas, the number of periods, and the number of periods; The chapter-level features include any one or more of the following: sentence order, sentence order ratio, total number of sentences, number of pages, number of Chinese and English authors, and impact factor; The word-level features include any one or more of the following: the number of co-occurrences of words whose chi-square values are greater than a predetermined chi-square value threshold, the number of co-occurrences with article nouns, and the number of co-occurrences with keywords.
5. The method according to claim 1, characterized in that The extracting of semantic features from each initial sentence to obtain semantic features specifically includes: Using the pre-trained target feature extraction model, respectively extracting semantic features from each of the initial sentences to obtain semantic features corresponding to each of the initial sentences; The target feature extraction model includes: a language representation model or a word embedding model.
6. The method according to claim 2, characterized in that Before performing sentence segmentation processing on the target document object, the method further includes: Acquire several sample documents in advance and create a vocabulary based on each sample document; A predetermined number of words are obtained based on the word library to construct the target word set.
7. The method according to claim 1, characterized in that Based on the statistical features and semantic features corresponding to the same initial sentence, feature fusion processing is performed respectively to obtain the target fusion features corresponding to each initial sentence, specifically including: Performing horizontal feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a first fused feature; Performing vertical feature fusion processing on the feature vectors of the statistical features and the feature vectors of the semantic features corresponding to the same initial sentence to obtain a second fused feature; The target fusion feature is obtained based on the first fusion feature and the second fusion feature corresponding to the same initial sentence.
8. The method according to any one of claims 1 to 7, characterized in that: The target knowledge element extraction model includes an embedding layer, a long short-term memory network layer, a convolutional layer, a maximum pooling layer and an output layer which are connected in sequence.
9. A knowledge element extraction device, characterized in that: include: The sentence segmentation module is used to segment the target document object into sentences and obtain a number of initial sentences; A first feature extraction module is used to extract statistical features for each initial sentence to obtain statistical features corresponding to each initial sentence; A second feature extraction module is used to extract semantic features for each initial sentence to obtain semantic features corresponding to each initial sentence; A fusion module is used to perform feature fusion processing based on the statistical features and semantic features corresponding to the same initial sentence, so as to obtain target fusion features corresponding to each initial sentence; The knowledge element extraction module is used to extract knowledge elements based on the target fusion features using a pre-trained target knowledge element extraction model to obtain a target sentence corresponding to the target document object to obtain the target knowledge element.
10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the knowledge element extraction method described in any one of claims 1 to 7 are implemented.