Method and device for concept expansion of scientific and technological concept map

Abstract

This application proposes a method for concept expansion of a scientific and technological concept map, including: using full-text retrieval and rule filtering to query the documents associated with the node to be expanded in the input corpus, and selecting a predetermined proportion of documents as target documents according to the display order of the queried documents; selecting self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set and screening them to obtain the target seed concept; based on the target document, using N-Gram to generate a first candidate phrase set, and using a keyword extraction algorithm to generate a second candidate phrase set, and then generating a candidate concept set based on the first and second candidate phrase sets; screening the candidate concept set to obtain the target candidate concept; using the target seed concept to adjust the weight of the target candidate concept to generate a target concept set. This application using the above scheme can complete the expansion task in a variety of scientific and technological fields, and take into account both accuracy and recall.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method and device for conceptual expansion of a scientific and technological concept map. Background Art

[0002] The science and technology concept map is a tree structure generated based on the internal connections between existing scientific systems, which divides different types of scientific research. The science and technology concept map has built a knowledge system, but it still has the problem of insufficient granularity and cannot effectively support downstream tasks such as paper indexing and scholar recommendation.

[0003] Concept expansion concretizes abstract subject names into related concepts by mining the terms and knowledge in the field. Concept expansion can be used to divide the research directions of academic literature. For example, if words such as knowledge completion and entity disambiguation appear in papers, they can be classified into the field of knowledge graphs. It can also complete the association between scholars and scientific and technological concept graphs based on scholar portraits. For example, scholars’ keywords include image processing and target detection, which shows that their main research field is computer vision. The task most similar to concept expansion is domain phrase mining. Domain phrase mining methods can be divided into supervised algorithms and unsupervised algorithms according to whether there is labeled data or not. Candidate phrases are generated using methods such as N-Gram. Supervised algorithms score candidate phrases by training binary classifiers, which rely on manual annotation and are costly. Unsupervised algorithms use multiple features of candidate phrases to calculate scores and are suitable for scenarios with no labeled data.

[0004] There have been many research results based on unsupervised domain phrase mining methods. Park et al. [Park, 2002] used the frequency of phrases in specific and general domains to calculate their domain scores, used the frequency of phrases and the frequency of words in phrases to calculate their consistency scores, and finally sorted them by linear combination. Pan et al. [Pan, 2017] proposed a graph propagation-based method to sort candidate phrases, using mutual information to calculate phrase consistency as node information, learning phrase representations through Word2Vec [Mikolov, 2013] and using semantic similarity between phrases as side information, initializing phrase confidence and continuously iterating and updating until convergence. However, these methods cannot solve the concept extension problem of scientific and technological concept graphs well. Summary of the invention

[0005] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0006] To this end, the first purpose of this application is to propose a method for conceptual expansion of scientific and technological concept maps, which solves the technical problem that existing methods are only applicable to specific fields but have low accuracy, realizes the expansion task to multiple scientific and technological fields, and takes into account both accuracy and recall rate.

[0007] The second purpose of this application is to propose a device for conceptual expansion of a scientific and technological concept map.

[0008] A fourth object of the present application is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a method for concept expansion of a scientific and technological concept map, including: using full-text retrieval and rule filtering to query the documents associated with the node to be expanded in the input corpus, and selecting a predetermined proportion of documents as target documents according to the display order of the queried documents; selecting self-filled keywords in the target documents whose length is greater than a preset threshold as a candidate seed concept set; screening the candidate seed concept set to obtain a target seed concept; based on the target document, using N-Gram to generate a first candidate phrase set, and using a keyword extraction algorithm to generate a second candidate phrase set, and then generating a candidate concept set based on the first candidate phrase set and the second candidate phrase set; screening the candidate concept set to obtain a target candidate concept; using the target seed concept to adjust the weight of the target candidate concept to generate a target concept set.

[0010] The method for conceptual expansion of a scientific and technological concept map in an embodiment of the present application selects paper data as a corpus, selects self-filled keywords in the paper data as seed concepts to guide the generation of candidate phrases, sets penalty coefficients for concepts appearing under multiple sibling nodes during the guidance process, and calculates the contextual relevance between the seed concept and the candidate concept, so as to achieve the purpose of fine-tuning the weights of the candidate concepts.

[0011] Optionally, in one embodiment of the present application, screening the candidate seed concept set includes:

[0012] After removing duplicates from the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated;

[0013] The score of each candidate seed concept is calculated based on semantic similarity and string similarity, and all candidate seed concepts are sorted and screened according to the score to obtain the target seed concept.

[0014] Optionally, in one embodiment of the present application, a keyword extraction algorithm is used to generate a second candidate phrase set, including:

[0015] Based on the target document, use the word segmenter to segment the title and abstract of each document in the input corpus in advance;

[0016] Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set;

[0017] Taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set;

[0018] Screen the candidate concept set, including:

[0019] Calculate the score of each candidate concept in the candidate concept set, and retain the concepts with scores greater than a fixed threshold as the final target candidate concepts.

[0020] Among them, the TOPN method is used to sort and filter all the words after word segmentation to obtain the second candidate phrase set, including:

[0021] Calculate the TF-IDF value of the phrases contained in each document in the target document, and select candidate phrases for a single document based on the TF-IDF value;

[0022] Constructing a second candidate phrase set based on candidate phrases of all documents in the target document;

[0023] The score for each candidate concept is:

[0024]

[0025] Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

[0026] Optionally, in one embodiment of the present application, the weight of the target candidate concept is adjusted using the target seed concept, including:

[0027] According to the target candidate concept and the target seed concept, find the corresponding context in the target document abstract where they appear;

[0028] Calculate the semantic relevance of each target candidate concept with all target seed concepts, and the semantic relevance of the context of each target candidate concept with all target seed concepts;

[0029] The two scores are combined as the fine-tuned weights of the target candidate concepts, and all target candidate concepts with fine-tuned weights constitute the target concept set.

[0030] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a device for concept expansion of a scientific and technological concept map, including a target seed concept acquisition module, a target candidate concept generation module, and a concept weight fine-tuning module, wherein:

[0031] Get target seed concept module, used for:

[0032] Use full-text search and rule filtering to query the documents associated with the node to be expanded in the input corpus, and select a predetermined proportion of documents as target documents according to the display order of the queried documents;

[0033] Select self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set;

[0034] Screen the candidate seed concept set to obtain the target seed concept;

[0035] Generate target candidate concept module for:

[0036] Based on the target document, a first candidate phrase set is generated using N-Gram, and a second candidate phrase set is generated using a keyword extraction algorithm, and then a candidate concept set is generated based on the first candidate phrase set and the second candidate phrase set;

[0037] Screen the candidate concept set to obtain the target candidate concept;

[0038] The fine-tuning concept weight module is used to adjust the weights of the target candidate concepts using the target seed concepts to generate a target concept set.

[0039] Optionally, in one embodiment of the present application, screening the candidate seed concept set includes:

[0040] After removing duplicates from the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated;

[0041] The score of each candidate seed concept is calculated based on semantic similarity and string similarity, and all candidate seed concepts are sorted and screened according to the score to obtain the target seed concept.

[0042] Optionally, in one embodiment of the present application, a keyword extraction algorithm is used to generate a first candidate phrase set, including:

[0043] Based on the target document, use the word segmenter to segment the title and abstract of each document in the input corpus in advance;

[0044] Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set;

[0045] Taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set;

[0046] Screen the candidate concept set, including:

[0047] Calculate the score of each candidate concept in the candidate concept set, and retain the concepts with scores greater than a fixed threshold as the final target candidate concepts.

[0048] Among them, the TOPN method is used to sort and filter all the words after word segmentation to obtain the second candidate phrase set, including:

[0049] Calculate the TF-IDF value of the phrases contained in each document in the target document, and select candidate phrases for a single document based on the TF-IDF value;

[0050] Constructing a second candidate phrase set based on candidate phrases of all documents in the target document;

[0051] The score for each candidate concept is:

[0052]

[0053] Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

[0054] Optionally, in one embodiment of the present application, fine-tuning the concept weight module is specifically used to:

[0055] According to the target candidate concept and the target seed concept, find the corresponding context in the target document abstract where they appear;

[0056] Calculate the semantic relevance of each target candidate concept with all target seed concepts, and the semantic relevance of the context of each target candidate concept with all target seed concepts;

[0057] The two scores are combined as the fine-tuned weights of the target candidate concepts, and all target candidate concepts with fine-tuned weights constitute the target concept set.

[0058] In order to achieve the above objectives, the third aspect of the present invention proposes a non-temporary computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor, enables the execution of the above-mentioned method for conceptual expansion of the scientific and technological concept map.

[0059] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0061] Figure 1 A schematic diagram of a process for expanding the concept of a scientific and technological concept map provided in Example 1 of the present application;

[0062] Figure 2 The main framework process of the concept expansion of the embodiment of the present application;

[0063] Figure 3 This is an example sub-graph of the technology concept map of the embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of a device for expanding the concepts of a scientific and technological concept map provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0066] The following describes the method and device for conceptual expansion of a scientific and technological concept map in an embodiment of the present application with reference to the accompanying drawings.

[0067] Figure 1 A flowchart of a method for expanding the concepts of a scientific and technological concept map provided in Example 1 of the present application.

[0068] like Figure 1 As shown, the method for expanding the concept of the scientific and technological concept map includes the following steps:

[0069] Step 101, using full-text search and rule filtering to search for documents associated with the node to be expanded in the input corpus, and selecting a predetermined proportion of documents as target documents according to the display order of the searched documents;

[0070] Step 102, selecting self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set;

[0071] Step 103, screening the candidate seed concept set to obtain the target seed concept;

[0072] Step 104, based on the target document, using N-Gram to generate a first candidate phrase set, and using a keyword extraction algorithm to generate a second candidate phrase set, and then generating a candidate concept set based on the first candidate phrase set and the second candidate phrase set;

[0073] Step 105, screening the candidate concept set to obtain the target candidate concept;

[0074] Step 106: Use the target seed concept to adjust the weight of the target candidate concept to generate a target concept set.

[0075] The method for conceptual expansion of the science and technology concept map in the embodiment of the present application is not aimed at a specific field but at all underlying fields in the science and technology concept map. The present application aims to extract technology-related research terms from academic literature and organize them into a hierarchical structure. The basic challenge facing concept expansion is how to extract high-quality and informative phrases. Existing methods are applicable to specific fields and have low accuracy. The present application can complete the expansion task of multiple science and technology fields, and take into account both accuracy and recall.

[0076] The premise of the concept expansion of this application is to obtain text data related to the subject to be expanded. The paper itself is relatively standardized and there are many terms that are easy to extract. Therefore, this application selects paper data as a corpus. Due to the lack of high-quality phrases in the concept expansion of a specific field, existing methods require manual definition of a small number of phrases to judge the quality of candidate phrases. This application takes into account the high quality of self-filled keywords in paper data, so such words are selected as seed concepts to guide the generation of candidate phrases. There is a certain degree of similarity between brother nodes in the scientific and technological concept map. For example, the sub-fields under artificial intelligence include knowledge engineering and knowledge representation. This application sets a penalty coefficient for concepts that appear under multiple brother nodes. Based on the assumption that the context of the seed concept is similar to that of the candidate concept, this application, in addition to the information of the concept itself, also calculates the context relevance between the seed concept and the candidate concept to achieve the purpose of fine-tuning the weight of the candidate concept. Therefore, this application proposes a method for concept expansion using academic paper corpus for leaf research fields on a tree-like scientific and technological concept map.

[0077] From the above content, it can be seen that the solution framework of this application mainly includes three parts: obtaining seed concepts, generating candidate concepts, and fine-tuning concept weights. Obtaining seed concepts is to preprocess the input corpus and obtain high-quality self-filled keywords as seed concepts to guide subsequent processes. Generating candidate concepts is to extract keywords related to specific subjects in the text as candidate concepts to ensure the recall rate. Fine-tuning concept weights is to use the concept itself and context features to adjust the weights and sort them to improve accuracy.

[0078] Optionally, in one embodiment of the present application, screening the candidate seed concept set includes:

[0079] After removing duplicates from the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated;

[0080] The score of each candidate seed concept is calculated based on semantic similarity and string similarity, and all candidate seed concepts are sorted and screened according to the score to obtain the target seed concept.

[0081] Optionally, in one embodiment of the present application, a keyword extraction algorithm is used to generate a second candidate phrase set, including:

[0082] Based on the target document, use the word segmenter to segment the title and abstract of each document in the input corpus in advance;

[0083] Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set, where the topN method is to calculate the TF-IDF value of the phrases contained in each document in the target document set, and select the first n as the candidate phrases for a single document. Each document is processed once according to the topN method to obtain the second candidate phrase set;

[0084] Taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set;

[0085] Screen the candidate concept set, including:

[0086] Calculate the score of each candidate concept in the candidate concept set, and retain the concepts with scores greater than a fixed threshold as the final target candidate concepts, where the score of each candidate concept is:

[0087]

[0088] Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

[0089] Optionally, in one embodiment of the present application, the weight of the target candidate concept is adjusted using the target seed concept, including:

[0090] According to the target candidate concept and the target seed concept, find the corresponding context in the target document abstract where they appear;

[0091] Calculate the semantic relevance of each target candidate concept with all target seed concepts, and the semantic relevance of the context of each target candidate concept with all target seed concepts;

[0092] The two scores are combined as the fine-tuned weights of the target candidate concepts, and all target candidate concepts with fine-tuned weights constitute the target concept set.

[0093] In this application, the tree-like technology concept map can be formally defined as T = (D, R), where D = {v1, v2, ..., v |D|} represents a node set, R = { <v i ,v j >|v i ,v j ∈D} represents the set of edges between nodes. The leaf nodes are represented as The academic paper corpus can be formally defined as P = {p M ,p2,…,p |P|}, p i Represents the i-th paper, including the paper title t i 、Abstract a i and thesis self-filled keyword set k i . Concept set C = {C1, C2, ..., C |C|}, C i Represents a leaf node All related concepts. The main framework process of concept expansion is as follows Figure 2 shown.

[0094] Example sub-graphs of the science and technology concept map are as follows Figure 3 As shown, it includes the subject root node v0, the first-level subject nodes v1 and v2, and the leaf research direction nodes v3, v4 and v5. The following takes this subgraph as an example to introduce in detail the method for concept expansion of the science and technology concept map proposed in this application.

[0095] Step 1: Get the seed concept

[0096] Given research direction v3, in order to obtain high-quality and popular literature, this application uses elasticsearch to perform full-text search in the paper database, and defines the rule to only retain paper data from the past five years and with more than 100 citations to complete the filtering, and take the first third of the search results as the paper collection corresponding to v3 Indicates the number of papers. Then select the self-filled keywords with a length greater than 2 in each paper as the candidate seed concept set k i Represents the set of self-filled keywords in the i-th paper.

[0097] Assume that The set of all seed concepts after duplication removal is Indicates the number of candidate seed concepts after deduplication. Concepts related to a specific research direction are expressed in string form and word meaning, so this application first uses the Levenshtein ratio to calculate u i String similarity between .

[0098]

[0099] sum represents u i and the total length of the string of v3, Idlist represents the class edit distance. Then, Sentence-BERT is used to map the candidate seed concepts and research directions to a vector space of fixed dimensions, and then the candidate seed concept u is calculated by the following formula i Semantic similarity with v3.

[0100]

[0101] Indicates u i The semantic vector of represents the semantic vector of v3, d is the vector dimension, which is set to 512. Finally, the linear combination of the two is used as u i The final score.

[0102] score(u i )=0.1×levab(u i ,v3)+sim(u i ,v3)

[0103] Repeat the above steps for research directions v4 and v5 to obtain the corresponding deduplicated candidate seed concept sets. and In order to ensure the diversity of seed concepts between sibling nodes and reduce the scores of frequently appearing candidate seed concepts, this application uses the following method for adjustment.

[0104]

[0105] N is the number of sibling nodes, count(u i ) refers to u i In the corresponding sibling node and The frequency of occurrence in .

[0106] This step is based on and And the corresponding concept scores are sorted in descending order, and the top 30% are taken as the seed concepts of v3, v4 and v5. and

[0107] Step 2 Generate candidate concepts

[0108] The number of seed concepts is small, and further mining of other concepts in the paper is needed. Domain phrase mining generates candidate 0 phrases based on the N-Gram method, and this application also adopts a similar method. q i Represents the vocabulary sequence of the i-th sentence x i represents the i-th word, |q i | represents the sequence length. Assuming that the algorithm complexity of N-Gram is o(n!), the number of words in the dictionary corresponding to n is L(n), and this application determines the value of n by maximizing the following objective function.

[0109]

[0110] Where α is a fixed scaling value. i Perform full permutation and generate the first part of the candidate concepts under the condition that the maximum length of the phrase does not exceed 6 and appears in the predefined vocabulary calculate Semantic similarity with v3

[0111] The N-Gram-based approach is limited by fixed length and cannot effectively distinguish good phrases from bad ones. This application is based on all paper data, uses a word segmenter to segment the titles and abstracts in the papers in advance, and counts the TF-IDF values ​​of the words in each paper. This application retains the M words with the highest TF-IDF values ​​in the paper as the second part of the candidate concepts This application sets different M on 100 papers, conducts keyword extraction experiments, and statistically analyzes the accuracy of the results to determine the value of M. Then the two parts of candidate concepts are merged to obtain Candidate concept c i The score is calculated as follows.

[0112]

[0113] tf-idf(c i ) means c i TF-IDF value in the papers in which it appears.

[0114] In this step, only score(c i ) is greater than a fixed threshold, repeat the above operation to generate candidate concepts v3, v4 and v5 and

[0115] Step 3: Fine-tune concept weights

[0116] Step 2 generates a large number of candidate concepts, and thus introduces some noise data that is irrelevant to the research field. Therefore, this step uses seed concepts to adjust the weights of candidate concepts.

[0117] After completing step 1, you will get a collection of papers for research direction v3. and seed concept After completing step 2, we get the candidate concept set of v3

[0118] This application first calculates the candidate concept c by the degree of association between the candidate concept and all seed concepts i The score is as shown in the following formula.

[0119]

[0120] Indicates c i The semantic vector of Represents the seed concept s j The semantic vector of . refer to The number of seed concepts in .

[0121] Under the assumption that concepts with similar semantics also have similar contexts, this application Find the characters in the fixed window length of 30 as s j and c i Contextual data and in and Respectively represent the position index of its appearance in the paper. Then calculate the consistency of the context of the candidate concept with the context of all seed concepts.

[0122]

[0123] express The semantic vector of express The semantic vector of .

[0124] If a candidate concept appears in multiple sister research fields, it indicates that the concept is relatively general, and the goal of this application is to expand the field concepts closely related to the research topic, so the weight of the candidate concept needs to be penalized.

[0125]

[0126]

[0127] Where N is the number of sibling nodes. Then the linear combination of the two is used as c i Weights after fine-tuning.

[0128]

[0129] This step repeats the above operation for the candidate concepts of v3, v4 and v5 and The weights are adjusted and screened. This application combines the seed concepts in a specific research field with the screened candidate concepts as the final concept set.

[0130] In order to implement the above-mentioned embodiments, the present application also proposes a device for conceptually expanding a scientific and technological concept map.

[0131] Figure 4 A schematic diagram of the structure of a device for expanding the concepts of a scientific and technological concept map provided in an embodiment of the present application.

[0132] like Figure 4 As shown, the device for expanding the concept of the scientific and technological concept map includes a target seed concept acquisition module, a target candidate concept generation module, and a concept weight fine-tuning module, wherein:

[0133] Get target seed concept module, used for:

[0134] Use full-text search and rule filtering to query the documents associated with the node to be expanded in the input corpus, and select a predetermined proportion of documents as target documents according to the display order of the queried documents;

[0135] Select self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set;

[0136] Screen the candidate seed concept set to obtain the target seed concept;

[0137] Generate target candidate concept module for:

[0138] Based on the target document, a first candidate phrase set is generated using N-Gram, and a second candidate phrase set is generated using a keyword extraction algorithm, and then a candidate concept set is generated based on the first candidate phrase set and the second candidate phrase set;

[0139] Screen the candidate concept set to obtain the target candidate concept;

[0140] The fine-tuning concept weight module is used to adjust the weights of the target candidate concepts using the target seed concepts to generate a target concept set.

[0141] Optionally, in one embodiment of the present application, screening the candidate seed concept set includes:

[0142] After removing duplicates from the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated;

[0143] The score of each candidate seed concept is calculated based on semantic similarity and string similarity, and all candidate seed concepts are sorted and screened according to the score to obtain the target seed concept.

[0144] Optionally, in one embodiment of the present application, a keyword extraction algorithm is used to generate a first candidate phrase set, including:

[0145] Based on the target document, use the word segmenter to segment the title and abstract of each document in the input corpus in advance;

[0146] Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set;

[0147] Taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set;

[0148] Screen the candidate concept set, including:

[0149] Calculate the score of each candidate concept in the candidate concept set, and retain the concepts with scores greater than a fixed threshold as the final target candidate concepts.

[0150] Among them, the TOPN method is used to sort and filter all the words after word segmentation to obtain the second candidate phrase set, including:

[0151] Calculate the TF-IDF value of the phrases contained in each document in the target document, and select candidate phrases for a single document based on the TF-IDF value;

[0152] Constructing a second candidate phrase set based on candidate phrases of all documents in the target document;

[0153] The score for each candidate concept is:

[0154]

[0155] Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

[0156] Optionally, in one embodiment of the present application, fine-tuning the concept weight module is specifically used to:

[0157] According to the target candidate concept and the target seed concept, find the corresponding context in the target document abstract where they appear;

[0158] Calculate the semantic relevance of each target candidate concept with all target seed concepts, and the semantic relevance of the context of each target candidate concept with all target seed concepts;

[0159] The two scores are combined as the fine-tuned weights of the target candidate concepts, and all target candidate concepts with fine-tuned weights constitute the target concept set.

[0160] It should be noted that the above explanation of the embodiment of the method for expanding the concept of the scientific and technological concept map is also applicable to the device for expanding the concept of the scientific and technological concept map in this embodiment, and will not be repeated here.

[0161] In order to implement the above embodiments, the present invention further proposes a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method of the above embodiments is implemented.

[0162] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0163] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0164] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0165] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0166] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0167] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0169] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for concept expansion of a scientific and technological concept map, characterized in that: The following steps are involved: Use full-text search and rule filtering to query the documents associated with the node to be expanded in the input corpus, and select a predetermined proportion of documents as target documents according to the display order of the queried documents; Selecting self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set; Screening the candidate seed concept set to obtain a target seed concept; Based on the target document, a first candidate phrase set is generated using N-Gram, and a second candidate phrase set is generated using a keyword extraction algorithm, and then a candidate concept set is generated according to the first candidate phrase set and the second candidate phrase set; Screening the candidate concept set to obtain a target candidate concept; Using the target seed concept to adjust the weight of the target candidate concept to generate a target concept set; The step of screening the candidate seed concept set includes: After deduplication of the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated; Calculating a score for each of the candidate seed concepts according to the semantic similarity and the string similarity, and sorting and screening all the candidate seed concepts according to the score to obtain a target seed concept; The screening of the candidate concept set includes: Calculating the score of each candidate concept in the candidate concept set, and retaining the concepts whose scores are greater than a fixed threshold as final target candidate concepts; The step of adjusting the weight of the target candidate concept using the target seed concept includes: Searching for corresponding contexts in the target document abstracts where the target candidate concepts and the target seed concepts appear; Calculating the semantic relevance between each of the target candidate concepts and all of the target seed concepts, and the semantic relevance between the contexts of each of the target candidate concepts and all of the target seed concepts; The two scores are combined to serve as the fine-tuned weight of the target candidate concept, and all target candidate concepts with the fine-tuned weight constitute the target concept set.

2. The method according to claim 1, characterized in that The step of using a keyword extraction algorithm to generate the second candidate phrase set includes: Based on the target document, a word segmenter is used in advance to segment the title and abstract of each document in the input corpus; Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set; taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set; The method of using TOPN to sort and filter all the words after word segmentation to obtain a second candidate phrase set includes: Calculating the TF-IDF value of the phrases contained in each document in the target document, and selecting candidate phrases for the single document according to the TF-IDF value; Constructing the second candidate phrase set based on candidate phrases of all documents in the target document; The score of each candidate concept is: Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

3. A device for expanding the concept of a scientific and technological concept map, characterized in that: It includes a target seed concept acquisition module, a target candidate concept generation module, and a concept weight fine-tuning module, among which: The target seed concept acquisition module is used to: Use full-text search and rule filtering to query the documents associated with the node to be expanded in the input corpus, and select a predetermined proportion of documents as target documents according to the display order of the queried documents; Selecting self-filled keywords in the target document whose length is greater than a preset threshold as a candidate seed concept set; Screening the candidate seed concept set to obtain a target seed concept; The module for generating target candidate concepts is used to: Based on the target document, a first candidate phrase set is generated using N-Gram, and a second candidate phrase set is generated using a keyword extraction algorithm, and then a candidate concept set is generated according to the first candidate phrase set and the second candidate phrase set; Screening the candidate concept set to obtain a target candidate concept; The fine-tuning concept weight module is used to adjust the weight of the target candidate concept using the target seed concept to generate a target concept set; The step of screening the candidate seed concept set includes: After deduplication of the candidate seed concept set, the semantic similarity and string similarity between each candidate seed concept in the set and the node to be expanded are calculated; Calculating a score for each of the candidate seed concepts according to the semantic similarity and the string similarity, and sorting and screening all the candidate seed concepts according to the score to obtain a target seed concept; The screening of the candidate concept set includes: Calculating the score of each candidate concept in the candidate concept set, and retaining the concepts whose scores are greater than a fixed threshold as final target candidate concepts; The fine-tuning concept weight module is specifically used for: Searching for corresponding contexts in the target document abstracts where the target candidate concepts and the target seed concepts appear; Calculating the semantic relevance between each of the target candidate concepts and all of the target seed concepts, and the semantic relevance between the contexts of each of the target candidate concepts and all of the target seed concepts; The two scores are combined to serve as the fine-tuned weight of the target candidate concept, and all target candidate concepts with the fine-tuned weight constitute the target concept set.

4. The device according to claim 3, characterized in that The step of using a keyword extraction algorithm to generate a first candidate phrase set includes: Based on the target document, a word segmenter is used in advance to segment the title and abstract of each document in the input corpus; Use the TOPN method to sort and filter all the words after word segmentation to obtain the second candidate phrase set; taking the union of the first candidate phrase set and the second candidate phrase set as the candidate concept set; The method of using TOPN to sort and filter all the words after word segmentation to obtain a second candidate phrase set includes: Calculating the TF-IDF value of the phrases contained in each document in the target document, and selecting candidate phrases for the single document according to the TF-IDF value; Constructing the second candidate phrase set based on candidate phrases of all documents in the target document; The score of each candidate concept is: Among them, score(c i ) is the candidate concept c i The score, sim(c i ,v3) is the candidate concept c i The semantic similarity between the node v3 to be expanded, is the first candidate concept set, is the second candidate concept set, tf-idf(c i ) represents the candidate concept c i The TF-IDF value in the documents in which it appears.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images