Subject domain division method based on thesis keywords, semantic co-occurrence and centrality index

Abstract

The invention discloses a subject domain division method based on paper keywords and semantic co-occurrence and centrality indexes, which comprises the following steps of: retrieving keywords of a given subject paper based on a big data technology, and merging the same keywords to form a subject keyword set; calculating the co-occurrence frequency of the paper keywords based on the subject keyword set, performing statistics on the co-occurrence frequency of the paper keywords to obtain a statistical result, and constructing a co-occurrence undirected graph based on the statistical result; calculating the paper keyword centrality according to the co-occurrence undirected graph, and determining paper core words based on the paper keyword centrality; obtaining keyword diversity sets of different core words according to the paper core words; performing fine division on the subject field based on the keyword diversity set to obtain a division result; and through thesis keyword co-occurrence and centrality indexes, field subdivision and division of subjects are carried out, so that subject evaluation and talent evaluation in each field can be accurately benchmarked.

Description

Technical Field

[0001] The present invention relates to the technical field of subject division, and more specifically, to a subject field division method based on paper keywords and semantic co-occurrence and centrality indicators. Background Art

[0002] With the development of society and science and technology, disciplinary talents have also become the focus of people's attention. With so many disciplines, how to evaluate disciplines and disciplinary talents is a major challenge. It has become a problem worth studying to conduct disciplinary evaluation based on the subdivision of disciplines. Similarly, it is very important to determine the influence and technical level of a disciplinary talent in a specific subdivision based on his previous scientific research output.

[0003] At present, the mainstream method for the problem of field segmentation and talent field affiliation at home and abroad is manual judgment based on expert advice. Although this method is worthy of recognition in terms of accuracy and professionalism, it has problems such as each person has his own standards, which makes it not objective enough, manual judgment cannot be quantified and has difficulty in having unified specific standards, and manual review progresses slowly. Therefore, how to develop a method for subject field segmentation with unified standards, objective evaluation, and faster speed is the current main research direction.

[0004] With the development of computer technology and big data technology, many tasks can be realized by algorithms. Algorithms are objective and fast. Algorithms are used to form methods to achieve the subdivision of subject areas, obtain appropriate, reasonable and accurate subject subdivision areas, and solve the problems of striving for humanity and refinement in subject evaluation and talent evaluation. Summary of the invention

[0005] In order to solve at least one of the above technical problems, the present invention proposes a subject field division method based on paper keywords and semantic co-occurrence and centrality indicators.

[0006] The first aspect of the present invention provides a method for dividing subject areas based on paper keywords and semantic co-occurrence and centrality indicators, comprising the following steps:

[0007] Retrieve keywords of papers in a given discipline based on big data technology, merge the same keywords, and form a discipline keyword set;

[0008] Calculate the co-occurrence frequency of the paper keywords based on the subject keyword set, count the co-occurrence frequency of the paper keywords, obtain the statistical results, and construct a co-occurrence undirected graph based on the statistical results;

[0009] Calculate the keyword centrality of the paper based on the co-occurrence undirected graph, and determine the core words of the paper based on the keyword centrality of the paper;

[0010] Get keyword subsets of different core words based on the core words of the paper;

[0011] Based on the keyword subsets, the subject areas are finely divided to obtain the division results.

[0012] In a preferred embodiment of the present invention, the co-occurrence frequency of the paper keywords is calculated based on the subject keyword set, the co-occurrence frequency of the paper keywords is counted to obtain statistical results, and a co-occurrence undirected graph is constructed based on the statistical results, which specifically includes:

[0013] Obtain data on M papers and N keywords, and retrieve the frequency of each two keywords appearing in a paper at the same time based on big data technology to obtain the keyword co-occurrence frequency;

[0014] With the paper keywords as nodes, co-occurrence as connecting edges, and the co-occurrence frequency as the weight of the connecting edge, an undirected graph G = (V, E) is constructed, where V is the keyword set, that is, all nodes of the graph V = {v1, v2, ..., v N};

[0015] For any two nodes v in V i and v j , define k ij The weight of the edge connecting them;

[0016] There is an edge connecting two nodes v i and v j , k ij >0, and k ij =k ji , indicating the frequency of co-occurrence of two keywords;

[0017] For two nodes k that are not connected by an edge i and k j , k ij =k ji =0, indicating that the two keywords have never co-occurred.

[0018] In a preferred embodiment of the present invention, the paper keyword centrality includes betweenness centrality, neighbor centrality, eigenvector centrality and degree centrality.

[0019] In a preferred embodiment of the present invention, the centrality of paper keywords is calculated based on the co-occurrence undirected graph, specifically including:

[0020] Obtain the co-occurrence undirected graph and obtain several nodes corresponding to the paper keywords. The calculation formula for the betweenness centrality of node i is:

[0021]

[0022] In the formula, p j，k (i) is the number of shortest paths between nodes j and k that pass through node i, pjk is the number of shortest paths between nodes j and k;

[0023] The calculation formula of the neighboring centrality of node i is:

[0024]

[0025] Where N is the number of nodes in the network, that is, the number of keywords; d(i, j) is the shortest path between nodes i and j;

[0026] The calculation formula of the eigenvector centrality of node i is:

[0027]

[0028] In the formula, a ij =1 means if and only if nodes i and j are connected, otherwise a ij =1=0,x j is the importance measure of node i, and is the weight of the edge;

[0029] The degree centrality calculation formula of node i is:

[0030] C D (i) = d i

[0031] Where, d i is the number of edges connected to node i.

[0032] In a preferred embodiment of the present invention, determining the core words of the paper based on the centrality of the paper keywords specifically includes:

[0033] Get betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality;

[0034] Sort the keywords in descending order according to betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality to obtain the sorting results;

[0035] Based on the sorting results, the keywords that meet the top ten percent of betweenness centrality, proximity centrality, eigenvector centrality and degree centrality are taken as core words, where the number of core words is N*10%.

[0036] In a preferred embodiment of the present invention, keyword subsets of different core words are obtained according to the core words of the paper, specifically including:

[0037] The keywords co-occurring with each core word are ranked by frequency, and the top 10% of the co-occurring keywords with each core word are selected to obtain a subset of keywords associated with the core word.

[0038] In a preferred embodiment of the present invention, the subject field is finely divided based on the keyword diversity set to obtain the division result, which specifically includes:

[0039] After obtaining the core words of the paper, the keyword clusters associated with the core words are determined by sorting the co-occurrence frequencies of different core words, and the keyword clusters of different core words are analyzed for similarity, and the keyword clusters greater than or equal to the set similarity threshold are merged; the details are as follows:

[0040] Define a1, a2, …, a s are keywords related to the core word a, which is a set A; b1, b2, ..., b t are the keywords related to the core word b, recorded as set B; C = A∩B = [c1, c2, ..., c k ]; then the similarity of sets A and B relative to A is

[0041]

[0042] When S(A)≥80%, A is merged into B, and core word a is classified as the keyword of core word b; and so on, a new core word and the corresponding keyword set are obtained;

[0043] Arrange the keyword subsets corresponding to the core word set to obtain the subdivided fields corresponding to the keyword subsets centered on the core words of the paper.

[0044] Compared with the prior art, the beneficial technical effects obtained by the present invention are:

[0045] Through the co-occurrence of paper keywords and centrality indicators, the disciplines are subdivided into different fields, which can be used to accurately conduct discipline assessment and talent evaluation in each field. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The flowchart of the subject field division method based on paper keywords and semantic co-occurrence and centrality indicators of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0049] like Figure 1As shown, the first aspect of the present invention provides a method for dividing subject areas based on paper keywords and semantic co-occurrence and centrality indicators, comprising the following steps:

[0050] S1, based on big data technology, retrieve keywords of papers in a given discipline, merge the same keywords, and form a discipline keyword set;

[0051] Specifically: determine a certain subject, use big data technology to retrieve all publicly published papers in the subject, obtain the keywords of each paper, merge the same keywords, and obtain data on M papers and N keywords.

[0052] S2, calculate the co-occurrence frequency of the paper keywords based on the subject keyword set, count the co-occurrence frequency of the paper keywords, obtain the statistical results, and construct a co-occurrence undirected graph based on the statistical results;

[0053] S3, calculate the paper keyword centrality according to the co-occurrence undirected graph, and determine the paper core words based on the paper keyword centrality;

[0054] S4, obtain keyword subsets of different core words based on the core words of the paper;

[0055] S5, finely divide the subject areas based on keyword sets to obtain the division results.

[0056] According to an embodiment of the present invention, the co-occurrence frequency of the paper keywords is calculated based on the subject keyword set, the co-occurrence frequency of the paper keywords is counted to obtain statistical results, and a co-occurrence undirected graph is constructed based on the statistical results, specifically including:

[0057] Obtain data on M papers and N keywords, and retrieve the frequency of each two keywords appearing in a paper at the same time based on big data technology to obtain the keyword co-occurrence frequency;

[0058] With the paper keywords as nodes, co-occurrence as connecting edges, and the co-occurrence frequency as the weight of the connecting edge, an undirected graph G = (V, E) is constructed, where V is the keyword set, that is, all nodes of the graph V = {v1, v2, ..., v N};

[0059] For any two nodes v in V i and v j , define k ij The weight of the edge connecting them;

[0060] There is an edge connecting two nodes v i and v j , k ij >0, and k ij =k ji , indicating the frequency of co-occurrence of two keywords;

[0061] For two nodes k that are not connected by an edge i and k j , k ij =k ji =0, indicating that the two keywords have never co-occurred.

[0062] According to an embodiment of the present invention, the paper keyword centrality includes betweenness centrality, neighbor centrality, eigenvector centrality and degree centrality.

[0063] According to an embodiment of the present invention, calculating the centrality of paper keywords based on the co-occurrence undirected graph specifically includes:

[0064] Obtain the co-occurrence undirected graph and obtain several nodes corresponding to the paper keywords. The calculation formula of the betweenness centrality of node is:

[0065]

[0066] In the formula, p j，k (i) is the number of shortest paths between nodes j and k that pass through node i, p jk is the number of shortest paths between nodes j and k;

[0067] The calculation formula of the neighboring centrality of node i is:

[0068]

[0069] Where N is the number of nodes in the network, that is, the number of keywords; d(i, j) is the shortest path between nodes i and j;

[0070] The calculation formula of the eigenvector centrality of node i is:

[0071]

[0072] In the formula, a ij =1 means if and only if nodes i and j are connected, otherwise a ij =1=0,x j is the importance measure of node i, and is the weight of the edge;

[0073] The degree centrality calculation formula of node i is:

[0074] C D (i) = d i

[0075] Where, d i is the number of edges connected to node i.

[0076] According to an embodiment of the present invention, determining the core words of the paper based on the centrality of the paper keywords specifically includes:

[0077] Get betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality;

[0078] Sort the keywords in descending order according to betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality to obtain the sorting results;

[0079] Based on the sorting results, the keywords that meet the top ten percent of betweenness centrality, proximity centrality, eigenvector centrality and degree centrality are taken as core words, where the number of core words is N*10%.

[0080] According to an embodiment of the present invention, keyword subsets of different core words are obtained according to the core words of the paper, specifically including:

[0081] The keywords co-occurring with each core word are ranked by frequency, and the top 10% of the co-occurring keywords with each core word are selected to obtain a subset of keywords associated with the core word.

[0082] According to an embodiment of the present invention, the subject field is finely divided based on the keyword diversity set to obtain the division result, which specifically includes:

[0083] After obtaining the core words of the paper, the keyword clusters associated with the core words are determined by sorting the co-occurrence frequencies of different core words, and the keyword clusters of different core words are analyzed for similarity, and the keyword clusters greater than or equal to the set similarity threshold are merged; the details are as follows:

[0084] Define a1, a2, …, a s are keywords related to the core word a, which is a set A; b1, b2, ..., b t are the keywords related to the core word b, recorded as set B; C = A∩B = [c1, c2, ..., c k ]; then the similarity of sets A and B relative to A is

[0085]

[0086] When S(A)≥80%, A is merged into B, and core word a is classified as the keyword of core word b; and so on, a new core word and the corresponding keyword set are obtained;

[0087] Arrange the keyword subsets corresponding to the core word set to obtain the subdivided fields corresponding to the keyword subsets centered on the core words of the paper.

[0088] In summary, through the co-occurrence of paper keywords and centrality indicators, the disciplines are subdivided and divided into different fields, which can be used to accurately benchmark the discipline assessment and talent evaluation in each field.

[0089] The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0090] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0092] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for dividing subject areas based on paper keywords and semantic co-occurrence and centrality indicators, characterized in that: The steps include: Retrieve keywords of papers in a given discipline based on big data technology, merge the same keywords, and form a discipline keyword set; Calculate the co-occurrence frequency of the paper keywords based on the subject keyword set, count the co-occurrence frequency of the paper keywords, obtain the statistical results, and construct a co-occurrence undirected graph based on the statistical results; Calculate the keyword centrality of the paper based on the co-occurrence undirected graph, and determine the core words of the paper based on the keyword centrality of the paper; Get keyword subsets of different core words based on the core words of the paper; Based on the keyword subsets, the subject areas are finely divided to obtain the division results.

2. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 1 is characterized in that: The co-occurrence frequency of the paper keywords is calculated based on the subject keyword set, and the co-occurrence frequency of the paper keywords is counted to obtain the statistical results. Based on the statistical results, a co-occurrence undirected graph is constructed, which specifically includes: Obtain data on M papers and N keywords, and retrieve the frequency of each two keywords appearing in a paper at the same time based on big data technology to obtain the keyword co-occurrence frequency; With the paper keywords as nodes, co-occurrence as connecting edges, and the co-occurrence frequency as the weight of the connecting edge, an undirected graph G = (V, E) is constructed, where V is the keyword set, that is, all nodes of the graph V = {v1, v2, ..., v N }; For any two nodes v in V i and v j , define k ij The weight of the edge connecting them; There is an edge connecting two nodes v i and v j , k ij >0, and k ij =k ji , indicating the frequency of co-occurrence of two keywords; For two nodes k that are not connected by an edge i and k j , k ij =k ji =0, indicating that the two keywords have never co-occurred.

3. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 2 is characterized in that: The paper keyword centrality includes betweenness centrality, neighbor centrality, eigenvector centrality and degree centrality.

4. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 3 is characterized in that: The centrality of paper keywords is calculated based on the co-occurrence undirected graph, including: Obtain the co-occurrence undirected graph and obtain several nodes corresponding to the paper keywords. The calculation formula for the betweenness centrality of node i is: In the formula, p j，k (i) is the number of shortest paths between nodes j and k that pass through node i, p jk is the number of shortest paths between nodes j and k; The calculation formula of the neighboring centrality of node i is: Where N is the number of nodes in the network, that is, the number of keywords; d(i, j) is the shortest path between nodes i and j; The calculation formula of the eigenvector centrality of node i is: In the formula, a ij =1 means if and only if nodes i and j are connected, otherwise a ij =1=0,x j is the importance measure of node i, and is the weight of the edge; The degree centrality calculation formula of node i is: C D (i)=d i Where, d i is the number of edges connected to node i.

5. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 4 is characterized in that: Determine the core words of the paper based on the centrality of the paper keywords, including: Get betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality; Sort the keywords in descending order according to betweenness centrality, neighbor centrality, eigenvector centrality, and degree centrality to obtain the sorting results; Based on the sorting results, the keywords that meet the top ten percent of betweenness centrality, proximity centrality, eigenvector centrality and degree centrality are taken as core words, where the number of core words is N*10%.

6. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 3 is characterized in that: According to the core words of the paper, we get keyword subsets of different core words, including: The keywords co-occurring with each core word are ranked by frequency, and the top 10% of the co-occurring keywords with each core word are selected to obtain a subset of keywords associated with the core word.

7. The subject field division method based on paper keywords and semantic co-occurrence and centrality index according to claim 1 is characterized in that: Based on the keyword subsets, the subject areas are finely divided and the division results are obtained, including: After obtaining the core words of the paper, the keyword clusters associated with the core words are determined by sorting the co-occurrence frequencies of different core words, and the keyword clusters of different core words are analyzed for similarity, and the keyword clusters greater than or equal to the set similarity threshold are merged; the details are as follows: Define a1, a2, …, a s are keywords related to the core word a, which is a set A; b1, b2, ..., b t are the keywords related to the core word b, recorded as set B; C = A∩B = [c1, c2, ..., c k ]; then the similarity of sets A and B relative to A is When S(A)≥80%, A is merged into B, and core word a is classified as the keyword of core word b; and so on, a new core word and the corresponding keyword set are obtained; Arrange the keyword subsets corresponding to the core word set to obtain the subdivided fields corresponding to the keyword subsets centered on the core words of the paper.

Images