Subject subdivision field division method based on thesis keyword spectral clustering

Abstract

The invention discloses a discipline subdivision field division method based on thesis keyword spectral clustering, which comprises the following steps: retrieving all publicly published theses of a discipline based on a big data technology, obtaining keywords of each thesis, and merging the same keywords to obtain merged keywords; constructing a spectrogram in which every two paper keywords appear on the basis of the combined keywords; carrying out graph segmentation based on a spectral clustering algorithm, and carrying out clustering division on paper keywords to obtain clusters of all keywords; performing subject subdivision field division based on the clustering of all keywords to obtain a division result; according to the method, graph segmentation is performed through a spectral clustering algorithm, clustering division of paper keywords is performed, and keyword classification is formed, so that different subdivision fields of a subject are formed, subject evaluation in the subject subdivision fields is realized, and the requirements of subject evaluation and talent evaluation on fineness and accuracy are met.

Description

Technical Field

[0001] The present invention relates to the technical field of subject division, and more specifically, to a subject subdivision field division method based on paper keyword spectrum clustering. Background Art

[0002] With the development of big data technology and artificial intelligence technology, third-party evaluation provides a good reference for the society to understand the domestic and foreign disciplines and professional conditions, and is increasingly valued by the government, universities and enterprises. However, in the current discipline classification, most disciplines have a wide coverage. For example, the Springer system abroad divides journals and books into 13 disciplines: architecture and design, behavioral sciences, biomedicine and life sciences, business and economics, chemistry and materials science, computer science, earth and environmental sciences, engineering, humanities, social sciences and law, mathematics and statistics, medicine, physics and astronomy, computer vocational technology and professional computer applications. In China, disciplines are divided into 14 disciplines, including philosophy, economics, law, education, literature, history, science, engineering, agriculture, medicine, military science, management, art, and interdisciplinary disciplines, with a total of 113 first-level disciplines; these "disciplines" are basically set up according to the classification of graduate students and degree authorization, and are universal and broad, and appear to be very broad in specific discipline evaluation and talent evaluation. For example, in the category of science, there are first-level disciplines such as mathematics, physics, chemistry, astronomy, geography, atmospheric science, ocean science, geophysics, and geology. In fact, the scope of disciplines such as mathematics and physics is very broad. For example, a scholar is very outstanding in the research of "Hall quantum effect", but compared with the whole physics, it may not be very outstanding; similarly, a university has outstanding research results in "Hall quantum effect", but it may not be very good in the evaluation of physics disciplines. This situation has brought great limitations to the evaluation of disciplines and talents, especially in the introduction of talents, which often requires special talents in a certain aspect of the discipline. In this way, it is particularly necessary to divide the discipline into sub-fields.

[0003] When determining the academic level and technical ability of the evaluated talents in the discipline, if we compare them from the entire discipline, there are so many talents that it is difficult to reflect their characteristics and find talents who truly have special research capabilities in a certain research direction of the discipline. Summary of the invention

[0004] In order to solve at least one of the above technical problems, the present invention proposes a method for dividing subject subdivision fields based on paper keyword spectrum clustering.

[0005] The first aspect of the present invention provides a method for dividing subject subdivision fields based on paper keyword spectrum clustering, comprising the following steps:

[0006] Retrieve all publicly published papers on the subject based on big data technology, obtain the keywords of each paper, merge the same keywords, and obtain the merged keywords;

[0007] Based on the merged keywords, a spectrum of two-by-two occurrences of the paper keywords is constructed;

[0008] Based on the spectral clustering algorithm, the graph is segmented and the paper keywords are clustered to obtain the clustering of all keywords;

[0009] Based on the clustering of all keywords, the subject subdivision is carried out to obtain the division results.

[0010] In a preferred embodiment of the present invention, the keywords of each paper are obtained, and the same keywords are merged to obtain the merged keywords, which specifically includes:

[0011] Based on big data technology, all keywords of publicly published papers in the specified discipline are retrieved, and statistical results are obtained when two keywords appear in the same paper at the same time;

[0012] Based on the statistical results, the same keywords are screened out, and the same keywords are merged to obtain merged keywords.

[0013] In a preferred embodiment of the present invention, a spectrum of paper keywords appearing in pairs is constructed based on the merged keywords, specifically including:

[0014] Get data of M papers and N keywords;

[0015] Based on big data technology, the frequency of each two keywords appearing in a paper is retrieved;

[0016] With keywords as nodes and their two-by-two appearances as connection relationships, a spectrum of two-by-two keyword appearances is constructed.

[0017] In a preferred embodiment of the present invention, the graph segmentation is performed based on the spectral clustering algorithm, and the clustering of the paper keywords is performed to obtain the clustering of all keywords, which specifically includes:

[0018] Based on the spectral clustering method, an undirected graph is constructed with keywords as nodes, pairs of keywords as connecting edges, and the number of pairs of keywords as the weight of the connecting edge. The degree of any node is calculated to form the degree matrix, adjacency matrix, and Laplace matrix. The matrix is ​​optimized to obtain the feature matrix.

[0019] Clustering is performed based on the K-means clustering method to obtain clusters of all keywords.

[0020] In a preferred embodiment of the present invention, the method for analyzing the number of categories in the clustering process is as follows:

[0021] After retrieving the frequency of each two keywords appearing in a paper based on big data technology, the 10% keywords with the highest frequency ranking are selected as core keywords to obtain the core keyword values;

[0022] The value of the core keyword is used as the number of categories for classification.

[0023] In a preferred embodiment of the present invention, the graph segmentation is performed based on the spectral clustering algorithm to cluster the keywords of the paper, which specifically includes:

[0024] The keywords are nodes, the two-by-two occurrences are connected edges, and the frequency of two-by-two occurrences is the weight of the connecting edge. Thus, an undirected graph G = (K, E) is constructed, where K is the keyword set, i.e., all the nodes of the undirected graph, K = {k1, k2, ..., k N}; For any two points k in K i and k j , define w ij For the weights between them, there are w ij =w ji ; For two points k connected by an edge i and k j , w ij > 0, and indicates the frequency of two keywords appearing simultaneously. For two points k that are not connected by an edge i and k j , w ij =0, indicating that the two keywords have never appeared at the same time;

[0025] For any point k in the graph i , its degree d i is the sum of the weights of all edges connected to it, that is

[0026]

[0027] In the formula, n is the point k i The number of all connected edges;

[0028] The degree matrix D of all keywords N×N is as follows:

[0029]

[0030] Using the weight values ​​between all nodes, we get the N×N adjacency matrix W of all keywords. The formula is as follows:

[0031]

[0032] Calculate the Laplace matrix L, that is, L = DW, and construct the optimized H matrix, H = D -1 / 2 LD -1 / 2 ;

[0033] Solve the optimization matrix H and calculate H = D -1 / 2 LD -1 / 2 The eigenvectors f1, f2, ..., f corresponding to the smallest c eigenvalues c ;

[0034] The corresponding eigenvectors f1, f2, ..., f c The composed matrix is ​​normalized by row to form a feature matrix F of dimension N×c;

[0035] Each row in F is taken as a c-dimensional sample, with a total of N samples. K-means clustering method is used for clustering, and the clustering dimension is J. The cluster Z is obtained and the categories are divided into (z1, z2, ..., z J ).

[0036] Compared with the prior art, the beneficial technical effects obtained by the present invention are:

[0037] Through the spectral clustering algorithm, the graph is segmented and the paper keywords are clustered to form a classification of keywords, thereby forming different sub-fields of the discipline, so as to realize the discipline evaluation in the sub-fields of the discipline and meet the requirements of precision and accuracy of discipline evaluation and talent evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The flowchart of the subject subdivision field division method based on paper keyword spectrum clustering of the present invention is shown. DETAILED DESCRIPTION

[0039] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0041] like Figure 1 As shown, the first aspect of the present invention provides a method for dividing subject subdivision fields based on paper keyword spectrum clustering, comprising the following steps:

[0042] S1, based on big data technology, retrieve all publicly published papers in the subject, obtain the keywords of each paper, merge the same keywords, and obtain the merged keywords;

[0043] S2, constructs a spectrum of two-by-two occurrences of paper keywords based on the merged keywords;

[0044] S3, based on the spectral clustering algorithm, the graph is segmented and the paper keywords are clustered to obtain the clusters of all keywords;

[0045] S4, based on the clustering of all keywords, the subject subdivision is carried out to obtain the division results.

[0046] In a preferred embodiment of the present invention, the keywords of each paper are obtained, and the same keywords are merged to obtain the merged keywords, which specifically includes:

[0047] Based on big data technology, all keywords of publicly published papers in the specified discipline are retrieved, and statistical results are obtained when two keywords appear in the same paper at the same time;

[0048] Based on the statistical results, the same keywords are screened out, and the same keywords are merged to obtain merged keywords.

[0049] In a preferred embodiment of the present invention, a spectrum of paper keywords appearing in pairs is constructed based on the merged keywords, specifically including:

[0050] Get data of M papers and N keywords;

[0051] Based on big data technology, the frequency of each two keywords appearing in a paper is retrieved;

[0052] With keywords as nodes and their two-by-two appearances as connection relationships, a spectrum of two-by-two keyword appearances is constructed.

[0053] In a preferred embodiment of the present invention, the graph segmentation is performed based on the spectral clustering algorithm, and the clustering of the paper keywords is performed to obtain the clustering of all keywords, which specifically includes:

[0054] Based on the spectral clustering method, an undirected graph is constructed with keywords as nodes, pairs of keywords as connecting edges, and the number of pairs of keywords as the weight of the connecting edge. The degree of any node is calculated to form the degree matrix, adjacency matrix, and Laplace matrix. The matrix is ​​optimized to obtain the feature matrix.

[0055] Clustering is performed based on the K-means clustering method to obtain clusters of all keywords.

[0056] In a preferred embodiment of the present invention, the method for analyzing the number of categories in the clustering process is as follows:

[0057] After retrieving the frequency of each two keywords appearing in a paper based on big data technology, the 10% keywords with the highest frequency ranking are selected as core keywords to obtain the core keyword values;

[0058] The value of the core keyword is used as the number of categories for classification.

[0059] In a preferred embodiment of the present invention, the graph segmentation is performed based on the spectral clustering algorithm to cluster the keywords of the paper, which specifically includes:

[0060] The keywords are nodes, the two-by-two occurrences are connected edges, and the frequency of two-by-two occurrences is the weight of the connecting edge. Thus, an undirected graph G = (K, E) is constructed, where K is the keyword set, i.e., all the nodes of the undirected graph, K = {k1, k2, ..., k N}; For any two points k in K i and k j , define w ij For the weights between them, there are w ij =w ji ; For two points k connected by an edge i and k j , w ij > 0, and indicates the frequency of two keywords appearing simultaneously. For two points k that are not connected by an edge i and k j , w ij =0, indicating that the two keywords have never appeared at the same time;

[0061] For any point k in the graph i , its degree d i is the sum of the weights of all edges connected to it, that is

[0062]

[0063] In the formula, n is the point k i The number of all connected edges;

[0064] The degree matrix D of all keywords N×N is as follows:

[0065]

[0066] Using the weight values ​​between all nodes, we get the N×N adjacency matrix W of all keywords. The formula is as follows:

[0067]

[0068] Calculate the Laplace matrix L, that is, L = DW, and construct the optimized H matrix, H = D -1 / 2 LD -1 / 2 ;

[0069] Solve the optimization matrix H and calculate H = D -1 / 2 LD -1 / 2The eigenvectors f1, f2, ..., f corresponding to the smallest c eigenvalues c ;

[0070] The corresponding eigenvectors f1, f2, ..., f c The composed matrix is ​​normalized by row to form a feature matrix F of dimension N×c;

[0071] Each row in F is taken as a c-dimensional sample, with a total of N samples. K-means clustering method is used for clustering, and the clustering dimension is J. The cluster Z is obtained and the categories are divided into (z1, z2, ..., z J ).

[0072] In summary, the spectral clustering algorithm is used to perform graph segmentation, cluster the paper keywords, and form a classification of keywords, thereby forming different sub-fields of the discipline, so as to realize subject evaluation in the subject sub-fields and meet the requirements of precision and accuracy of subject evaluation and talent evaluation.

[0073] The device embodiments described above are only illustrative. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0074] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0075] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0076] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for subdividing disciplines based on paper keyword spectrum clustering, characterized in that: The steps include: Retrieve all publicly published papers on the subject based on big data technology, obtain the keywords of each paper, merge the same keywords, and obtain the merged keywords; Based on the merged keywords, a spectrum of two-by-two occurrences of the paper keywords is constructed; Based on the spectral clustering algorithm, the graph is segmented and the paper keywords are clustered to obtain the clustering of all keywords; Based on the clustering of all keywords, the subject subdivision is carried out to obtain the division results.

2. The subject subdivision method based on paper keyword spectrum clustering according to claim 1 is characterized in that: Get the keywords of each paper, merge the same keywords, and get the merged keywords, including: Based on big data technology, all keywords of publicly published papers in the specified discipline are retrieved, and statistical results are obtained when two keywords appear in the same paper at the same time; Based on the statistical results, the same keywords are screened out, and the same keywords are merged to obtain merged keywords.

3. The subject subdivision method based on paper keyword spectrum clustering according to claim 2 is characterized in that: Based on the merged keywords, a spectrum of two-by-two occurrences of keywords in the paper is constructed, including: Get data of M papers and N keywords; Based on big data technology, the frequency of each two keywords appearing in a paper is retrieved; With keywords as nodes and their two-by-two appearances as connection relationships, a spectrum of two-by-two keyword appearances is constructed.

4. The subject subdivision method based on paper keyword spectrum clustering according to claim 3 is characterized in that: Based on the spectral clustering algorithm, the graph is segmented and the paper keywords are clustered to obtain the clustering of all keywords, including: Based on the spectral clustering method, an undirected graph is constructed with keywords as nodes, pairs of keywords as connecting edges, and the number of pairs of keywords as the weight of the connecting edge. The degree of any node is calculated to form the degree matrix, adjacency matrix, and Laplace matrix. The matrix is ​​optimized to obtain the feature matrix. Clustering is performed based on the K-means clustering method to obtain clusters of all keywords.

5. The subject subdivision method based on paper keyword spectrum clustering according to claim 4 is characterized in that: The method for analyzing the number of categories in the clustering process is as follows: After retrieving the frequency of each two keywords appearing in a paper based on big data technology, the 10% keywords with the highest frequency ranking are selected as core keywords to obtain the core keyword values; The value of the core keyword is used as the number of categories for classification.

6. The subject subdivision method based on paper keyword spectrum clustering according to claim 1 is characterized in that: Based on the spectral clustering algorithm, the graph is segmented and the keywords of the paper are clustered, including: The keywords are nodes, the two-by-two occurrences are connected edges, and the frequency of two-by-two occurrences is the weight of the connecting edge. Thus, an undirected graph G = (K, E) is constructed, where K is the keyword set, i.e., all the nodes of the undirected graph, K = {k1, k2, ..., k N }; For any two points k in K i and k j , define w ij For the weights between them, there are w ij =w ji ; For two points k connected by an edge i and k j , w ij > 0, and indicates the frequency of two keywords appearing simultaneously. For two points k that are not connected by an edge i and k j , w ij =0, indicating that the two keywords have never appeared at the same time; For any point k in the graph i , its degree d i is the sum of the weights of all edges connected to it, that is In the formula, n is the point k i The number of all connected edges; The degree matrix D of all keywords N×N is as follows: Using the weight values ​​between all nodes, we get the N×N adjacency matrix W of all keywords. The formula is as follows: Calculate the Laplace matrix L, that is, L = DW, and construct the optimized H matrix, H = D -1 / 2 LD -1 / 2 ; Solve the optimization matrix H and calculate H = D -1 / 2 LD -1 / 2 The eigenvectors f1, f2, ..., f corresponding to the smallest c eigenvalues c ; The corresponding eigenvectors f1, f2, ..., f c The composed matrix is ​​normalized by row to form a feature matrix F of dimension N×c; Each row in F is taken as a c-dimensional sample, with a total of N samples. K-means clustering method is used for clustering, and the clustering dimension is J. The cluster Z is obtained and divided into (Z1, Z2, ..., Z J ).

Images