Subject subdivision field division method and system based on thesis keyword K-means clustering and medium

Abstract

The invention discloses a subject subdivision field division method and system based on thesis keyword K-means clustering and a medium, and the method comprises the following steps: extracting keywords from public theses in a specified subject field by using a big data technology, defining a retrieval range which comprises a database or an academic periodical, extracting keywords of each thesis, and defining the retrieval range of the database or the academic periodical; combining the same keywords to obtain a complete keyword set; searching the occurrence frequency of each keyword in each thesis, taking each keyword as a feature vector, taking the occurrence frequency of each keyword in each thesis as a dimension value, and constructing a keyword feature vector; word frequency-inverse document word frequency processing is carried out on the occurrence frequency of the keywords in each thesis, and thesis keyword word frequency-inverse document word frequency feature vectors are established; calculating the total word frequency-inverse document word frequency of keywords in all papers; sorting from small word frequency to large word frequency, and selecting keywords with a preset proportion as head words; taking the number of the head words as an initial category number of K-means clustering, taking the word frequency-inverse document word frequency feature vector of each head word as a category center value, and clustering other keywords based on a K-means clustering calculation method to obtain a clustering result; and the subject field is divided based on the clustering result to obtain the subject subdivision field, so that the subject subdivision precision is improved.

Description

Technical Field

[0001] The present invention relates to a subject field division technology, and in particular to a method, system and computer-readable storage medium for dividing subject fields into subdivided fields by using a K-means clustering algorithm based on paper keywords. Background Art

[0002] With the advancement of science and technology, humanity has entered an information-based and intelligent age, placing increasingly sophisticated and accurate demands on disciplinary and talent evaluation. With the development of big data and artificial intelligence technologies, third-party evaluations are gaining increasing attention from society, governments, and businesses. However, current disciplinary classification methods are often too broad and fail to accurately reflect the strengths of specific fields. For example, Springer categorizes journals and books into 13 major categories: architecture and design, behavioral sciences, biomedicine and life sciences, business and economics, chemistry and materials sciences, computer science, earth and environmental sciences, engineering, humanities, social sciences and law, mathematics and statistics, medicine, physics and astronomy, and computer vocational technology and professional computer applications. In China, based on graduate education and degree authorization, disciplines are divided into 14 disciplinary categories: philosophy, economics, law, education, literature, history, science, engineering, agriculture, medicine, military science, management, art, and interdisciplinary studies, totaling 113 first-level disciplines. This classification method fails to reflect the specific characteristics and precise differences within each discipline, and therefore cannot effectively support disciplinary and talent evaluation. For example, within the science category, there are first-level disciplines such as mathematics, physics, chemistry, astronomy, geography, atmospheric science, ocean science, geophysics, and geology. In reality, disciplines like mathematics and physics encompass a very broad range of disciplines. For example, a scholar may excel in the Hall quantum effect, but not necessarily in the entire field of physics. Similarly, a university may have outstanding research results in the Hall quantum effect, but not necessarily in physics. This situation significantly limits disciplinary and talent evaluation, particularly in talent recruitment. Talent recruitment requires specialized expertise in a specific area of ​​a discipline, necessitating the categorization of disciplines into specific sub-fields.

[0003] When evaluating talent's academic level and technical capabilities, comparing across the entire discipline results in a large pool of talent, making it difficult to highlight individual strengths and accurately identifying scholars with outstanding capabilities in specific research areas. To address this issue, this paper proposes a method for categorizing disciplines based on K-means clustering of paper keywords. This method uses big data technology to divide disciplines into several sub-fields and evaluates talent within these sub-fields. Summary of the Invention

[0004] This paper provides a method for subdividing disciplines based on K-means clustering of paper keywords. By using big data technology and the K-means clustering algorithm, it subdivides papers within a discipline, providing more accurate disciplinary classification results. This method is suitable for fields such as discipline evaluation and talent recruitment. The specific implementation process includes the following steps:

[0005] S1, using big data technology to extract keywords from public papers in a specified subject area, defining a search scope that includes databases and academic journals, extracting keywords from each paper, and merging identical keywords to obtain a complete keyword set;

[0006] S2, retrieve the number of times each keyword appears in each paper, take each keyword as a feature vector, and use the number of times it appears in each paper as the dimension value to construct the keyword feature vector;

[0007] S3, perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper, and establish the paper keyword word frequency-inverse document word frequency feature vector;

[0008] S4: Calculate the total keyword frequency minus the inverse document frequency (TF-IDF) of all keywords in all papers; sort them in ascending order of frequency, and select the keywords with the predetermined ratio as the core words;

[0009] S5, using the number of central words as the initial number of categories for K-means clustering, using the word frequency-inverse document word frequency feature vector of each central word as the class center value, clustering the remaining keywords based on the K-means clustering calculation method to obtain the clustering results;

[0010] S6, divide the subject areas based on the clustering results to obtain subject sub-areas.

[0011] Furthermore, step S1 specifically includes:

[0012] S101: Determine the disciplines for field division, retrieve all publicly published papers in the discipline based on big data technology, obtain the keywords of each paper, and merge the same keywords to obtain M papers and N keyword data;

[0013] S102, analyzing whether there are identical keywords based on the keyword data;

[0014] S103, if there are identical keywords, merge the identical keywords to obtain a keyword set;

[0015] S104: If there is no identical keyword, directly obtain the keyword set.

[0016] Furthermore, step S2 constructs a keyword feature vector, specifically including:

[0017] After obtaining data on M papers and N keywords, we applied big data technology to retrieve the number of times each keyword appeared in each paper;

[0018] Taking each keyword as a feature vector and the number of times it appears in each paper as the dimension value, the constructed keyword feature vector is

[0019] Keyword K1=[k 11 , k 12 ,......k 1M ]

[0020] Keyword K2=[k 21 , k 22 ,......k 2M ] ......

[0022] Keyword K N =[k N1 , k N2 ,......,k NM ].

[0023] Furthermore, in step S3, analyzing word frequency-inverse document word frequency specifically includes:

[0024] Construct the frequency feature vector of the keywords in the paper, eliminate the influence of common words on the frequency, and perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper. The calculation formula is:

[0025]

[0026] Where k ij is the number of times keyword i appears in article j, is the total number of times keyword i appears in all papers, log is the logarithm, D is the total number of papers in the subject, D i is the number of papers in which keyword i appears in the subject papers; the feature vector of keyword frequency-inverse document frequency is established as:

[0027] Keyword Ks1=[S 11 , S 12 ,,......,S 1M ]

[0028] Keyword Ks2=[S 21 , S 22 ,,......,S 2M ] ......

[0030] Keyword Ks N =[S N1, S N2 ,,......,S NM ].

[0031] Furthermore, the analysis of the core words of the paper in step S4 includes:

[0032] Obtain the feature vector of the keyword frequency of the paper - inverse document frequency, and calculate the total word frequency of each keyword in all papers - inverse document frequency. The calculation formula is as follows:

[0033]

[0034] K A (i) Sort the keywords from smallest to largest and take the top 10% of the keywords as the core words of the paper.

[0035] Furthermore, in step S5, the spatial distance between the word frequency-inverse document word frequency feature vector of the remaining keywords and the center value of each class is calculated, and the one with the smallest distance is divided into the corresponding class. The calculation formula is as follows:

[0036]

[0037] Where: D KZ (i, k) represents the distance between the i-th (i=1,2,...,z) center word and the k-th (k=1,2,...,NZ) other keywords;

[0038] Comparison D KZ The size of (i, k) (i=1, 2, ..., z) is the smallest when i=1, then the keyword k is classified into the first category, and so on, until all the keywords are classified.

[0039] The present invention also provides a subject subdivision field division system based on K-means clustering of paper keywords, including a processor, a memory and at least one program, wherein the program is stored in the memory and is configured to be executed by the processor, and the program includes instructions for executing the subject subdivision field division method based on K-means clustering of paper keywords as described in any of the above items.

[0040] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program enables a computer to execute to implement any of the above-mentioned methods for dividing subject subdivision fields based on K-means clustering of paper keywords.

[0041] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0042] The present invention calculates the spatial distance between the keyword's word frequency-inverse document word frequency feature vector and the center value of each class, and divides it into the corresponding category with the minimum distance; uses the classification accuracy coefficient to judge the accuracy of the clustering division of other keywords, and forms a new clustering division category; based on the average word frequency-inverse document word frequency feature vector value of the clustered division category, re-clustering the subject keywords using the K-means algorithm, and finally determines the clustering division of keywords, forming different subject subdivisions, and improving the accuracy of subject division. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of the subject subdivision method based on K-means clustering of paper keywords provided in this embodiment.

[0044] Figure 2 A flow chart of a keyword set acquisition method for a subject segmentation method based on K-means clustering of paper keywords provided in this embodiment is shown. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] like Figure 1-Figure 2 As shown, the embodiment of the present invention provides a method for dividing subject subdivision fields based on K-means clustering of paper keywords, comprising the following steps:

[0049] S1, using big data technology to extract keywords from public papers in a specified subject area, defining the search scope, which includes databases and academic journals, extracting keywords from each paper, and merging the same keywords to obtain a complete keyword set;

[0050] S2, retrieve the number of times each keyword appears in each paper, take each keyword as a feature vector, and use the number of times it appears in each paper as the dimension value to construct the keyword feature vector;

[0051] S3, perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper, and establish the paper keyword word frequency-inverse document word frequency feature vector;

[0052] S4: Calculate the total keyword frequency minus the inverse document frequency (TF-IDF) of all keywords in all papers; sort them in ascending order of frequency, and select the keywords with the predetermined ratio as the core words;

[0053] Specifically, the existing formula fully considers the division by zero phenomenon in extreme cases. To effectively avoid division by zero errors and improve the numerical stability of the algorithm, it is recommended to add a minimum value to the denominator, such as (1e-6) or (1e-8) for higher precision. The TF-IDF calculation formula is as follows:

[0054]

[0055] Among them, tf(i,j) represents the number of times keyword i appears in paper j, D represents the total number of all papers in the discipline, and df(i) represents the number of papers in the discipline where keyword i appears.

[0056] S5, using the number of central words as the initial number of categories for K-means clustering, using the word frequency-inverse document word frequency feature vector of each central word as the class center value, clustering the remaining keywords based on the K-means clustering calculation method to obtain the clustering results;

[0057] Specifically, the K-means clustering calculation formula is as follows:

[0058] In the K-means clustering algorithm, we use Euclidean distance to calculate the distance between each point (such as a keyword) and its cluster center. To avoid extremely small values ​​or denominators close to zero in the calculation, a small constant is added for smoothing.

[0059]

[0060] Where Dis(i,k) represents the Euclidean distance between keyword i and keyword k; TF-IDF(i,j) and TF-IDF(k,j) represent the TF-IDF feature vectors of keyword i and keyword k in paper j, respectively; ε is a small constant, such as (1e-6), which is used to avoid very small values ​​in distance calculation and ensure numerical stability.

[0061] S6, divide the subject areas based on the clustering results to obtain subject sub-areas.

[0062] Specifically, Figure 1 The main steps of the K-means clustering method for dividing subject sub-fields based on paper keywords are demonstrated. Figure 1 The operation sequence of steps S1 to S6 is the core operation of subject segmentation.

[0063] Furthermore, big data technology is a technology that uses advanced computing methods and tools to process massive data sets and quickly retrieve and analyze data.

[0064] K-means clustering is a common unsupervised learning algorithm that effectively groups data by assigning data points to different categories and reducing the variance within the category.

[0065] TF-IDF stands for Term Frequency-Inverse Document Frequency. It is a statistical method to measure the importance of a word in a document collection and is often used in text mining and information retrieval.

[0066] It should be noted that the method for obtaining the number of categories and determining the class center value is as follows:

[0067] After obtaining the core words of the paper, the number of core words is used as the initial number of categories for K-means clustering. Assuming that there are Z core words, the paper keyword frequency-inverse document frequency feature vector is rearranged to obtain the first Z keywords as the core words, and the Z+1th keyword and subsequent keywords are other keywords except the core words, i.e., the core words;

[0068] Keyword Ks1=[S 11 , S 12 ,,......,S 1M ]

[0069] Keyword Ks2=[S 21 , S 22 ,,......,S 2M ] ......

[0071] Keyword K SZ =[S z1 ,S z2 ,,......,S zM ]

[0072] Other keywords:

[0073] Keyword K Sz+1 =[S z+1 1,S z+1 2,,......,S z+1 M] ...... ......

[0076] Keyword Ks N =[S N1 , S N2 ,,......,S NM ];

[0077] Thus, the word frequency-inverse document word frequency feature vector of each central word is determined as the class center value.

[0078] According to an embodiment of the present invention, step S1 specifically includes:

[0079] S101: Determine the disciplines for field division, retrieve all publicly published papers in the discipline based on big data technology, obtain the keywords of each paper, and merge the same keywords to obtain M papers and N keyword data;

[0080] S102, analyzing whether there are identical keywords based on the keyword data;

[0081] S103, if there are identical keywords, merge the identical keywords to obtain a keyword set;

[0082] S104: If there is no identical keyword, directly obtain the keyword set.

[0083] According to an embodiment of the present invention, constructing a keyword feature vector in step S2 specifically includes:

[0084] After obtaining data on M papers and N keywords, we applied big data technology to retrieve the number of times each keyword appeared in each paper;

[0085] Taking each keyword as a feature vector and the number of times it appears in each paper as the dimension value, the constructed keyword feature vector is:

[0086] Keyword K1=[k 11 , k 12 ,......,k 1M ]

[0087] Keyword K2 ＝ [k 21 , k 22 ,......,k 2M ] ......

[0089] Keyword K N =[k N1 , k N2 ,......,k NM ].

[0090] According to an embodiment of the present invention, in step S3, analyzing word frequency-inverse document word frequency specifically includes:

[0091] Construct the frequency feature vector of the keywords in the paper, eliminate the influence of common words on the frequency, and perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper. The calculation formula is:

[0092]

[0093] Where k ij is the number of times keyword i appears in article j, is the total number of times keyword i appears in all papers, log is the logarithm, D is the total number of papers in the subject, D i for

[0094] The number of papers in which keyword l appears in the subject papers;

[0095] According to an embodiment of the present invention, the analysis of the core words of the paper in step S4 includes:

[0096] Obtain the feature vector of the keyword frequency of the paper - inverse document frequency, and calculate the total word frequency of each keyword in all papers - inverse document frequency. The calculation formula is as follows:

[0097]

[0098] Sort KA(i) from small to large, and take the top 10% of keywords as the core words of the paper.

[0099] According to an embodiment of the present invention, in step S5, the spatial distance between the word frequency-inverse document word frequency feature vector of the remaining keywords and the center value of each class is calculated, and the class with the smallest distance is divided into the corresponding class. The calculation formula is as follows:

[0100]

[0101] Where: D KZ (i, k) represents the distance between the i-th (i=1,2,...,z) center word and the k-th (k=1,2,...,NZ) other keywords;

[0102] Comparison D KZThe size of (i, k) (i=1,2,...,z) is the smallest when i=1, then keyword k is classified into the first category, and so on, until all keywords are classified.

[0103] According to an embodiment of the present invention, the accuracy of the subject keyword classification is determined, including:

[0104] When selecting the central word, the distance between the central words is not considered, so the distance may be very large or very small, which may affect the accuracy of the classification of other keywords. The classification accuracy coefficient is used to judge the accuracy of the clustering of other keywords. Taking other keyword q as an example, the steps are:

[0105] (1) Calculate the average distance between q and keywords in the same category, denoted as a(q).

[0106] (2) Calculate the average distance between q and the keywords in the other i-th category, denoted as b(q,i), and let b(q) = min(b(q,1),b(q,2),…b(q,i),…,b(q,Z)), i = 1,2,…,Z, i ≠ q. (3) Calculate the classification accuracy coefficient, the calculation formula is as follows:

[0107]

[0108] The closer F(q) is to 1, the more reasonable the clustering of q is; the closer F(q) is to -1, the more suitable q is for clustering into other categories; the closer F(q) is to 0, the more likely q is on the boundary between two categories. Thus, the accuracy judgment standard for the classification of subject keywords is given. When F(q)>0, the clustering of q remains unchanged; when F(q)<0, q is clustered into the category with the b(q) value obtained in step (2). Similarly, other keywords are judged one by one to form new clustering categories, completing the first clustering.

[0109] Based on the K-means clustering algorithm with the average word frequency of the clustered categories and the inverse document word frequency feature vector value as the center value, the subject keywords are re-clustered and the subject subdivision fields are determined. The steps are as follows:

[0110] (1) After the accuracy of the subject keyword classification is determined, the number of categories needs to be re-determined. That is, after the keyword classification is adjusted, if the number of keywords in a certain cluster category is less than the specified value (generally 5), this cluster category will be cancelled, and the number of cluster categories will be reduced by the corresponding value to obtain a new number of cluster categories.

[0111] (2) After obtaining the new number of cluster categories, calculate the average word frequency-inverse document word frequency feature vector value of each cluster category, denoted as SMi,j, where i = 1, 2, ..., z', z' is the new number of cluster categories; j = 1, 2, ..., M. Using this as the center value of the cluster category, reapply the K-means clustering calculation method to re-cluster all keywords, that is, by calculating the spatial distance between the word frequency-inverse document word frequency feature vector of each keyword and the new cluster center value, divide it into the corresponding category with the minimum distance. The calculation formula is

[0112]

[0113] Denotes the distance between the i-th (i=1,2,...,z′) cluster category and the k-th (k=1,2,...,N) keyword. Compare D KM The size of (i, k) (i=1, 2, ..., z′) is the smallest when i=1, then the keyword k is divided into the first cluster category, and so on, until all the keywords are divided.

[0114] (3) After re-clustering and division, the clustering of all keywords is completed; the keywords divided by the new clusters form different subject sub-fields.

[0115] According to an embodiment of the present invention, the method further includes calculating the spatial distance between the word frequency-inverse document word frequency feature vector of other keywords and the center value of each class, and classifying them into corresponding categories with the minimum distance.

[0116] The classification accuracy coefficient is used to judge the accuracy of clustering of other keywords to form new clustering categories.

[0117] Based on the average word frequency-inverse document word frequency feature vector value of the clustered categories, the K-means algorithm is used to re-cluster the subject keywords, and finally the clustering of keywords is determined to form different sub-fields of the subject.

[0118] This embodiment also provides a subject subdivision field division system based on K-means clustering of paper keywords, including a processor, a memory and at least one program, the program is stored in the memory and is configured to be executed by the processor, and the program includes instructions for executing any of the above-mentioned subject subdivision field division methods based on K-means clustering of paper keywords.

[0119] Those skilled in the art will appreciate that, for ease of explanation, the following example illustrates a configuration in which one memory and one processor are provided. In an actual terminal or server, multiple processors and memories may exist. A memory may also be referred to as a storage medium or storage device, etc., which is not limited in the present embodiment.

[0120] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may also be a general-purpose microprocessor, a graphics processing unit (GPU), or one or more integrated circuits for executing relevant programs to implement the functions required to be executed in the embodiments of the present application.

[0121] The processor can also be an integrated circuit chip with signal processing capabilities. During the implementation process, the various steps of the present application can be completed by the integrated logic circuit of the hardware in the processor or by instructions in the form of software. The above-mentioned processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory and read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in combination with its hardware, completes the functions required to be performed by the units included in the method, device and storage medium of the embodiments of the present application.

[0122] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache.

[0123] By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0124] The memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be independent and connected to the processor via a bus. The memory may also be integrated with the processor, and the memory may store a program. When the program stored in the memory is executed by the processor, the processor is used to execute the various steps of the determination method in the above-mentioned embodiment of the present application.

[0125] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor. It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0126] It should be understood that the term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0127] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0128] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a processor, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a computer network, or other programmable device.

[0130] This embodiment also provides a computer-readable storage medium, which stores a computer program. The computer program enables a computer to execute to implement the above-mentioned method for dividing subject subdivision fields based on K-means clustering of paper keywords.

[0131] It should be noted that computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means, or can be transmitted from one website, computer, server or data center to a mobile phone processor via wired means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media. Available media can be magnetic media (e.g., floppy disk, hard disk), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive), etc.

[0132] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for dividing subject areas into sub-fields based on K-means clustering of paper keywords, characterized by: The following steps are involved: S1, using big data technology to extract keywords from public papers in a specified subject area, defining a search scope that includes databases and academic journals, extracting keywords from each paper, and merging identical keywords to obtain a complete keyword set; S2, retrieve the number of times each keyword appears in each paper, take each keyword as a feature vector, and use the number of times it appears in each paper as the dimension value to construct the keyword feature vector; S3, perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper, and establish the paper keyword word frequency-inverse document word frequency feature vector; S4: Calculate the total keyword frequency in all papers minus the inverse document frequency; sort by frequency from smallest to largest, and select the keywords with the previously set ratio as the central words; S5, using the number of central words as the initial number of categories for K-means clustering, using the word frequency-inverse document word frequency feature vector of each central word as the class center value, clustering the remaining keywords based on the K-means clustering calculation method to obtain the clustering results; S6, divide the subject areas based on the clustering results to obtain subject sub-areas.

2. The subject subdivision method based on K-means clustering of paper keywords according to claim 1 is characterized in that Step S1 specifically includes: S101: Determine the disciplines for field division, retrieve all publicly published papers in the discipline based on big data technology, obtain the keywords of each paper, and merge the same keywords to obtain M papers and N keyword data; S102, analyzing whether there are identical keywords based on the keyword data; S103, if there are identical keywords, merge the identical keywords to obtain a keyword set; S104: If there is no identical keyword, directly obtain the keyword set.

3. The subject subdivision method based on K-means clustering of paper keywords according to claim 2 is characterized in that In step S2, a keyword feature vector is constructed, which specifically includes: After obtaining data on M papers and N keywords, we applied big data technology to retrieve the number of times each keyword appeared in each paper; Taking each keyword as a feature vector and the number of times it appears in each paper as the dimension value, the constructed keyword feature vector is: Keyword K1=[k 11 , k 12 ,......,k 1M ] Keyword K2=[k 21 , k 22 ,......,k 2M ] ...... Keyword K N =[k N1 , k N2 ,......,k NM ].

4. The subject subdivision method based on K-means clustering of paper keywords according to claim 3 is characterized in that In step S3, analyzing word frequency-inverse document word frequency specifically includes: Construct the frequency feature vector of the keywords in the paper, eliminate the influence of common words on the frequency, and perform word frequency-inverse document word frequency processing on the frequency of keywords in each paper. The calculation formula is: Where k ij is the number of times keyword i appears in article j, is the total number of times keyword i appears in all papers, log is the logarithm, D is the total number of papers in the subject, D i is the number of papers in which keyword i appears in the subject papers; the feature vector of keyword frequency-inverse document frequency is established as: Keyword Ks1=[S 11 , S 12 ,,......,S 1M ] Keyword Ks2=[S 21 , S 22 ,,......,S 2M ] ...... Keyword Ks N =[S N1 , S N2 ,,......,S NM ].

5. The subject subdivision method based on K-means clustering of paper keywords according to claim 4 is characterized in that The analysis of the core words of the paper in step S4 includes: Obtain the feature vector of the keyword frequency of the paper - inverse document frequency, and calculate the total word frequency of each keyword in all papers - inverse document frequency. The calculation formula is as follows: K A (i) Sort the keywords from smallest to largest and take the top 10% of the keywords as the core words of the paper.

6. The subject subdivision method based on K-means clustering of paper keywords according to claim 5 is characterized in that: In step S5, the spatial distance between the word frequency-inverse document word frequency feature vector of the remaining keywords and the center value of each class is calculated, and the class with the smallest distance is divided into the corresponding class. The calculation formula is as follows: Where: D KZ (i, k) represents the distance between the i-th (i=1,2,...,z) center word and the k-th (k=1,2,...,NZ) other keywords; Comparison D KZ The size of (i, k) (i=1,2,...,z) is the smallest when i=1, then keyword k is classified into the first category, and so on, until all keywords are classified.

7. A subject subdivision system based on K-means clustering of paper keywords, characterized by: It includes a processor, a memory and at least one program, wherein the program is stored in the memory and is configured to be executed by the processor, and the program includes instructions for executing the subject subdivision field division method based on K-means clustering of paper keywords as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which enables a computer to execute to implement the subject subdivision field division method based on K-means clustering of paper keywords according to any one of claims 1 to 6.

Images