A biomedical literature hierarchical topic tree construction method and device

Abstract

The present application relates to biomedical literature content retrieval, text classification and other technical fields, in particular to a biomedical literature hierarchical topic tree construction method and device, the method comprising obtaining the full text content of the biomedical literature to be processed, and obtaining a standard to-be-processed document set after preprocessing; the three sentences adjacent to each document in the standard to-be-processed document set are processed as a document segment respectively; all document segments are taken as root nodes, and hidden Dirichlet distribution method is used for topic modeling of all document segments, and two topics are generated; the relevance of each document segment to a certain topic is calculated, and two key values are obtained by calculating and searching all relevance; the document segments are divided according to the key values; if the document segments related to the topic are greater than a set threshold, a new topic is continuously generated; the generation efficiency of the present application is far superior to other hierarchical topic modeling methods.

Description

Technical Field

[0001] This invention relates to the technical fields of biomedical literature content retrieval and text classification, and in particular to a method and apparatus for constructing a hierarchical topic tree for biomedical literature. Background Technology

[0002] Topic modeling methods, through topic mining of text data, can obtain text topic information and document-topic distributions, and can then be used as a basis for further processing tasks such as information retrieval, text classification, and recommendation systems. Existing hierarchical topic tree construction methods, by constructing a tree-like hierarchical topic structure, can obtain the relationships between topics. Currently, these methods mainly include two categories: hierarchical topic models based on LDA (Latent Dirichlet Allocation), represented by hLDA (hierarchical Latent Dirichlet Allocation), and latent topic trees, represented by HLTA (Hierarchical Latent Tree Analysis). Both of these methods are probabilistic generative methods. During topic tree generation, a considerable number of latent variables need to be processed, resulting in high time and space complexity. When processing large text sets such as full-text biomedical literature, it is difficult to generate a complete topic tree within a limited time.

[0003] Biomedical literature constitutes the most important textual information resource in the biomedical field. With the rapid development of scientific research in biomedicine, the number of biomedical articles is constantly increasing; for example, Medline adds 2,000 to 4,000 new biomedical articles daily. How to fully mine this massive amount of textual information and discover new medical knowledge is of great significance to life science research. By constructing hierarchical topic trees from the full-text content of these biomedical articles, further tasks such as topic analysis, content retrieval, and text classification can be carried out. Summary of the Invention

[0004] To expedite the task of hierarchical topic modeling for biomedical literature, this invention proposes a method and apparatus for constructing a hierarchical topic tree for biomedical literature. The method includes the following steps:

[0005] S1. Obtain the full text of the biomedical literature to be processed, and perform data cleaning, word form restoration, stop word removal and stemming operations on the obtained data in sequence to obtain a standard set of documents to be processed;

[0006] S2. Process the three consecutive sentences in each document of the standard document set into a document fragment;

[0007] S3. Take all document fragments as the root node and use the Hidden Dirichlet Distribution method to perform topic modeling on all document fragments to generate two topics;

[0008] S4. Calculate the relevance of each document fragment to a specific topic. Calculate and search for all relevance values ​​to obtain two key values. If the relevance of a document fragment is greater than the larger key value, then classify the document fragment under that topic and treat that topic and its document fragments as a leaf node. Conversely, if the relevance of a document fragment is less than the smaller key value, then classify the document fragment under another topic and treat that topic and its document fragments as a leaf node.

[0009] S5. If the number of document fragments related to the topic exceeds the set threshold, the hidden Dirichlet distribution algorithm is used to model the topic of the document fragments under each topic, generating two topics, and then returning to step S4.

[0010] Furthermore, the topics generated using the Hidden Dirichlet Distribution method consist of 20 keywords.

[0011] Furthermore, the relevance of a document fragment to a topic is the topic distribution that generated the text fragment.

[0012] Furthermore, the process of calculating the relevance of each document fragment to a specific topic, calculating and searching for all relevance scores to obtain two key values, and classifying document fragments into different topics based on the relationship between relevance and key values ​​includes the following steps:

[0013] The Hidden Dirichlet Distribution method is used to calculate the relevance of each document fragment to a specific topic.

[0014] We count the different document fragments with the same relevance to form the relevance distribution of all document fragments, and use a Gaussian mixture model curve with two Gaussian distributions to represent the distribution, as follows:

[0015]

[0016] Where 0≤x≤1, x represents the relevance of each document fragment to a specific topic; α1 and α2 are the fitting coefficients; The expectation is μ. k variance is The standard Gaussian distribution of , k∈{1,2};

[0017] If (x0, y0) is the lowest point of the Gaussian mixture model curve f(x), then let:

[0018]

[0019] in,

[0020] Extract points m located between [μ1, x0] and n located between [x0, μ2] on the x-axis, respectively, and calculate g(m) and g(n) respectively. If 1 - min(g(m), g(n)) / max(g(m), let m = x1 and n = x2 and calculate v, that is:

[0021]

[0022] Select all m and n that satisfy 1-min(g(m),g(n)) / max(g(m)) and continue to calculate the v value. Finally, take the m and n corresponding to the minimum v value as the optimal key values ​​x1 and x2.

[0023] Document fragments distributed between (x2,1] will be assigned to this topic, and document fragments distributed between [0,x1) will be assigned to another topic.

[0024] This invention also provides a biomedical literature hierarchical topic tree construction device, including a data preprocessing module, a document fragmentation module, and a hierarchical topic tree generation module, wherein:

[0025] The data preprocessing module is used to perform data cleaning, word form restoration, stop word removal, and stemming operations on the acquired full-text content of biomedical literature in sequence to obtain a standard set of documents to be processed;

[0026] The document fragment segmentation module is used to process three consecutive sentences in a document from a standard set of documents to be processed into a single document fragment.

[0027] The hierarchical topic tree generation module is used to construct a hierarchical topic tree. During construction, based on all document fragments, a topic generation model using the Hidden Dirichlet (HDD) distribution method is invoked to generate two topics. The relevance of each document fragment to one of these topics is calculated. Two key values ​​are obtained by calculating and searching all relevance scores. If the relevance score of a document fragment is greater than the larger key value, the document fragment is assigned to that topic. The number of document fragments under that topic is checked. If it exceeds a set threshold, the HDD distribution-based topic generation model is invoked again to generate two new topics. This new topic and its associated document fragments are treated as leaf nodes. If the relevance score of a document fragment is less than the smaller key value, the document fragment is assigned to the other topic. The number of document fragments under that topic is checked. If it exceeds a set threshold, the HDD distribution-based topic generation model is invoked again to generate two new topics. This new topic and its associated document fragments are treated as another leaf node. This process is iterated until no further assignment is possible.

[0028] This invention also proposes a computer program for constructing a hierarchical topic tree for biomedical literature, which is used to implement a method for constructing a hierarchical topic tree for biomedical literature.

[0029] This invention also proposes a computer for constructing a hierarchical topic tree for biomedical literature, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program in the memory to implement a method for constructing a hierarchical topic tree for biomedical literature.

[0030] This invention recursively applies LDA to adaptively generate a binary topic tree from top to bottom. Compared with hLDA and HLTA, the hierarchical topic tree generated by this invention has good interpretability and topic quality, and the topic hierarchy structure is more reasonable. Furthermore, due to the use of a top-down heuristic generation method, the generation process of the multiple topic hierarchy structures constructed by this invention is particularly fast, and the generation efficiency is far superior to other hierarchical topic modeling methods, making it more suitable for handling hierarchical topic modeling tasks in biomedical literature. Attached Figure Description

[0031] Figure 1 This is a flowchart of a method for constructing a hierarchical topic tree for biomedical literature according to the present invention;

[0032] Figure 2 A schematic diagram of a hierarchical topic tree constructed according to the method of the present invention;

[0033] Figure 3 This diagram illustrates the execution time of the existing hLDA method on different Medline summary datasets.

[0034] Figure 4 This diagram illustrates the execution time of the existing HLTA method on different Medline summary datasets.

[0035] Figure 5 This diagram illustrates the execution time of the biomedical literature hierarchical topic tree construction method of the present invention on different Medline abstract datasets. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This invention proposes a method and apparatus for constructing a hierarchical topic tree for biomedical literature. The method includes the following steps:

[0038] S1. Obtain the full text of the biomedical literature to be processed, and perform data cleaning, word form restoration, stop word removal and stemming operations on the obtained data in sequence to obtain a standard set of documents to be processed;

[0039] S2. Process the three consecutive sentences in each document of the standard document set into a document fragment;

[0040] S3. Take all document fragments as the root node and use the Hidden Dirichlet Distribution method to perform topic modeling on all document fragments to generate two topics;

[0041] S4. Calculate the relevance of each document fragment to a specific topic. Calculate and search for all relevance values ​​to obtain two key values. If the relevance of a document fragment is greater than the larger key value, then classify the document fragment under that topic and treat that topic and its document fragments as a leaf node. Conversely, if the relevance of a document fragment is less than the smaller key value, then classify the document fragment under another topic and treat that topic and its document fragments as a leaf node.

[0042] S5. If the number of document fragments related to the topic exceeds the set threshold, the hidden Dirichlet distribution method is used to model the topic of the document fragments under each topic, generating two topics, and then returning to step S4.

[0043] In this embodiment, the topic generated using the Hidden Dirichlet Distribution method consists of 20 topic words.

[0044] In this embodiment, the relevance of a document fragment to a topic is determined by the topic distribution that generates the text fragment. The relevance of each document fragment to a specific topic is calculated. Two key values ​​are obtained by calculating and searching all relevance values. If the relevance of a document fragment is greater than the larger key value, the document fragment is assigned to that topic. If the relevance of a document fragment is less than the smaller key value, the document fragment is assigned to another topic. When the number of document fragments under a topic is greater than or equal to 20, the document fragments under that topic are further divided into topics using that topic as the parent node. That is, the document fragments under that topic are used as the input of the Hidden Dirichlet Distribution method to obtain two new topics, until the number of document fragments under the divided topics is less than a threshold.

[0045] This embodiment provides a specific implementation for dividing document fragments, which includes the following steps:

[0046] The Hidden Dirichlet Distribution method is used to calculate the relevance of each document fragment to a specific topic.

[0047] We count the different document fragments with the same relevance to form the relevance distribution of all document fragments, and use a Gaussian mixture model curve with two Gaussian distributions to represent the distribution, as follows:

[0048]

[0049] Where 0≤x≤1, x represents the relevance of each document fragment to a specific topic; α1 and α2 are the fitting coefficients; The expectation is μ. k variance is The standard Gaussian distribution of , k∈{1,2};

[0050] If (x0, y0) is the lowest point of the Gaussian mixture model curve f(x), then let:

[0051]

[0052] in,

[0053] In this embodiment, during the process of obtaining the minimum point, let y0 = f1(x0) = f2(x0), then the value of the minimum point (x0, y0) of the Gaussian mixture model curve f(x) is expressed as:

[0054]

[0055]

[0056] In this embodiment, document fragments distributed between (x2, 1) will be assigned to one topic, document fragments distributed between [0, x1) will be assigned to another topic, and document fragments distributed between [x1, x2] will be discarded. The objective function for obtaining the key values ​​x1 and x2 is expressed as:

[0057]

[0058] Existing technologies struggle to obtain precise analytical values ​​of the objective function. Therefore, this embodiment first searches for a certain number of sample values ​​that satisfy the conditions, and then finds the optimal solution from them. The specific process is as follows:

[0059] Set the search precision for x, which is the number of points between [0,1]. For example, in this embodiment, 10,000 points evenly distributed on the x-axis are selected as the search samples.

[0060] Extract points m located between [μ1, x0] and n located between [x0, μ2] on the x-axis, respectively, and calculate g(m) and g(n) respectively. If 1 - min(g(m), g(n)) / max(g(m)) is satisfied, then the value is a sample value that meets the condition. Let m = x1, n = x2 and calculate v, that is:

[0061]

[0062] Select all m and n that satisfy 1-min(g(m),g(n)) / max(g(m)) and continue to calculate the v value. Finally, the m and n that have the minimum v value are the optimal key values ​​x1 and x2.

[0063] Document fragments distributed between (x2,1] will be assigned to one topic, document fragments distributed between [0,x1) will be assigned to another topic, and document fragments distributed between [x1,x2] will be discarded.

[0064] This invention also provides a biomedical literature hierarchical topic tree construction device, including a data preprocessing module, a document fragmentation module, and a hierarchical topic tree generation module, wherein:

[0065] The data preprocessing module is used to perform data cleaning, word form restoration, stop word removal, and stemming operations on the acquired full-text content of biomedical literature in sequence to obtain a standard set of documents to be processed;

[0066] The document fragment segmentation module is used to process three consecutive sentences in a document from a standard set of documents to be processed into a single document fragment.

[0067] The hierarchical topic tree generation module is used to construct a hierarchical topic tree. During the construction process, based on all document fragments, a topic generation model based on the Hidden Dirichlet distribution method is called to generate two topics. The relevance of each document fragment to one of the topics is calculated. For all relevances, two key values ​​are obtained through calculation and lookup. If the relevance of a document fragment is greater than the larger key value, the document fragment is assigned to that topic. The number of document fragments under that topic is checked. If it exceeds a set threshold, the topic generation model based on the Hidden Dirichlet distribution method is called again to generate two new topics. This new topic and its document fragments are treated as a leaf node. If the relevance of a document fragment is less than the smaller key value, the document fragment is assigned to another topic. The number of document fragments under that topic is checked. If it exceeds a set threshold, the topic generation model based on the Hidden Dirichlet distribution method is called again to generate two new topics. This new topic and its document fragments are treated as another leaf node. This process is iterated until no further division is possible.

[0068] like Figures 3-5The execution efficiency of hLDA, HLTA, and the biomedical literature hierarchical topic tree construction method of this invention was analyzed using Medline summary datasets of different sizes. The comparative experiment was carried out on a fat node server with a 96-core Intel E7 Xeon processor (3.0 GHz) and 6TB DDR3 high-speed memory.

[0069] In comparative experiments, the LDA estimation in the method of this invention is executed in two different ways: serial and parallel. In serial execution, each LDA estimation step is executed individually, and only one LDA is executed at a time. In parallel execution, LDA estimation can be executed concurrently on a fat-node server at maximum concurrency. As the corpus size increases, the execution time of all methods increases; however, the execution time of hLDA and HLTA is one or two orders of magnitude longer than that of the method of this invention. Figures 3 to 5 As shown, HLTA has a shorter execution time than hLDA, with hLDA taking the most time. Furthermore, if the LDA estimation in the method of this invention is executed in parallel at all levels, the execution time of the method can be reduced by 1 / 3 to 1 / 2. Therefore, it is evident that the execution efficiency of the method of this invention is far superior to hLDA and HLTA, and thus, the method of this invention is capable of meeting the needs of hierarchical topic modeling for large-scale datasets.

[0070] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a hierarchical topic tree for biomedical literature, characterized in that, Specifically, it includes the following steps: S1. Obtain the full text content of the biomedical literature to be processed, and successively perform data cleaning, lemmatization, stop word removal, and stemming extraction operations on the obtained data to obtain a standard document set to be processed; S2. Process the three adjacent sentences in each document of the standard document set to be processed into a document fragment one by one; S3. Take all document fragments as root nodes, and use the Latent Dirichlet Allocation (LDA) method to perform topic modeling on all document fragments to generate two topics; S4. Calculate the relevance of each document fragment to one of the topics respectively, calculate and find two key values for all relevance values. If the relevance of a document fragment is greater than the larger of the two key values, then divide the document fragment into that topic. Specifically, it includes: Use the Latent Dirichlet Allocation (LDA) method to calculate the relevance of each document fragment to one of the topics respectively; Count different document fragments with the same relevance to form the relevance distribution of all document fragments, and use a Gaussian mixture model curve with two Gaussian distributions to represent it, which is expressed as: ， Where 0≤x≤1, x represents the relevance of each document fragment to a particular topic; , These are the fitting coefficients; Indicates expectation as variance is The standard Gaussian distribution of , k∈{1,2}; If (x0, y0) is a Gaussian mixture model curve The lowest point, then let: in, , ; Extract points m between [μ1, x0] and points n between [x0, μ2] on the x-axis, calculate g(m) and g(n) respectively. If 1 - min(g(m), g(n)) / max(g(m), g(n)) < t, where t is a fixed threshold, let m = x1, n = x2 and calculate v, that is: ； Select all m and n that satisfy 1 - min(g(m), g(n)) / max(g(m), g(n)) < t and continue to calculate the v value. Finally, take m and n when the minimum v value is obtained as the optimal key values x1 and x2; Document fragments distributed between (x2, 1] will be divided into one topic, and document fragments distributed between [0, x1) will be divided into another topic; And take this topic and the document fragments belonging to this topic as a leaf node; conversely, if the relevance of a document fragment is less than the smaller key value, then divide the document fragment into another topic, and take this topic and the document fragments belonging to this topic as a leaf node; S5. If the number of document fragments related to a certain topic is greater than the set threshold, continue to use the Latent Dirichlet Allocation (LDA) method to perform topic modeling on the document fragments under this topic to generate two topics, and return to step S4.

2. The method for constructing a hierarchical topic tree for biomedical literature according to claim 1, characterized in that, The topics generated by using the Latent Dirichlet Allocation (LDA) method are composed of 20 topic words.

3. The method for constructing a hierarchical topic tree for biomedical literature according to claim 1, characterized in that, The relevance of a document fragment to a topic is the topic distribution for generating this document fragment.

4. A biomedical literature hierarchical topic tree construction device, characterized in that, It includes a data preprocessing module, a document fragment division module, and a hierarchical topic tree generation module. Among them: The data preprocessing module is used to successively perform data cleaning, lemmatization, stop word removal, and stemming extraction operations on the full text content of the biomedical literature obtained to obtain a standard document set to be processed; The document fragment division module is used to process three adjacent sentences in each document of the standard document set into a document fragment respectively; A hierarchical topic tree generation module is used to construct a hierarchical topic tree. During the construction process, according to all document fragments, a topic generation model based on the Latent Dirichlet Allocation (LDA) method is called to generate two topics. The relevance of each document fragment to one of the topics is calculated, and two key values are obtained through calculation and search of all relevances. If the relevance of a document fragment is greater than the larger of the two key values, the document fragment is classified under that topic. Specifically, it includes: Calculating the relevance of each document fragment to one of the topics using the Latent Dirichlet Allocation method; Counting different document fragments with the same relevance to form the relevance distribution of all document fragments, and using a Gaussian mixture model curve with two Gaussian distributions to represent it, which is expressed as: ， Where 0≤x≤1, x represents the relevance of each document fragment to a particular topic; , These are the fitting coefficients; Indicates expectation as variance is The standard Gaussian distribution of , k∈{1,2}; If (x0, y0) is a Gaussian mixture model curve The lowest point, then let: in, , ; Selecting points m between [μ1, x0] and points n between [x0, μ2] on the x-axis, and calculating g(m) and g(n) respectively. If 1 - min(g(m), g(n)) / max(g(m), g(n)) < t, where t is a fixed threshold, let m = x1, n = x2, and calculate v, that is: ； Selecting all m and n that satisfy 1 - min(g(m), g(n)) / max(g(m), g(n)) < t and continuing to calculate the v value. Finally, the m and n when the minimum v value is obtained are the optimal key values x1 and x2; Document fragments distributed between (x2, 1] will be classified under one topic, and document fragments distributed between [0, x1) will be classified under another topic; And judging the number of document fragments under the topic. If it is greater than the set threshold, the topic generation model based on the Latent Dirichlet Allocation method is called to continue generating two new topics, and this topic and the document fragments belonging to this topic are used as a leaf node. If the relevance of a document fragment is less than the smaller key value, the document fragment is classified under the other topic, and the number of document fragments under the topic is judged. If it is greater than the set threshold, the topic generation model based on the Latent Dirichlet Allocation method is called to continue generating two new topics, and this topic and the document fragments belonging to this topic are used as another leaf node. It is continuously iterated until no further classification is possible.

5. The biomedical literature hierarchical topic tree construction device according to claim 4, characterized in that, The topics generated using the Latent Dirichlet Allocation method consist of 20 topic words.

6. The biomedical literature hierarchical topic tree construction device according to claim 4, characterized in that, The relevance of a document fragment to a topic, that is, the topic distribution for generating this document fragment.

7. A computer program for constructing a hierarchical topic tree for biomedical literature, characterized in that, It is used to implement a method for constructing a hierarchical topic tree of biomedical literature described in claim 1.

8. A computer for constructing a hierarchical topic tree for biomedical literature, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor runs the computer program in the memory to implement a method for constructing a hierarchical topic tree of biomedical literature described in claim 1.

Images