A data mining and analysis system based on journal literature

Abstract

The application discloses a kind of data mining and analysis system based on journal literature, it is related to literature data processing technical field, including data acquisition module, node association module, bridge identification module, theme modeling module, theme matching module and knowledge injection module.Data acquisition module pre-processes literature data and identifies core and edge literature node;Node association module iteratively calculates association strength and screens strong association node set;Bridge identification module screens cross-cluster bridge node that has strong association with different core nodes in history iteration;Theme modeling module extracts the core theme distribution vector of cross-cluster bridge node;Theme matching module calculates theme matching degree;Knowledge injection module extracts key sentence for low matching degree theme and constructs knowledge fragment, injects core node in reverse and updates its feature vector.The system optimizes literature node association analysis and core node representation, improves the comprehensiveness of journal literature data mining and analysis.

Description

Technical Field

[0001] This invention belongs to the field of literature data processing technology, specifically a data mining and analysis system based on journal articles. Background Technology

[0002] Currently, data mining and analysis systems for journal articles have been widely used in academic research, intelligence analysis and other scenarios. In existing technologies, such systems typically first acquire the metadata and full text of journal articles, then identify core and peripheral document nodes after preprocessing, and then filter out a set of strongly related nodes by calculating the correlation strength between core document nodes and other nodes. Subsequently, they carry out follow-up analysis work such as topic modeling and topic matching to achieve the mining and organization of journal article data.

[0003] Existing technologies for document node association analysis only focus on the association strength between core document nodes and other nodes in the current iteration, neglecting the association status of nodes in historical iterations. This fails to identify key nodes connecting different core document node clusters, resulting in fragmented relationships between different clusters and hindering knowledge sharing. Furthermore, after topic matching, existing technologies do not perform targeted knowledge extraction and feedback for content that does not match the topic of a cross-node cluster. This means that the attribute representation of core document nodes relies solely on their own textual information, failing to supplement specific knowledge from associated nodes. Consequently, the feature vectors of core document nodes are not comprehensive, affecting the accuracy and completeness of data mining and analysis. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; Therefore, this invention proposes a data mining and analysis system based on journal articles, comprising: The data acquisition module acquires the original document metadata and full-text content, and performs preprocessing to identify core document nodes and peripheral document nodes. The node association module calculates the association strength between the current core document node and other document nodes in each iteration, and selects nodes with association strength higher than a preset threshold to form a set of strongly associated nodes. The bridge identification module selects those nodes that have strong associations with different core document nodes in the historical iteration process from the set of strongly associated nodes and marks them as cross-cluster bridge nodes. The topic modeling module obtains all unstructured text content blocks of the marked cross-cluster bridge nodes, performs topic modeling analysis on the text content blocks of the cross-cluster bridge nodes, and extracts the distribution vectors of multiple core topics they carry. The topic matching module compares the topic distribution of different core document nodes connected by the cross-cluster bridge node to the clusters in which they belong, and calculates the matching degree between the core topic distribution vector of the cross-cluster bridge node and the topic distribution of each cluster. The knowledge injection module, for core topics with a matching degree lower than a preset matching degree threshold, locates and extracts a set of key sentences expressing the core topic from the text content blocks of the cross-cluster bridge nodes, constructs knowledge fragments reflecting the specificity of the cross-cluster bridge nodes, and injects the knowledge fragments back into the attribute representation of the core document nodes connected to them, thereby updating the feature vector of the core document nodes.

[0005] Furthermore, the identification of core document nodes and peripheral document nodes includes: Obtain the original metadata and full-text content of the target journal's literature database; The original document metadata and full-text content are preprocessed to extract structured document attribute fields and unstructured text content blocks. The structured document attribute fields are processed using the density peak clustering algorithm to identify core document nodes and peripheral document nodes. The preprocessing of the original document metadata and full-text content includes: The title, author list, institution list, publication year, volume, issue and page number, abstract, keyword list and reference list fields are parsed from the original document metadata, and the non-numeric fields are standardized and encoded. The entire text was cleaned to remove headers, footers, figure and table titles, reference paragraphs, and acknowledgments paragraphs. The cleaned text is divided into continuous text blocks, each containing a preset number of paragraphs; Named entity recognition is performed on each text block to identify personal names, organization names, place names, and technical terms. The identified named entities are linked with the author and organization fields parsed from the metadata to establish an entity association network between the text content and the metadata; The preprocessed metadata structure, text content blocks, and entity association network are stored together as a preprocessed document data object that can be directly called in subsequent steps.

[0006] Furthermore, the processing of the structured document attribute fields based on the density peak clustering algorithm includes: For each document, a multidimensional attribute vector is constructed, which contains normalized values ​​of fields such as publication year, subject classification code, number of references, number of authors, and number of institutions. For each document, calculate the Euclidean distance between its multidimensional attribute vector and the multidimensional attribute vectors of all other documents; For each document, count the number of other documents whose surrounding distance is less than the specified neighborhood radius, and use this count as the local density of the corresponding document node; For each document, find all other document nodes with a local density higher than the currently processed document node, and calculate the minimum distance between the currently processed document node and all document nodes with a higher local density. If there are no other document nodes with a local density higher than the currently processed document node, set the aforementioned minimum distance to the preset maximum value. Draw a two-dimensional distribution map of the local density and minimum distance of all document nodes, and select those document nodes whose local density and minimum distance are both higher than their respective preset quantile values ​​as the initial core document nodes; The remaining document nodes are assigned to the clusters of the nearest core document nodes, thus completing the initial partitioning of the document set.

[0007] Furthermore, the calculation of the association strength between the current core document node and other document nodes includes: For the core document node currently being processed, obtain its multidimensional attribute vector; For any other document node within the current cluster, obtain its multidimensional attribute vector; Calculate the cosine similarity between the multidimensional attribute vectors of the core document node and the other document nodes, and use it as the basic correlation. If there is a direct citation or cited relationship between the other document nodes and the core document node, then set a citation relationship bonus coefficient. Determine whether the other document nodes and the core document nodes share at least one identical keyword. If they do, set a keyword overlap bonus coefficient based on the number of shared keywords. The weighted sum of the basic relevance, citation relationship bonus coefficient, and keyword overlap bonus coefficient yields a comprehensive score, which represents the relevance strength between the current core document node and any other document node within the current cluster.

[0008] Furthermore, topic modeling analysis is performed on the text content blocks of the cross-cluster bridge nodes, including: The full text of the cross-cluster bridge node is segmented, stop words are removed, and lemmatization is performed to obtain a clean word sequence. Based on the pure term sequence, construct the term-document co-occurrence matrix of the cross-cluster bridge node documents; The term-document co-occurrence matrix is ​​decomposed using the hidden Dirichlet distribution model to estimate the document-topic distribution matrix and the topic-term distribution matrix; From the document-topic distribution matrix, extract the probability values ​​of the cross-cluster bridge node documents belonging to each topic; The topics with the highest probability values ​​are selected as the core topics of the cross-cluster bridge nodes; For each core topic, find the terms with the highest probability of being associated with the core topic from the topic-term distribution matrix, and form the topic term set of the core topic; Each core theme, its corresponding probability value, and the set of keywords are combined to form a core theme distribution vector.

[0009] Furthermore, the calculation of the matching degree between the core topic distribution vectors of the cross-cluster bridge nodes and the topic distributions of each cluster includes: For each cluster containing a core document node connected by a cross-cluster bridge node, the topic distribution of all documents within that cluster is statistically analyzed. Calculate the average probability of all documents within the cluster for each topic to form the average topic distribution vector of the cluster; For each core topic distribution vector of a cross-cluster bridge node, calculate the Jaccard similarity coefficient between it and the average topic distribution vector of each connected cluster, the Jaccard similarity coefficient being calculated based on the topic word sets of the two vectors; The calculated Jaccard similarity coefficient is multiplied by the probability value of the core topic in the core topic distribution vector to obtain a comprehensive matching score; This overall matching score serves as the degree of matching between the core topic and the corresponding cluster.

[0010] Furthermore, a set of key sentences expressing the core theme is located and extracted from the text content blocks of the cross-cluster bridge nodes, including: For each core topic with a matching degree lower than a preset matching degree threshold, obtain its topic keyword set; The full text of the cross-cluster bridge node is segmented into independent sentences; For each sentence, its semantic relevance score with the core topic word set is calculated. The semantic relevance score is obtained by calculating the cosine similarity between the embedding vector of the word in the sentence and the average embedding vector of the topic word set. Sentences with semantic relevance scores higher than the sentence selection threshold are selected to form a set of candidate key sentences; Sentences in the candidate key sentence set are sorted according to their semantic coherence with the core document node summary and the sentence's position in the original text. Select a predetermined number of sentences from the sorted list to form the final set of key sentences used to construct knowledge fragments.

[0011] Furthermore, the construction of knowledge fragments reflecting the specificity of cross-cluster bridge nodes includes: Arrange all sentences in the aforementioned set of key sentences in the natural order in which they appear in the original text; The sorted sentence sequence is grammatically checked and corrected to ensure fluency. At the beginning of the sentence sequence, add a guiding sentence that summarizes the low-matching core topic that this knowledge fragment targets; If there is a semantic transition identified by a preset semantic transition threshold between adjacent sentences in a sentence sequence, a short conjunction or phrase is inserted to enhance coherence. The entire sentence sequence is compressed, redundant modifiers are removed, and the core subject-verb-object or subject-linking verb-complement structure is retained to form a concise knowledge statement; This concise knowledge statement is encapsulated to generate a unique fragment identifier, which is then associated with the cross-cluster bridge node identifier of its source, the corresponding core topic identifier, and the target core document node identifier. The encapsulated whole is the knowledge fragment.

[0012] Furthermore, the process of back-injecting knowledge fragments into the attribute representations of the core document nodes they are connected to includes: Obtain the current multidimensional attribute vector of the target core document node; The knowledge fragment is input into a sentence encoder to obtain a fixed-dimensional semantic vector representation of the knowledge fragment; The semantic vector representation of the knowledge fragment is concatenated with the current multidimensional attribute vector of the target core document node to form a new attribute vector that is extended and has higher dimensions. The new attribute vector is then dimensionality-reduced to restore its dimension to that of the original multidimensional attribute vector. The dimensionality-reduced vector is the updated core document node feature vector. During the dimensionality reduction process, linear discriminant analysis is used to perform projection transformation with the goal of maximizing the distinguishability between the core document nodes connected to the knowledge fragment and other document nodes in the same cluster, as well as between document nodes in other clusters, before and after knowledge fragment injection.

[0013] Furthermore, it also includes an iterative update module, used to trigger a new round of clustering and analysis based on the updated feature vector after the knowledge injection module performs the knowledge fragment back-injection operation, including: Using the updated feature vectors from all the literature, the calculation process based on density peak clustering is re-executed; The new clustering results identify new core document nodes and peripheral document nodes; Recalculate the association strength and identify new sets of strongly associated nodes and cross-cluster bridge nodes; For new cross-cluster bridge nodes, repeat the steps of topic analysis, matching degree calculation, knowledge fragment extraction and reverse injection; The process of recalculating, identifying, and analyzing the injection is repeated until the rate of change of the core document node set is lower than the preset convergence threshold in two consecutive iterations. Then the iteration stops, and the final document cluster, core nodes, bridge nodes, and derived knowledge fragment set are output.

[0014] Compared with the prior art, the beneficial effects of the present invention are: After iteratively calculating the association strength between the current core document node and other document nodes to obtain a set of strongly associated nodes exceeding a preset threshold, nodes that have strong associations with different core document nodes throughout historical iterations are selected from this set and marked as cross-cluster bridge nodes. This technical solution can accurately capture the key nodes connecting different core document node clusters, breaking the limitation of conventional techniques that only focus on current iteration associations and ignore historical associations. This allows the association relationships between different clusters to be revealed, achieving precise positioning of association paths between clusters. It also makes the association analysis of document nodes no longer limited to within a single cluster, making the association structure of the entire document network clearer.

[0015] After calculating the matching degree between the core topic distribution vector of the cross-cluster bridge node and the topic distribution of each cluster, for core topics with a matching degree lower than a preset matching degree threshold, the corresponding key sentence set is extracted from the text content block of the cross-cluster bridge node, a specific knowledge fragment is constructed, and this knowledge fragment is injected back into the attribute representation of the core document node it connects to, directly updating the feature vector of the core document node. This technical solution can specifically compensate for the limitations of the attribute representation of core document nodes in conventional techniques. By extracting the specific knowledge of the cross-cluster bridge node and injecting it back, it enriches the feature dimensions of the core document node, avoiding the problem of single feature due to the core document node relying solely on its own text information. This makes the feature vector of the core document node more closely match the actual knowledge content of the document, making subsequent document data mining and analysis more targeted and comprehensive. Attached Figure Description

[0016] Figure 1 This is a time series diagram of a data mining and analysis system based on journal articles as described in this invention; Figure 2This is a flowchart of the process based on the density peak clustering algorithm; Figure 3 A flowchart for the modeling and analysis of cross-cluster bridge node topics; Figure 4 This is a graph showing the rate of change of the core document node set during the iterative update process. Figure 5 This is a graph showing the change rate of core nodes and the number of knowledge fragments. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1The data acquisition module retrieves metadata and full-text content of original documents from the target journal literature database and preprocesses this data. Preprocessing includes structured parsing of metadata, cleaning and segmentation of the full-text content, and establishing a network of associations between text content and metadata through named entity recognition. Based on this, the system processes structured document attribute fields using a density peak clustering algorithm to identify core and peripheral document nodes in the document set, completing the initial clustering. The node association module is activated in each iteration of the analysis. It calculates the association strength between the current core document node and other document nodes within its cluster. This association strength considers multiple factors such as attribute vector similarity, citation relationships, and keyword overlap, and selects nodes with association strengths higher than a preset threshold to form a strongly associated node set. The bridge identification module further filters from this strongly associated node set, identifying nodes that have strong associations with different core document nodes throughout historical iterations. These nodes are marked as cross-cluster bridge nodes, acting as bridges connecting different knowledge clusters. For a marked cross-cluster bridge node, the topic modeling module acquires all its unstructured text content blocks and performs topic modeling analysis to extract multiple core topic distribution vectors carried by the node's document. The topic matching module compares the topic distributions of different core document nodes connected to the cross-cluster bridge node within their respective clusters, specifically calculating the matching degree between each core topic distribution vector of the cross-cluster bridge node and the topic distributions of each related cluster. The knowledge injection module operates on core topics with matching degrees below a preset matching degree threshold. It locates and extracts a set of key sentences expressing the core topic from the text content blocks of the cross-cluster bridge node, thereby constructing a knowledge fragment reflecting the specificity of the cross-cluster bridge node. Finally, this knowledge fragment is injected back into the attribute representations of the core document nodes it connects to, thereby updating the feature vectors of the core document nodes and achieving knowledge fusion and enhancement.

[0019] In one embodiment of the present invention, the system obtains original document metadata and full-text content from a target journal literature database. The original document metadata includes fields such as title, author list, affiliation list, publication year, volume, issue, page number, abstract, keyword list, and reference list. The full-text content is the complete body of the document. The original document metadata and full-text content are preprocessed to extract structured document attribute fields and unstructured text content blocks. The title, author list, affiliation list, publication year, volume, issue, page number, abstract, keyword list, and reference list fields are parsed from the original document metadata. The non-numerical fields of author, affiliation, and keywords are standardized and encoded; for example, institution names are standardized to their official full names, and synonym keywords are mapped to standard terms. The full-text content is then formatted and cleaned, removing headers, footers, figure and table titles, reference paragraphs, and acknowledgments paragraphs. The cleaned text is divided into continuous text blocks, each containing a preset number of paragraphs, which can be three or five paragraphs. Named entity recognition (NAME) is performed on each text block. This process identifies personal names, organization names, location names, and technical terms within the text block. The identified named entities are then linked to the author and organization fields parsed from the metadata, establishing an entity association network between the text content and the metadata. In practice, the preprocessed metadata structure, text content blocks, and entity association network are collectively stored as a preprocessed document data object that can be directly accessed in subsequent steps.

[0020] The density peak clustering algorithm is used to process structured document attribute fields, identifying core and peripheral document nodes during the process. (See also...) Figure 2 For each document, a multidimensional attribute vector is constructed. This vector contains normalized values ​​for fields such as publication year, subject classification code, number of references, number of authors, and number of institutions. Normalization linearly scales the values ​​of each field to the [0,1] interval. For each document, the Euclidean distance between its multidimensional attribute vector and the multidimensional attribute vectors of all other documents is calculated. The formula for calculating the Euclidean distance between the multidimensional attribute vectors of document i and document j is as follows:

[0021] in: This represents the Euclidean distance between document i and document j. This represents the total number of dimensions in a multidimensional attribute vector. This represents the normalized attribute value of document i in the k-th dimension. This represents the normalized attribute value of document j in the k-th dimension. For each document, the number of other documents whose surrounding distance is less than a specified neighborhood radius is counted, and this count is used as the local density of the corresponding document node. For each document, all other document nodes with a local density higher than the currently processed document node are found, and the minimum distance between the currently processed document node and all document nodes with higher local densities is calculated. If no other document nodes have a local density higher than the currently processed document node, the aforementioned minimum distance is set to a preset maximum value. A two-dimensional distribution map of the local density and minimum distance of all document nodes is plotted, and document nodes whose local density and minimum distance are both higher than their respective preset quantile values ​​are selected as the initial core document nodes. It can be understood that the preset quantile values ​​can be set according to the characteristics of the data distribution, for example, the thresholds for local density and minimum distance can be set to the 80th quantile of the corresponding values ​​of all nodes. The remaining document nodes are assigned to the clusters to which the nearest core document node belongs, completing the initial partitioning of the document set.

[0022] In one embodiment of the present invention, during each iteration, the node association module obtains the multi-dimensional attribute vector of the currently processed core document node. The multi-dimensional attribute vector of the currently processed core document node includes normalized values ​​of fields such as publication year, subject classification code, number of references, number of authors, and number of institutions. For any other document node within the current cluster, its multi-dimensional attribute vector is also obtained. The multi-dimensional attribute vector of any other document node contains the same set of fields and undergoes normalization. The cosine similarity between the multi-dimensional attribute vectors of the core document node and other document nodes is calculated, and the cosine similarity result serves as the basic association degree. The module also determines whether there is a direct citation or cited relationship between other document nodes and the core document node. In a specific implementation, this is done by parsing the reference list field of the document to determine whether the identifier of the other document node appears in the reference list of the core document node, or vice versa. If a direct citation or cited relationship exists, a citation relationship bonus coefficient is set. The system determines whether other document nodes share at least one identical keyword with the core document node. The determination of shared keywords is based on a keyword list field parsed and standardized from metadata. If at least one identical keyword is shared, a keyword overlap bonus coefficient is assigned based on the number of shared keywords. It can be understood that the citation relationship bonus coefficient can be a fixed constant, while the keyword overlap bonus coefficient can have a linear or non-linear positive correlation with the number of shared keywords.

[0023] In some embodiments, the keyword overlap scoring coefficient is directly related to the number of shared keywords. A base overlap score is set, and the product of the base overlap score and the number of shared keywords is used as the final keyword overlap scoring coefficient. The association strength is calculated by weighting and summing the base association degree, the citation relationship scoring coefficient, and the keyword overlap scoring coefficient. The comprehensive score obtained by the weighted sum is the association strength between the current core document node and any other document node within the current cluster. In a specific implementation, the association strength calculation formula can be expressed as:

[0024] in: This indicates the strength of the association between core document node a and other document nodes b within the cluster. The cosine similarity, or basic correlation, between the multidimensional attribute vectors of core document node a and document node b is represented. This represents the scoring coefficient for reference relationships, assigned based on their existence. This represents the keyword overlap bonus coefficient calculated based on the number of shared keywords. , , These are the weight coefficients for the corresponding items, and the sum of the weight coefficients is 1. Optional, the weight coefficients... , , The value can be preset based on the characteristics of literature associations in a specific subject area. For example, in fields where citation relationships are important, it can be set to... Assign a higher value. After calculating the association strength between the current core document node and all other document nodes within the cluster, the node association module selects nodes with association strength higher than a preset threshold to form a set of strongly associated nodes. The preset threshold can be the median or average of all association strength values.

[0025] In one embodiment of the present invention, the topic modeling module obtains all unstructured text content blocks of the marked cross-cluster bridge nodes, and performs topic modeling analysis on the text content blocks of the cross-cluster bridge nodes, see reference. Figure 3First, the full-text content of the cross-cluster bridge nodes is processed through word segmentation, stop word removal, and lemmatization. Word segmentation divides the continuous text into independent terms. Stop word removal filters out common words without actual semantic meaning. Lexical reconstruction restores the words to their dictionary prototypes, resulting in a clean term sequence. Based on this clean term sequence, a term-document co-occurrence matrix is ​​constructed for the cross-cluster bridge node documents. Rows in the term-document co-occurrence matrix correspond to terms, and columns correspond to documents. Matrix elements represent the frequency or weighted frequency of term occurrence in the document. The term-document co-occurrence matrix is ​​decomposed using a latent Dirichlet distribution model to estimate the document-topic distribution matrix and the topic-term distribution matrix. The document-topic distribution matrix describes the probability distribution of each document belonging to each topic, and the topic-term distribution matrix describes the probability distribution of each term occurrence under each topic. From the document-topic distribution matrix, the probability values ​​of the cross-cluster bridge node documents belonging to each topic are extracted. Several topics with the highest probability values ​​are selected as the core topics of the cross-cluster bridge nodes, and the number of selected topics is a preset parameter K. For each core topic, identify the terms with the highest probability of association with the core topic from the topic-term distribution matrix, forming a set of keywords for the core topic. Combine each core topic, its corresponding probability value, and the set of keywords to form a core topic distribution vector.

[0026] The topic matching module compares the topic distributions of different core document nodes connected by the cross-cluster bridge node to the topic distributions of each cluster, calculating the matching degree between the core topic distribution vectors of the cross-cluster bridge node and the topic distributions of each cluster. For each cluster of core document nodes connected to the cross-cluster bridge node, the topic distribution of all documents within the cluster is statistically analyzed using a Latent Dirichlet distribution model. The average probability of all documents within the cluster on each topic is calculated to form the average topic distribution vector of the cluster. For each core topic distribution vector of the cross-cluster bridge node, the Jaccard similarity coefficient is calculated between it and the average topic distribution vector of each connected cluster. The Jaccard similarity coefficient is calculated based on the topic word sets of the two vectors. The Jaccard similarity coefficient measures the degree of overlap between two topic word sets. The calculated Jaccard similarity coefficient is multiplied by the probability value of the core topic in the core topic distribution vector to obtain a comprehensive matching score, which is used as the matching degree between the core topic and the corresponding cluster. In specific implementation, the matching degree... The calculation formula can be expressed as:

[0027] in: Representing the core theme of cross-cluster bridge nodes With the connected clusters The degree of matching between them Represents the core themes in the core theme distribution vector. The probability value, Indicates the core theme Keyword Collection With clusters The set of keywords in the average topic distribution vector The Jaccard similarity coefficient between them. Optionally, when calculating the average topic distribution vector, a weighted average of the topic probabilities of documents within a cluster can be used, with the weights being the association strength between the document and the cluster core. The knowledge injection module operates on each core topic with a matching degree lower than a preset matching degree threshold, locating and extracting a set of key sentences expressing the core topic from the text content blocks of the cross-cluster bridge nodes. For each core topic with a matching degree lower than the preset matching degree threshold, its topic word set is obtained. The full-text content of the cross-cluster bridge nodes is segmented into independent sentences, with sentence segmentation based on punctuation and paragraph marks. For each sentence, its semantic relevance score with the core topic word set is calculated. The semantic relevance score is obtained by calculating the cosine similarity between the embedding vector of the word in the sentence and the average embedding vector of the topic word set. The embedding vector of the word comes from a pre-trained word vector model. Sentences with semantic relevance scores higher than the sentence selection threshold are selected to form a candidate key sentence set.

[0028] In some embodiments, the sentence selection threshold can be set as the average of the semantic relevance scores of all sentences. Sentences in the candidate key sentence set are ranked according to their semantic coherence with the core document node summary and their position in the original text. Semantic coherence with the core document node summary is obtained by calculating the cosine similarity between the sentence embedding vector and the summary text embedding vector. A predetermined number of sentences are selected from the ranked list to form the final key sentence set used to construct knowledge fragments; the predetermined number can be three or five sentences. This process aims to locate the core sentences that best represent the unique, low-matching themes of bridging documents.

[0029] In one embodiment of the present invention, the knowledge injection module, targeting core topics with a matching degree lower than a preset matching degree threshold, locates and extracts a set of key sentences expressing the core topic from the text content blocks of cross-cluster bridge nodes. This set of key sentences is used as input for the process of constructing knowledge fragments. All sentences in the key sentence set are arranged in their natural order of appearance in the original text, determined by their position index within the original full-text content. The arranged sentence sequence undergoes grammatical checking and correction to ensure fluency and grammatical correctness, correcting any spelling and grammatical errors. A guiding sentence is added at the beginning of the sentence sequence, summarizing the low-matching core topic targeted by this knowledge fragment. The guiding sentence is generated based on a set of keywords related to the core topic. If a semantic transition, identified by a preset semantic transition threshold, exists between adjacent sentences in the sentence sequence, a short connecting word or phrase is inserted to enhance coherence. Semantic transitions are determined by comparing the cosine similarity of the embedding vectors of adjacent sentences to see if it is lower than the semantic transition threshold. The entire sentence sequence is compressed, removing redundant modifiers and retaining the core subject-verb-object or subject-linking verb-complement structure to form a concise knowledge statement. This concise knowledge statement is then encapsulated, generating a unique fragment identifier and associating it with the cross-cluster bridge node identifier of its source, the corresponding core topic identifier, and the target core document node identifier. The encapsulated whole is the knowledge fragment. In essence, a knowledge fragment is a structured data object containing text content and its metadata.

[0030] The knowledge injection module back-injects the constructed knowledge fragment into the attribute representation of the core document node it is connected to. It obtains the current multi-dimensional attribute vector of the target core document node, which is the document node to which the knowledge fragment is connected and needs to be updated. The knowledge fragment is input into a sentence encoder, which is a pre-trained neural network model, to obtain a fixed-dimensional semantic vector representation of the knowledge fragment. The semantic vector representation of the knowledge fragment is then concatenated with the current multi-dimensional attribute vector of the target core document node, forming a new, expanded, higher-dimensional attribute vector. In some embodiments, the concatenation operation can be expressed as the following formula:

[0031] in: This represents the extended new attribute vector formed after concatenation. This represents the current multidimensional attribute vector of the target core document node. A fixed-dimensional semantic vector represents a knowledge fragment, and [;] denotes the concatenation operation of vectors along the dimensional direction. This new attribute vector undergoes dimensionality reduction to restore its dimension to that of the original multi-dimensional attribute vector. The dimensionality-reduced vector is the updated feature vector of the core document node. During dimensionality reduction, linear discriminant analysis (LDA) is used. LDA aims to maximize the discriminability between the core document node connected to the knowledge fragment and other document nodes in the same cluster, as well as between document nodes in other clusters, before and after knowledge fragment injection. Table 1 illustrates an example scenario of feature vector dimensionality changes during knowledge fragment injection.

[0032] Table 1: Examples of Feature Vector Dimension Changes During Knowledge Fragment Back-Injection Process

[0033] In practice, the goal of linear discriminant analysis (LDA) is to find a projection matrix that projects the high-dimensional extended attribute vectors into a low-dimensional space, while minimizing the intra-class divergence and maximizing the inter-class divergence of the core document node samples connected by knowledge fragments. Optionally, principal component analysis (PCA) can be used for dimensionality reduction, retaining most of the original information while reducing dimensionality. The updated core document node feature vectors will replace the original multi-dimensional attribute vectors for subsequent iterative calculations or result output. It can be understood that through this back-injection process, the features of the core document nodes incorporate specific knowledge from cross-cluster bridge nodes, enhancing and expanding their representation.

[0034] See Figure 4 This is a graph showing the rate of change of the core document node set during iterative updates, illustrating the trend of this rate of change in the core document node set during the iterative update process of a journal literature data mining system. The rate of change of core nodes exhibits a rapid decline followed by an exponential decay trend, decreasing continuously from approximately 0.45 in the first iteration to below 0.02 in the tenth iteration. This indicates that the clustering structure of the system undergoes drastic adjustments in the early stages of iteration, and gradually stabilizes with knowledge injection and feature updates. The black dashed line in the graph represents the convergence threshold of 0.05. The system first reaches the threshold in the eighth iteration and continues to decrease to 0.03 and 0.02 in the ninth and tenth iterations, respectively, fully satisfying the convergence condition. In iterations 1-4, the system performs clustering based on the original feature vectors, resulting in unstable core document node partitioning and frequent adjustments to cross-cluster bridge nodes and topic distribution. In iterations 5-7, knowledge fragment injection is completed, feature vectors are optimized and expanded, and the clustering structure begins to converge towards real document associations. After 8-10 iterations, the system reaches convergence and the set of core literature nodes no longer changes significantly. The iteration can then be stopped and the final analysis results can be output.

[0035] In one embodiment of the present invention, after the knowledge injection module performs the knowledge fragment reverse injection operation, the iterative update module triggers a new round of clustering and analysis based on the updated feature vectors. The iterative update module uses the updated feature vectors of all documents to re-execute the density peak clustering calculation process. In a specific implementation, the density peak clustering calculation process adopts the same method as the initial clustering, recalculating the local density of each document node and the minimum distance to nodes with higher local density using the updated feature vectors. Based on these quantities, a distribution map is redrawn to select new core document nodes, and the remaining document nodes are assigned to the clusters belonging to the nearest new core document nodes, thereby generating new clustering results. In the new clustering results, the iterative update module identifies new core document nodes and marginal document nodes.

[0036] The iterative update module recalculates the association strength based on the updated feature vectors and the new clustering structure. For each newly identified core document node, it calculates the comprehensive association score between it and other document nodes within the new cluster, and selects nodes with association strengths higher than a preset threshold to form a new set of strongly associated nodes. The iterative update module then filters from this new set of strongly associated nodes those nodes that have strong associations with different core document nodes throughout historical iterations, identifying them as new cross-cluster bridge nodes. For these new cross-cluster bridge nodes, the iterative update module repeats the steps of topic analysis, matching degree calculation, knowledge fragment extraction, and back-injection. Topic analysis involves modeling the text content blocks of the new cross-cluster bridge nodes to extract core topic distribution vectors. Matching degree calculation compares the matching degree between the core topics of the new cross-cluster bridge nodes and the topic distribution of the new clusters they connect to. Knowledge fragment extraction and back-injection involve constructing knowledge fragments for topics with low matching degrees and integrating their semantic vectors into the updated feature vectors of the associated core document nodes.

[0037] The iterative update module repeatedly performs the process of recalculating, identifying, and analyzing the injection. The iterative update module continues to loop until, in two consecutive iterations, the set change rate of the core document nodes is lower than a preset convergence threshold, at which point the iteration stops. The set change rate of the core document nodes quantifies the degree of change in the identity of the core document nodes between two iterations. In some embodiments, the formula for calculating the set change rate of the core document nodes can be expressed as:

[0038] in: This represents the rate of change of the set of core document nodes between two consecutive iterations. This represents the set of core document nodes determined after the previous iteration. This represents the set of core document nodes determined after the current iteration, denoted by [symbol]. The symbol represents the number of elements in a set. The intersection operation of sets is represented by the symbol [symbol missing]. This represents the union operation of sets. It calculates the rate of change between two consecutive iterations. When all convergence values ​​are below the preset convergence threshold, the iterative update module determines that the system state has stabilized. Optionally, the preset convergence threshold can be set to 0.05 or 0.1. After stopping the iteration, the iterative update module outputs the final document clustering results, the final set of core document nodes, the final set of cross-cluster bridge nodes, and the set of all knowledge fragments derived during the entire iteration process. It can be understood that through multiple iterations, the system continuously optimizes the feature representations of document nodes during the fusion of cross-cluster bridge knowledge, ultimately obtaining a more stable and insightful knowledge structure partition.

[0039] See Figure 5 This is a graph showing the relationship between the core node set change rate and the number of knowledge fragments in a journal literature data mining system during iterations. The core node set change rate rapidly decreased from 0.50 in the first iteration to 0.02 in the tenth iteration, exhibiting an overall exponential decay trend. The system experienced drastic adjustments to the clustering structure in the early stages of iteration; however, with knowledge injection, the core node set gradually stabilized, conforming to convergence logic. The number of knowledge fragments steadily increased from approximately 50 in the first iteration to approximately 530 in the tenth iteration, showing a steady upward trend. This indicates that as iterations progressed, the system continuously accumulated specific knowledge fragments extracted from cross-cluster bridge nodes. In iterations 1-4, the core node change rate rapidly decreased, while the number of knowledge fragments increased rapidly in tandem, indicating that the system was rapidly correcting the clustering structure while simultaneously mining a large amount of cross-cluster knowledge. In iterations 5-8, the core node change rate dropped below 0.10, and the growth rate of the number of knowledge fragments slowed but continued to increase, indicating that the system entered a stable optimization phase, with knowledge injection focusing more on detailed supplementation. After 9-10 iterations, the change rate of the core nodes is below the convergence threshold of 0.05, the number of knowledge fragments tends to level off, the system reaches convergence, the iteration can be stopped and the final result can be output.

[0040] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A data mining and analysis system based on journal articles, characterized in that, include: The data acquisition module acquires the original document metadata and full-text content, and performs preprocessing to identify core document nodes and peripheral document nodes. The node association module calculates the association strength between the current core document node and other document nodes in each iteration, and selects nodes with association strength higher than a preset threshold to form a set of strongly associated nodes. The bridge identification module selects those nodes that have strong associations with different core document nodes in the historical iteration process from the set of strongly associated nodes and marks them as cross-cluster bridge nodes. The topic modeling module obtains all unstructured text content blocks of the marked cross-cluster bridge nodes, performs topic modeling analysis on the text content blocks of the cross-cluster bridge nodes, and extracts the distribution vectors of multiple core topics they carry. The topic matching module compares the topic distribution of different core document nodes connected by the cross-cluster bridge node to the clusters in which they belong, and calculates the matching degree between the core topic distribution vector of the cross-cluster bridge node and the topic distribution of each cluster. The knowledge injection module, for core topics with a matching degree lower than a preset matching degree threshold, locates and extracts a set of key sentences expressing the core topic from the text content blocks of the cross-cluster bridge nodes, constructs knowledge fragments reflecting the specificity of the cross-cluster bridge nodes, and injects the knowledge fragments back into the attribute representation of the core document nodes connected to them, thereby updating the feature vector of the core document nodes.

2. The data mining and analysis system based on journal articles according to claim 1, characterized in that, The identification of core document nodes and peripheral document nodes includes: Obtain the original metadata and full-text content of the target journal's literature database; The original document metadata and full-text content are preprocessed to extract structured document attribute fields and unstructured text content blocks. The structured document attribute fields are processed using the density peak clustering algorithm to identify core document nodes and peripheral document nodes. The preprocessing of the original document metadata and full-text content includes: The title, author list, institution list, publication year, volume, issue and page number, abstract, keyword list and reference list fields are parsed from the original document metadata, and the non-numeric fields are standardized and encoded. The entire text was cleaned to remove headers, footers, figure and table titles, reference paragraphs, and acknowledgments paragraphs. The cleaned text is divided into continuous text blocks, each containing a preset number of paragraphs; Named entity recognition is performed on each text block to identify personal names, organization names, place names, and technical terms. The identified named entities are linked with the author and organization fields parsed from the metadata to establish an entity association network between the text content and the metadata; The preprocessed metadata structure, text content blocks, and entity association network are stored together as a preprocessed document data object that can be directly called in subsequent steps.

3. The data mining and analysis system based on journal articles according to claim 2, characterized in that, The processing of the structured document attribute fields based on the density peak clustering algorithm includes: For each document, a multidimensional attribute vector is constructed, which contains normalized values ​​of fields such as publication year, subject classification code, number of references, number of authors, and number of institutions. For each document, calculate the Euclidean distance between its multidimensional attribute vector and the multidimensional attribute vectors of all other documents; For each document, count the number of other documents whose surrounding distance is less than the specified neighborhood radius, and use this count as the local density of the corresponding document node; For each document, find all other document nodes with a local density higher than the currently processed document node, and calculate the minimum distance between the currently processed document node and all document nodes with a higher local density. If there are no other document nodes with a local density higher than the currently processed document node, set the aforementioned minimum distance to the preset maximum value. Draw a two-dimensional distribution map of the local density and minimum distance of all document nodes, and select those document nodes whose local density and minimum distance are both higher than their respective preset quantile values ​​as the initial core document nodes; The remaining document nodes are assigned to the clusters of the nearest core document nodes, thus completing the initial partitioning of the document set.

4. The data mining and analysis system based on journal articles according to claim 3, characterized in that, The calculation of the association strength between the current core document node and other document nodes includes: For the core document node currently being processed, obtain its multidimensional attribute vector; For any other document node within the current cluster, obtain its multidimensional attribute vector; Calculate the cosine similarity between the multidimensional attribute vectors of the core document node and the other document nodes, and use it as the basic correlation. If there is a direct citation or cited relationship between the other document nodes and the core document node, then set a citation relationship bonus coefficient. Determine whether the other document nodes and the core document nodes share at least one identical keyword. If they do, set a keyword overlap bonus coefficient based on the number of shared keywords. The weighted sum of the basic relevance, citation relationship bonus coefficient, and keyword overlap bonus coefficient yields a comprehensive score, which represents the relevance strength between the current core document node and any other document node within the current cluster.

5. The data mining and analysis system based on journal articles according to claim 4, characterized in that, Topic modeling analysis is performed on the text content blocks of the cross-cluster bridge nodes, including: The full text of the cross-cluster bridge node is segmented, stop words are removed, and lemmatization is performed to obtain a clean word sequence. Based on the pure term sequence, construct the term-document co-occurrence matrix of the cross-cluster bridge node documents; The term-document co-occurrence matrix is ​​decomposed using the hidden Dirichlet distribution model to estimate the document-topic distribution matrix and the topic-term distribution matrix; From the document-topic distribution matrix, extract the probability values ​​of the cross-cluster bridge node documents belonging to each topic; The topics with the highest probability values ​​are selected as the core topics of the cross-cluster bridge nodes; For each core topic, find the terms with the highest probability of being associated with the core topic from the topic-term distribution matrix, and form the topic term set of the core topic; Each core theme, its corresponding probability value, and the set of keywords are combined to form a core theme distribution vector.

6. The data mining and analysis system based on journal articles according to claim 5, characterized in that, The calculation of the matching degree between the core topic distribution vectors of the cross-cluster bridge nodes and the topic distributions of each cluster includes: For each cluster containing a core document node connected by a cross-cluster bridge node, the topic distribution of all documents within that cluster is statistically analyzed. Calculate the average probability of all documents within the cluster for each topic to form the average topic distribution vector of the cluster; For each core topic distribution vector of a cross-cluster bridge node, calculate the Jaccard similarity coefficient between it and the average topic distribution vector of each connected cluster, the Jaccard similarity coefficient being calculated based on the topic word sets of the two vectors; The calculated Jaccard similarity coefficient is multiplied by the probability value of the core topic in the core topic distribution vector to obtain a comprehensive matching score. This overall matching score serves as the degree of matching between the core topic and the corresponding cluster.

7. The data mining and analysis system based on journal articles according to claim 6, characterized in that, Locate and extract a set of key sentences expressing the core theme from the text content blocks of the cross-cluster bridge nodes, including: For each core topic with a matching degree lower than a preset matching degree threshold, obtain its topic keyword set; The full text of the cross-cluster bridge node is segmented into independent sentences; For each sentence, its semantic relevance score with the core topic word set is calculated. The semantic relevance score is obtained by calculating the cosine similarity between the embedding vector of the word in the sentence and the average embedding vector of the topic word set. Sentences with semantic relevance scores higher than the sentence selection threshold are selected to form a set of candidate key sentences; Sentences in the candidate key sentence set are sorted according to their semantic coherence with the core document node summary and the sentence's position in the original text. Select a predetermined number of sentences from the sorted list to form the final set of key sentences used to construct knowledge fragments.

8. The data mining and analysis system based on journal articles according to claim 7, characterized in that, The construction of knowledge fragments reflecting the specificity of cross-cluster bridge nodes includes: Arrange all sentences in the aforementioned set of key sentences in the natural order in which they appear in the original text; The sorted sentence sequence is grammatically checked and corrected to ensure fluency. At the beginning of the sentence sequence, add a guiding sentence that summarizes the low-matching core topic that this knowledge fragment targets; If there is a semantic transition identified by a preset semantic transition threshold between adjacent sentences in a sentence sequence, a short conjunction or phrase is inserted to enhance coherence. The entire sentence sequence is compressed, redundant modifiers are removed, and the core subject-verb-object or subject-linking verb-complement structure is retained to form a concise knowledge statement; This concise knowledge statement is encapsulated to generate a unique fragment identifier, which is then associated with the cross-cluster bridge node identifier of its source, the corresponding core topic identifier, and the target core document node identifier. The encapsulated whole is the knowledge fragment.

9. A data mining and analysis system based on journal articles according to claim 8, characterized in that, The process of back-injecting knowledge fragments into the attribute representation of the core document nodes they are connected to includes: Obtain the current multidimensional attribute vector of the target core document node; The knowledge fragment is input into a sentence encoder to obtain a fixed-dimensional semantic vector representation of the knowledge fragment; The semantic vector representation of the knowledge fragment is concatenated with the current multidimensional attribute vector of the target core document node to form a new attribute vector that is extended and has higher dimensions. The new attribute vector is then dimensionality-reduced to restore its dimension to that of the original multidimensional attribute vector. The dimensionality-reduced vector is the updated core document node feature vector. During the dimensionality reduction process, linear discriminant analysis is used to perform projection transformation with the goal of maximizing the distinguishability between the core document nodes connected to the knowledge fragment and other document nodes in the same cluster, as well as between document nodes in other clusters, before and after knowledge fragment injection.

10. A data mining and analysis system based on journal articles according to claim 9, characterized in that, It also includes an iterative update module, used to trigger a new round of clustering and analysis based on the updated feature vectors after the knowledge injection module performs the knowledge fragment back-injection operation, including: Using the updated feature vectors from all the literature, the calculation process based on density peak clustering is re-executed; The new clustering results identify new core document nodes and peripheral document nodes; Recalculate the association strength and identify new sets of strongly associated nodes and cross-cluster bridge nodes; For new cross-cluster bridge nodes, repeat the steps of topic analysis, matching degree calculation, knowledge fragment extraction and reverse injection; The process of recalculating, identifying, and analyzing the injection is repeated until the rate of change of the core document node set is lower than the preset convergence threshold in two consecutive iterations. Then the iteration stops, and the final document cluster, core nodes, bridge nodes, and derived knowledge fragment set are output.

Images