Large model theme identification method of research leading edge identification system based on multi-dimensional data

Abstract

The invention relates to a large model theme recognition method of a research leading edge recognition system based on multi-dimensional data, and belongs to the field of research leading edge recognition. The large model theme recognition method comprises the steps of multi-dimensional data integration, large model semantic embedding, multi-modal data fusion, theme clustering, theme screening and optimization, research leading edge recognition and visual display. According to the method, the accuracy and efficiency of research leading edge recognition are improved, the strong semantic understanding capability of the large model is utilized, the defects of an existing method are effectively overcome, and the theme recognition precision is improved. Multi-source heterogeneous data can be processed, multiple data sources are integrated, and the dynamic change of the research leading edge is reflected more comprehensively. The evolution trend of the research front can be captured, and the evolution trend of the research front can be effectively captured by analyzing indexes such as occurrence frequency and growth trend of themes. Visual display is provided, and a user can understand and use the system conveniently.

Description

Technical Field

[0001] The present invention relates to the field of research frontier identification, and in particular to a large-model topic identification method of a research frontier identification system based on multidimensional data. Background Art

[0002] Existing methods for identifying research frontiers mainly rely on keyword co-occurrence analysis, literature citation network analysis, etc., which have the following shortcomings: (1) It is difficult to process multi-source heterogeneous data; (2) The recognition accuracy of topics is limited, and topic drift or topic redundancy is prone to occur; (3) It is difficult to capture the evolution trend of research frontiers. Summary of the invention

[0003] Purpose of the invention: To provide a large-model topic identification method for a research frontier identification system based on multidimensional data to solve the above-mentioned problems existing in the prior art.

[0004] Technical solution: A large-model topic identification method for a research frontier identification system based on multidimensional data, comprising the following steps:

[0005] S1. Collect multidimensional data, including literature data, fund project data and network data, and clean and preprocess the collected data;

[0006] S2. Use a pre-trained large language model to convert text information in multi-dimensional data into high-dimensional semantic vectors;

[0007] S3. For non-text data in the collected data, use corresponding technologies to perform multimodal data fusion;

[0008] S4, clustering the semantic vectors using the corresponding topic clustering algorithm to identify the research topic;

[0009] S5. Screen and optimize the clustering results, remove redundant topics and noise topics, and name and describe the topics;

[0010] S6. Identify research frontiers based on the frequency of occurrence, growth trend, and other indicators of the topic;

[0011] S7. Present the identified research frontiers in a visual way.

[0012] In a further embodiment, the large model in step S2 is a pre-trained large language model.

[0013] In a further embodiment, the topic clustering algorithm in step S4 is an adaptive clustering algorithm.

[0014] In a further embodiment, the research frontier identification in step S6 uses multiple indicators to measure the importance of the research frontier, including the number of citations and the degree of attention of the topic.

[0015] In a further embodiment, the training and component steps of the pre-trained large language model include data reception, model architecture design, pre-training task selection, model initialization, training configuration, model training, model fine-tuning, evaluation and testing, model preservation and deployment, and continuous learning and updating.

[0016] In a further embodiment, the model structure design includes determining the structure of the model, designing the number of layers of the model, the size of the hidden layer, and the number of attention heads;

[0017] The pre-training tasks include masked language model task training and next sentence prediction model task training;

[0018] Model initialization uses either random initialization or predefined initialization methods;

[0019] Training configuration includes setting training parameters and configuring hardware resources.

[0020] In a further embodiment, model training uses pre-training tasks and a large amount of data to train the model, and monitors losses and performance indicators during the training process;

[0021] Model fine-tuning involves fine-tuning the pre-trained model on a specific task, fine-tuning the learning rate involved, and the type of optimizer used.

[0022] Beneficial effects: The present invention relates to a large-model topic identification method for a research frontier identification system based on multidimensional data, and the field of research frontier identification includes multiple steps such as multidimensional data integration, large-model semantic embedding, multimodal data fusion, topic clustering, topic screening and optimization, research frontier identification and visualization display. The present invention improves the accuracy and efficiency of research frontier identification, and utilizes the powerful semantic understanding ability of the large model to effectively solve the shortcomings of the existing methods and improve the accuracy of topic identification. It can process multi-source heterogeneous data, integrate multiple data sources, and more comprehensively reflect the dynamic changes of the research frontier. It can capture the evolution trend of the research frontier, and can effectively capture the evolution trend of the research frontier by analyzing indicators such as the frequency of occurrence and growth trend of the topic. It provides a visual display to facilitate user understanding and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a top view of the present invention as a whole. DETAILED DESCRIPTION

[0024] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0025] The applicant believes that existing research frontier identification methods mainly rely on keyword co-occurrence analysis, literature citation network analysis, etc., and have the following shortcomings: it is difficult to handle multi-source heterogeneous data, the recognition accuracy of topics is limited, topic drift or topic redundancy is prone to occur, and it is difficult to capture the evolutionary trend of research frontiers.

[0026] To this end, the applicant designed a large-model topic identification method for a research frontier identification system based on multidimensional data, which improved the accuracy and efficiency of research frontier identification. By using the powerful semantic understanding ability of the large model, it effectively solved the shortcomings of existing methods and improved the accuracy of topic identification. It can process multi-source heterogeneous data, integrate multiple data sources, and more comprehensively reflect the dynamic changes of research frontiers. It can capture the evolution trend of research frontiers and effectively capture the evolution trend of research frontiers by analyzing indicators such as the frequency of occurrence and growth trend of topics. It provides visual display to facilitate user understanding and use.

[0027] The present invention relates to a large model topic identification method based on a multidimensional data research frontier identification system, which comprises the following steps:

[0028] Step 1: Collect multidimensional data, including literature data, fund project data and network data, and clean and preprocess the collected data;

[0029] Step 2: Use a pre-trained large language model to convert text information in multi-dimensional data into a high-dimensional semantic vector;

[0030] Step 3: For non-text data in the collected data, use corresponding technology to perform multimodal data fusion;

[0031] Step 4: Use the corresponding topic clustering algorithm to cluster the semantic vectors and identify the research topics;

[0032] Step 5: Screen and optimize the clustering results, remove redundant topics and noise topics, and name and describe the topics;

[0033] Step 6: Identify research frontiers based on the frequency of occurrence, growth trend, and other indicators of the topic;

[0034] Step seven: Present the identified research frontiers in a visual way.

[0035] The large model in step 2 is a pre-trained large language model.

[0036] The topic clustering algorithm in step 4 is an adaptive clustering algorithm.

[0037] The research frontier identification in step six uses a variety of indicators to measure the importance of the research frontier, including the number of citations and the degree of attention paid to the topic.

[0038] The training and component steps of the pre-trained large language model include data reception, model architecture design, pre-training task selection, model initialization, training configuration, model training, model fine-tuning, evaluation and testing, model preservation and deployment, and continuous learning and updating.

[0039] Model structure design includes determining the structure of the model, designing the number of layers, the size of the hidden layer, and the number of attention heads;

[0040] The pre-training tasks include masked language model task training and next sentence prediction model task training;

[0041] Model initialization uses either random initialization or predefined initialization methods;

[0042] Training configuration includes setting training parameters and configuring hardware resources.

[0043] Model training uses pre-training tasks and a large amount of data to train the model, and monitors the loss and performance indicators during the training process;

[0044] Model fine-tuning involves fine-tuning the pre-trained model on a specific task, fine-tuning the learning rate involved, and the type of optimizer used.

[0045] The present invention proposes a large model topic identification method based on a research frontier identification system of multidimensional data. The method utilizes the powerful semantic understanding ability of the large model to effectively solve the shortcomings of the existing methods. The method comprises the following steps:

[0046] (1) Multidimensional data integration: Collect multidimensional data, including but not limited to:

[0047] Literature data: paper title, abstract, keywords, references, etc.;

[0048] Patent data: patent title, abstract, claims, etc.;

[0049] Fund project data: project name, abstract, research content, etc.;

[0050] Network data: news reports, blogs, forums and other information in related fields. Clean and pre-process the collected data, such as denoising and normalization.

[0051] (2) Large model semantic embedding: Use pre-trained large language models (such as BERT, GPT, etc.) to convert text information in multi-dimensional data into high-dimensional semantic vectors. This step uses the powerful semantic understanding ability of the large model to convert text information into computer-processable numerical representations, effectively avoiding the limitations of methods such as keyword matching.

[0052] (3) Multimodal data fusion: If there is non-text data (e.g., images, charts, etc.), appropriate techniques are used for multimodal data fusion, such as converting image data into feature vectors and fusing them with text vectors.

[0053] (4) Topic clustering: Use a suitable clustering algorithm (e.g., K-Means, DBSCAN, hierarchical clustering, etc.) to cluster semantic vectors and identify research topics. The choice of clustering algorithm should be adjusted according to the characteristics of the data and the research objectives. The present invention preferably uses an adaptive clustering algorithm that can automatically adjust clustering parameters according to the distribution of the data to improve clustering accuracy.

[0054] (5) Topic screening and optimization: Screen and optimize the clustering results, remove redundant topics and noise topics, and name and describe the topics. This step can combine the knowledge of domain experts to improve the accuracy of topic identification.

[0055] (6) Research frontier identification: Identify research frontiers based on the frequency of occurrence, growth trend, and other indicators of the topic. The present invention can use a variety of indicators to measure the importance of research frontiers, such as the number of citations and attention paid to the topic.

[0056] (7) Visual display: The identified research frontiers are presented in a visual way, such as knowledge graphs, heat maps, etc., to facilitate user understanding and use.

[0057] In a further preferred embodiment, pre-training a large language model involves multiple steps, specifically including:

[0058] a. Data collection and preprocessing:

[0059] Collect large amounts of text data, which can come from books, articles, web pages, etc.

[0060] Clean the data and remove useless information, such as HTML tags, special characters, etc.

[0061] Tokenization is the process of dividing text into words, phrases, or other meaningful units.

[0062] Build the vocabulary, which determines the set of words or tokens that the model will use.

[0063] b. Model architecture design:

[0064] Determine the model architecture, such as Transformer, BERT, etc.

[0065] Design model parameters such as the number of layers, size of hidden layers, number of attention heads, etc.

[0066] c. Pre-training task selection:

[0067] Select appropriate pre-training tasks, such as Masked Language Model (MLM), Next Sentence Prediction (NSP), etc.

[0068] These tasks can help the model learn the universal features and structure of language.

[0069] d. Model initialization:

[0070] Initialize the model's weights, typically using random initialization or a predefined initialization method.

[0071] e. Training configuration:

[0072] Set training parameters, such as learning rate, batch size, epochs, etc.

[0073] Configure hardware resources, such as GPU, TPU, etc., to accelerate the training process.

[0074] f. Model training:

[0075] Train the model using pre-training tasks and large amounts of data.

[0076] Monitor loss and performance metrics during training, such as accuracy, perplexity, etc.

[0077] g. Model fine-tuning:

[0078] Fine-tune a pre-trained model on a specific task to improve its performance on that task.

[0079] Fine-tuning may involve adjusting the learning rate, using a different optimizer, etc.

[0080] h. Evaluation and testing:

[0081] Evaluate the performance of the model on the validation set and the test set.

[0082] Adjust model parameters or training strategies based on the evaluation results.

[0083] Model saving and deployment:

[0084] Save the trained model weights for subsequent use.

[0085] i. Deploy the model to a production environment, such as an API, application, etc.

[0086] j.Continuous learning and updating:

[0087] The model is continuously updated and optimized based on user feedback and new data.

[0088] The model may need to be retrained periodically to adapt to changes in language.

[0089] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A large-model topic identification method for a research frontier identification system based on multidimensional data, characterized in that The following steps are involved: S1. Collect multidimensional data, including literature data, fund project data and network data, and clean and preprocess the collected data; S2, using a pre-trained large language model to convert text information in multi-dimensional data into high-dimensional semantic vectors; S3. For non-text data in the collected data, use corresponding technologies to perform multimodal data fusion; S4, clustering the semantic vectors using the corresponding topic clustering algorithm to identify the research topic; S5. Screen and optimize the clustering results, remove redundant topics and noise topics, and name and describe the topics; S6. Identify research frontiers based on the frequency of occurrence, growth trend, and other indicators of the topic; S7. Present the identified research frontiers in a visual way.

2. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 1 is characterized by: The large model in step S2 is a pre-trained large language model.

3. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 2 is characterized by: The topic clustering algorithm in step S4 is an adaptive clustering algorithm.

4. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 3 is characterized by: The research frontier identification in step S6 uses multiple indicators to measure the importance of the research frontier, including the number of citations and the degree of attention of the topic.

5. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 2 is characterized by: The training and component steps of the pre-trained large language model include data reception, model architecture design, pre-training task selection, model initialization, training configuration, model training, model fine-tuning, evaluation and testing, model preservation and deployment, and continuous learning and updating.

6. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 5, characterized in that: Model structure design includes determining the structure of the model, designing the number of layers, the size of the hidden layer, and the number of attention heads; The pre-training tasks include masked language model task training and next sentence prediction model task training; Model initialization uses either random initialization or predefined initialization methods; Training configuration includes setting training parameters and configuring hardware resources.

7. The large-model topic identification method of the research frontier identification system based on multidimensional data according to claim 5 is characterized by: Model training uses pre-training tasks and a large amount of data to train the model, and monitors the loss and performance indicators during the training process; Model fine-tuning involves fine-tuning the pre-trained model on a specific task, fine-tuning the learning rate involved, and the type of optimizer used.

Images