Selected question analysis model construction method and device based on journal paper big data
A topic analysis model for journal articles uses big data and machine learning to enhance topic recommendation services, addressing inefficiencies in existing methods by providing accurate and timely recommendations for researchers and editors.
Patent Information
- Application Number
- CN202410960876.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-12-03
AI Technical Summary
Existing methods for journal topic selection lack the use of big data analysis, leading to unreliable and inefficient topic recommendation services for researchers and editors, particularly in interdisciplinary research environments.
A method and device for constructing a topic analysis model based on journal article big data, involving data extraction, preprocessing, feature extraction, and machine learning to build a model that recommends topics and generates paper outlines.
Provides accurate and timely topic recommendations, enhancing the success rate of paper publication by leveraging big data cleaning, mining, and feature extraction, and optimizing the model for real-time updates and improved decision-making.
Smart Images

Figure CN119067098A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for constructing a topic analysis model based on journal article big data, an electronic device, a storage medium, and a computer program product. Background Art
[0002] With the deepening and expansion of academic research, the competition for journal article publication is becoming increasingly fierce. This requires researchers to consider not only innovation but also practical feasibility and market demand when selecting topics. In the current academic environment, interdisciplinary research has gradually become the mainstream. However, this also brings challenges, namely how to find common research points in different disciplinary backgrounds so that the paper has both depth and breadth. With the development of information technology, there are more and more channels for obtaining information, but this also increases the difficulty of screening and organizing information. How to effectively use this information to select topics is a problem that researchers need to face. Different journals may have different requirements and standards for paper topic selection, which requires researchers to consider the characteristics and requirements of the target journal when selecting topics in order to improve the success rate of paper publication.
[0003] Therefore, with the development of science and technology and the deepening of academic research, the number of journal papers has exploded. How to quickly and accurately analyze and explore the hot spots and trends has become a major challenge facing the academic community. Summary of the invention
[0004] In view of the above problems, this application is proposed to provide a method and device for constructing a topic analysis model based on journal article big data, an electronic device, a storage medium and a computer program product to overcome the above problems or at least partially solve the above problems. The technical solution is as follows:
[0005] In a first aspect, a method for constructing a topic analysis model based on journal article big data is provided, the method comprising:
[0006] Based on the big data of journal articles collected by the preset data platform, extract the journal article data of the set time interval;
[0007] Preprocessing the extracted journal paper data to obtain preprocessed paper data;
[0008] By deeply processing the preprocessed paper data, features including the paper's subject keywords, citation times, and citation frequency are extracted;
[0009] Utilize the extracted features and combine them with the preset machine learning algorithm to build a topic analysis model.
[0010] In a possible implementation, extracting journal paper data within a set time interval based on the journal paper big data collected by a preset data platform includes:
[0011] Determine the goals and needs of data collection;
[0012] Develop a data collection plan;
[0013] According to the data collection plan, data collection goals and requirements, data collection is carried out to extract journal article data within a set time period collected by the preset data platform;
[0014] Implement data collation, organize the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
[0015] In a possible implementation, the preprocessing includes one or more of the following:
[0016] Classify the journal article data according to subject information and / or industry information;
[0017] Handling missing values and outliers in the journal article data;
[0018] Deleting duplicate rows or records in the journal article data to ensure that each record of the journal article data is unique;
[0019] The data range, dimension, format and type of the journal article data are unified to facilitate subsequent data calculation and analysis.
[0020] In a possible implementation, the extracted features are combined with a preset machine learning algorithm to construct a topic analysis model, including:
[0021] Select a preset machine learning algorithm;
[0022] Divide the extracted features into training set and validation set;
[0023] Use the training set to train the preset machine learning algorithm to obtain a preliminary topic analysis model;
[0024] The validation set is used to optimize and adjust the parameters of the preliminary topic analysis model to obtain the optimized topic analysis model.
[0025] In a possible implementation, after constructing a topic analysis model by using the extracted features in combination with a preset machine learning algorithm, the method further includes:
[0026] Deploy the topic analysis model based on actual application scenarios;
[0027] Acquire publishing hotspots within a set time period based on subject information;
[0028] Select a publishing hotspot, pass it into the topic analysis model, and obtain recommended topics;
[0029] Select the recommended topic, pass it into the topic analysis model, and obtain the paper title;
[0030] Select a paper topic and generate a paper outline.
[0031] In a second aspect, a device for constructing a topic analysis model based on journal article big data is provided, the device comprising:
[0032] A data collection unit, used to extract journal article data within a set time interval based on the journal article big data collected by a preset data platform;
[0033] A data preprocessing unit, used for preprocessing the extracted journal paper data to obtain preprocessed paper data;
[0034] A feature extraction unit is used to extract features including the subject keywords, citation times, and citation frequency of the paper by performing deep processing on the preprocessed paper data;
[0035] The model building unit is used to build a topic analysis model by utilizing the extracted features in combination with a preset machine learning algorithm.
[0036] In a possible implementation, the data acquisition unit is further used for:
[0037] Determine the goals and needs of data collection;
[0038] Develop a data collection plan;
[0039] According to the data collection plan, data collection goals and requirements, data collection is carried out to extract journal article data within a set time period collected by the preset data platform;
[0040] Implement data collation, organize the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
[0041] In a possible implementation, the preprocessing includes one or more of the following:
[0042] Classify the journal article data according to subject information and / or industry information;
[0043] Handling missing values and outliers in the journal article data;
[0044] Deleting duplicate rows or records in the journal article data to ensure that each record of the journal article data is unique;
[0045] The data range, dimension, format and type of the journal article data are unified to facilitate subsequent data calculation and analysis.
[0046] In a possible implementation, the model building unit is further used to:
[0047] Select a preset machine learning algorithm;
[0048] Divide the extracted features into training set and validation set;
[0049] Use the training set to train the preset machine learning algorithm to obtain a preliminary topic analysis model;
[0050] The validation set is used to optimize and adjust the parameters of the preliminary topic analysis model to obtain the optimized topic analysis model.
[0051] In a possible implementation manner, the device further includes a recommendation unit, configured to:
[0052] Deploy the topic analysis model based on actual application scenarios;
[0053] Acquire publishing hotspots within a set time period based on subject information;
[0054] Select a publishing hotspot, pass it into the topic analysis model, and obtain recommended topics;
[0055] Select the recommended topic, pass it into the topic analysis model, and obtain the paper title;
[0056] Select a paper topic and generate a paper outline.
[0057] According to a third aspect, an electronic device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any of the above-mentioned methods for constructing a topic analysis model based on journal article big data.
[0058] In a fourth aspect, a storage medium is provided, which stores a computer program, wherein the computer program is configured to execute any of the above-mentioned methods for constructing a topic analysis model based on journal article big data when running.
[0059] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program is configured to execute any of the above-mentioned methods for constructing a topic analysis model based on journal article big data when running.
[0060] By means of the above-mentioned technical scheme, the embodiment of the present application provides a method and device for constructing a topic analysis model based on journal article big data, an electronic device, a storage medium and a computer program product. The method extracts journal article data of a set time period based on the journal article big data collected by a preset data platform; preprocesses the extracted journal article data to obtain preprocessed article data; extracts features including the subject keywords, number of citations and frequency of citations of the article by deep processing the preprocessed article data; and constructs a topic analysis model by using the extracted features in combination with a preset machine learning algorithm. It can be seen that this embodiment is based on the big data of journal papers collected by the preset data platform, and constructs a topic analysis model by cleaning the data, data mining, and feature extraction, updates the data in real time, and continuously optimizes the method; recommends topics to researchers, editors, and authors based on the model training data, and provides a basis for scientific topic decisions of researchers, librarians, and journal editors; and, the features extracted from the analysis results of this embodiment include the subject keywords, number of citations, and citation frequency of the paper, which reflect the influence and academic value of the paper, etc. These features provide strong support for building an intelligent topic analysis model; in addition, this embodiment constructs a topic analysis model based on the extracted features, and realizes topic recommendation for paper writing through the selection of training data sets and the optimization of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.
[0062] Figure 1 A flowchart of a method for constructing a topic analysis model based on journal article big data provided in an embodiment of the present application is shown;
[0063] Figure 2 A data collection flow chart provided by an embodiment of the present application is shown;
[0064] Figure 3 A data preprocessing flow chart provided in an embodiment of the present application is shown;
[0065] Figure 4 A flowchart of constructing a topic analysis model provided in an embodiment of the present application is shown;
[0066] Figure 5 The following is a flowchart of the topic analysis application provided by the embodiment of the present application;
[0067] Figure 6 A schematic diagram of publishing hotspot input provided by an embodiment of the present application is shown;
[0068] Figure 7A schematic diagram showing hotspot results of selected disciplines provided by an embodiment of the present application is shown;
[0069] Figure 8 A schematic diagram showing a topic recommendation result provided by an embodiment of the present application;
[0070] Fig. 9 A schematic diagram showing the recommended and generated paper titles provided in an embodiment of the present application is shown;
[0071] Fig.10 A structural diagram of a topic analysis model construction device based on journal article big data provided in an embodiment of the present application is shown;
[0072] Fig.11 A structural diagram of a topic analysis model construction device based on journal article big data provided by another embodiment of the present application is shown;
[0073] Fig.12 A structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0074] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.
[0075] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such use is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to".
[0076] Most of the existing topic selection methods in China currently rely on crawling books or journal articles, and then classifying the captured books or journal articles according to the classification method, and sorting the titles of books by common words and high-frequency words from high to low according to different types of books, but the data quality and authenticity cannot be guaranteed. In addition, these methods usually lack big data cleaning, data mining, model building and tuning, and cannot conduct in-depth analysis of the content and contextual information of the literature. They lack data mining and hot spot analysis based on publishing big data, and cannot provide real-time and accurate topic recommendation services to scientific researchers, librarians and journal editors, resulting in a low success rate of publication of topic papers and an inability to provide effective services for scientific and technological innovation.
[0077] At present, no method for constructing a journal topic analysis model based on journal article big data has been found in existing patents. Similar patents include:
[0078] Known patent 1: "A method and system for obtaining hot topics for the publishing industry" may include the following steps: S0 captures books in real time, and separates the captured books according to the book classification method, and sorts the book titles by common words and high-frequency words from high to low according to different types of books to obtain hot and cold words; establishes a hot word library of expert subject cutting-edge knowledge; editors input and obtain intended words through editing or interactive games; S1 obtains hot information from the material center, analyzes and generates the first candidate hot topic; S2 screens the first candidate hot topic according to the publishing direction and attention rules of different publishing units, and obtains the target hot topic suitable for the publishing unit; S3 pushes the obtained target hot topic to the media terminal of the corresponding publishing unit.
[0079] Known Patent 2: "Topic recommendation method, device, equipment and readable storage medium" may include the following steps: obtaining the user's topic needs and the hot information of major social platforms; determining at least one recommended topic based on the hot information of the major social platforms and the user's topic needs; for each recommended topic, determining works related to the recommended topic from the major social platforms; outputting each of the recommended topics and works related to each of the recommended topics for user reference.
[0080] The disadvantage of the prior art is that, in the known patent 1, the system's professionalism and authenticity are relatively poor, and it lacks massive paper data and journal information as support. It only performs simple analysis through data crawling, and the data reliability and credibility are poor.
[0081] Among the known patents 2, the system is less academic and only relies on the aggregation and recommendation of hot information from major platforms, and is unable to provide professional topic selection guidance and decision-making support to scientific researchers.
[0082] With the rapid development of natural language processing and machine learning technologies, automated processing and intelligent proofreading have become effective ways to solve this problem. Natural language processing technology can realize automatic understanding and analysis of text content. Machine learning technology can track real-time journal hot spots, subject hot spots, etc. through training models, conduct comprehensive analysis, recommend publishing hot spots, journal topics, and recommend related titles based on the topics, and recommend author literature that can be referenced for the topic.
[0083] In addition, the development of big data technology and cloud computing has provided powerful computing and storage capabilities for paper data analysis. Cloud services can achieve high-performance computing for large-scale document processing and analysis, while facilitating academic collaboration and resource sharing.
[0084] This invention aims to solve this problem by constructing a journal topic selection analysis model based on journal article big data to provide scientific topic selection decision-making basis for researchers, librarians and journal editors. Figure 1 As shown, the method for constructing a topic analysis model based on journal article big data may include the following steps S101 to S104:
[0085] Step S101, extracting journal paper data within a set time interval based on the journal paper big data collected by a preset data platform;
[0086] Step S102, preprocessing the extracted journal paper data to obtain preprocessed paper data;
[0087] Step S103, extracting features including the subject keywords, citation times, and citation frequency of the paper by performing deep processing on the preprocessed paper data;
[0088] Step S104, using the extracted features in combination with a preset machine learning algorithm, a topic analysis model is constructed.
[0089] In the above embodiment, the preset data platform may be a data platform such as HowNet, and this embodiment does not limit this. The set time interval here can be set according to actual needs, for example, the set time interval is the past three years or the past five years, and this embodiment does not limit this.
[0090] This embodiment is based on the big data of journal papers collected by the preset data platform, and constructs a topic analysis model by cleaning, mining and extracting features, updates data in real time, and continuously optimizes methods; recommends topics to researchers, editors and authors based on model training data, and provides a basis for scientific topic decisions of researchers, librarians and journal editors; and, the features extracted from the analysis results in this embodiment include the subject keywords, number of citations, citation frequency and other features of the paper, which reflect the influence and academic value of the paper, etc. These features provide strong support for building an intelligent topic analysis model; in addition, this embodiment constructs a topic analysis model based on the extracted features, and realizes topic recommendation for paper writing through the selection of training data sets and the optimization of model parameters.
[0091] The present application provides a possible implementation method. The above step S101 extracts journal paper data of a set time interval based on the journal paper big data collected by the preset data platform, which may specifically include the following steps A1 to A4:
[0092] Step A1, determine the goals and requirements of data collection;
[0093] Step A2, formulate a data collection plan;
[0094] Step A3, combining the data collection plan, data collection objectives and requirements, implement data collection, and extract journal article data within a set time period collected by the preset data platform;
[0095] Step A4, implement data sorting, sort the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
[0096] In this embodiment, data collection is implemented in combination with data collection plans and target requirements to ensure the timeliness and relevance of data and improve the efficiency of data collection.
[0097] A possible implementation is provided in an embodiment of the present application. The preprocessing mentioned in step S102 above may include one or more of the following:
[0098] Classify the journal article data according to subject information and / or industry information;
[0099] Handling missing values and outliers in the journal article data;
[0100] Deleting duplicate rows or records in the journal article data to ensure that each record of the journal article data is unique;
[0101] The data range, dimension, format and type of the journal article data are unified to facilitate subsequent data calculation and analysis.
[0102] This embodiment classifies journal article data according to subject information and / or industry information, which can make the data more orderly and facilitate subsequent retrieval and analysis; by processing missing values and outliers, the integrity and accuracy of the data can be improved and errors in the analysis process can be reduced; duplicate rows or records are deleted to ensure the uniqueness of each record, which helps to avoid data redundancy and deviations in analysis results; the data range, dimension, format, and type are unified so that the data is consistent during calculation and analysis, reducing calculation errors caused by inconsistent data formats.
[0103] The present application embodiment provides a possible implementation method. In the above step S104, the extracted features are combined with a preset machine learning algorithm to construct a topic analysis model, which may specifically include the following steps B1 to B4:
[0104] Step B1, selecting a preset machine learning algorithm;
[0105] Step B2, dividing the extracted features into a training set and a validation set;
[0106] Step B3, using the training set to train the preset machine learning algorithm to obtain a preliminary topic analysis model;
[0107] Step B4, using the validation set to optimize and adjust the parameters of the preliminary topic analysis model to obtain an optimized topic analysis model.
[0108] In the above embodiments, the preset machine learning algorithm may be TextRank (text ranking), LDA (Latent Dirichlet Allocation), TF-IDF (Term Frequency-Inverse Document Frequency), etc., and this embodiment does not limit this.
[0109] In this embodiment, the extracted features are divided into a training set and a validation set, which can ensure the effectiveness and generalization ability of the model training; the preset machine learning algorithm is trained using the training set to quickly build a preliminary topic analysis model; the preliminary topic analysis model is evaluated using the validation set to detect the accuracy and generalization ability of the model; through the training and optimization process, the optimized topic analysis model obtained has better stability and robustness in practical applications; the optimized model can more accurately predict or analyze topics and improve the quality and efficiency of decision-making.
[0110] The present application provides a possible implementation method. After the extracted features are used in step S104 to build a topic analysis model in combination with a preset machine learning algorithm, the following steps C1 to C5 may be further included:
[0111] Step C1, deploy the topic analysis model based on the actual application scenario;
[0112] Step C2, obtaining the publishing hot spots in a set time period according to the subject information;
[0113] Step C3, select a publishing hotspot, input the publishing hotspot into the topic analysis model, and obtain a recommended topic;
[0114] Step C4, select the recommended topic, input the recommended topic into the topic analysis model, and obtain the paper title;
[0115] Step C5, select the paper title and generate the paper outline.
[0116] The entire process of this embodiment is automated from topic analysis to paper outline generation, which reduces manual intervention and improves work efficiency and consistency. In addition, through automated topic analysis and paper outline generation, researchers can determine research directions and write papers more quickly, shortening the research cycle.
[0117] The above introduces Figure 1 There are various implementation methods for each link of the illustrated embodiment. The following will further illustrate the method for constructing a topic analysis model based on journal article big data in the embodiment of the present application through specific embodiments.
[0118] This embodiment is based on the journal article big data collected by HowNet, and builds a topic analysis model through data cleaning, data mining, and feature extraction, and updates the data in real time and continuously optimizes the method. Based on the model training data, topic selection is recommended to researchers, editors, and authors, providing a basis for scientific topic selection decisions for researchers, librarians, and journal editors.
[0119] Specifically, the method may include the following steps:
[0120] 1) Data collection: Based on the big data of journal papers collected by CNKI, we extracted the journal paper data of the past three years, including paper title, journal title, author, subject keywords of the paper, number of citations, frequency of citations, etc.
[0121] 2) Data preprocessing: Preprocess the collected data, including data cleaning, deduplication, formatting and other operations.
[0122] 3) Feature extraction: Through in-depth processing of paper data, the paper’s subject keywords, number of citations, citation frequency and other features are extracted to reflect the paper’s influence and academic value.
[0123] 4) Model construction: Utilize the extracted features and combine them with machine learning algorithms, such as TF-IDF, TextRank or LDA, to build a journal topic analysis model. This model can automatically identify and evaluate the popularity and novelty of topics and predict future research trends.
[0124] 5) Topic recommendation: Through model analysis, research hotspots in various disciplines are recommended for topic planning.
[0125] 6) Data update: Update data regularly to ensure the real-time and timely nature of topic recommendations.
[0126] 7) Method optimization: According to actual usage, the parameters of the topic analysis model are optimized to improve the accuracy of topic recommendations.
[0127] Figure 2 This is the data collection flow chart of this embodiment. Data collection mainly collects and confirms that journal article information is authentic and valid, providing a data basis for topic analysis. Figure 2Specifically, it can include: determining the goals and needs of data collection (journal articles); formulating a data collection plan; implementing data collection in combination with the data collection plan, the goals and needs of data collection, and extracting journal article data within a set time period included in the preset data platform; implementing data sorting, sorting the collected data into a set standardized format according to the resource type, and selecting a set storage device or database for storage to facilitate subsequent data analysis and application.
[0128] Figure 3 This is the preprocessing flow chart of this embodiment. The main purpose of data preprocessing is to remove invalid data, irregular data, erroneous data, as well as noise and redundancy, so that the data is more standardized and reliable; fill in missing values to ensure data integrity; unify the data range, dimension, format, and type to make the data easier to calculate and analyze later, and prepare for the establishment of a topic analysis model later. Figure 3 Specifically, it may include: classifying the journal paper data according to subject information and / or industry information; data cleaning, processing missing values and outliers in the journal paper data; data deduplication, deleting duplicate rows or records in the journal paper data to ensure that each record of the journal paper data is unique; data formatting, converting the data into a format suitable for analysis; data normalization, adjusting the size of the data to make it within a unified range, such as the year to which the resource belongs.
[0129] Figure 4 This is the model building flow chart of this embodiment, which may specifically include: features of paper data extraction (topic keywords, number of citations, citation frequency, etc.); model selection and training, including regression analysis, classification, clustering, etc.; model evaluation and adjustment, including evaluation, adjustment, optimization, etc.; and then deployment and application, including actual application scenarios and application effect monitoring and evaluation.
[0130] Figure 5 This is the topic recommendation flow chart of this embodiment, which may specifically include: obtaining publishing hotspots in the past three months, six months, and one year according to the subject; selecting publishing hotspots, passing the publishing hotspots to the topic analysis model, and obtaining recommended topics; selecting recommended topics, passing the recommended topics to the topic analysis model, and obtaining paper titles; selecting paper titles and generating a paper outline.
[0131] As an exemplary embodiment, see Figures 6 to 9 , Figure 6 Get the latest publishing hotspots based on disciplines; Figure 7 In the selected discipline, the publication hotspots are displayed; in Figure 8 The recommended topics are shown in the table below: Fig. 9Shown in the figure are the recommended paper names obtained by selecting the topic "Research on the Evaluation of Autonomous Knowledge System Based on Academic Journals: This article will explore how to evaluate the autonomous knowledge system based on academic journals through certain standards and methods. Including evaluating its completeness, accuracy, update, etc., to help readers better understand and grasp their own knowledge system."
[0132] It should be noted that the size of the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In practical applications, all the above possible implementation methods can be combined in any way to form possible embodiments of the present application, which will not be described one by one here.
[0133] Based on the methods for constructing a topic analysis model based on journal article big data provided in the above embodiments, and based on the same inventive concept, the embodiments of the present application also provide a device for constructing a topic analysis model based on journal article big data.
[0134] Fig.10 : is a structural diagram of a topic analysis model construction device based on journal article big data provided in an embodiment of the present application. Fig.10 As shown, the device for constructing a topic analysis model based on journal article big data may specifically include a data acquisition unit 1010 , a data preprocessing unit 1020 , a feature extraction unit 1030 and a model construction unit 1040 .
[0135] The data collection unit 1010 is used to extract journal paper data within a set time interval based on the journal paper big data collected by the preset data platform;
[0136] A data preprocessing unit 1020 is used to preprocess the extracted journal paper data to obtain preprocessed paper data;
[0137] The feature extraction unit 1030 is used to extract features including the subject keywords, citation times, and citation frequency of the paper by performing deep processing on the pre-processed paper data;
[0138] The model building unit 1040 is used to build a topic analysis model by utilizing the extracted features in combination with a preset machine learning algorithm.
[0139] A possible implementation method is provided in an embodiment of the present application, wherein the data acquisition unit 1010 is further configured to:
[0140] Determine the goals and needs of data collection;
[0141] Develop a data collection plan;
[0142] According to the data collection plan, data collection goals and requirements, data collection is carried out to extract journal article data within a set time period collected by the preset data platform;
[0143] Implement data collation, organize the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
[0144] A possible implementation method is provided in an embodiment of the present application, wherein the preprocessing includes one or more of the following:
[0145] Classify the journal article data according to subject information and / or industry information;
[0146] Handling missing values and outliers in the journal article data;
[0147] Deleting duplicate rows or records in the journal article data to ensure that each record of the journal article data is unique;
[0148] The data range, dimension, format and type of the journal article data are unified to facilitate subsequent data calculation and analysis.
[0149] A possible implementation method is provided in an embodiment of the present application, wherein the model building unit 1040 is further configured to:
[0150] Select a preset machine learning algorithm;
[0151] Divide the extracted features into training set and validation set;
[0152] Use the training set to train the preset machine learning algorithm to obtain a preliminary topic analysis model;
[0153] The validation set is used to optimize and adjust the parameters of the preliminary topic analysis model to obtain the optimized topic analysis model.
[0154] A possible implementation method is provided in the embodiment of the present application, such as Fig.11 As mentioned above Fig.10 The device shown may also include a recommendation unit 1110, which is used to:
[0155] Deploy the topic analysis model based on actual application scenarios;
[0156] Acquire publishing hotspots within a set time period based on subject information;
[0157] Select a publishing hotspot, pass it into the topic analysis model, and obtain recommended topics;
[0158] Select the recommended topic, pass it into the topic analysis model, and obtain the paper title;
[0159] Select a paper topic and generate a paper outline.
[0160] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned embodiments of a method for constructing a topic analysis model based on journal article big data.
[0161] In an exemplary embodiment, an electronic device is provided, such as Fig.12 As shown, Fig.12 The electronic device 1200 shown includes: a processor 1201 and a memory 1203. The processor 1201 and the memory 1203 are connected, such as through a bus 1202. Optionally, the electronic device 1200 may also include a transceiver 1204. It should be noted that in actual applications, the transceiver 1204 is not limited to one, and the structure of the electronic device 1200 does not constitute a limitation on the embodiments of the present application.
[0162] Processor 1201 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1201 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0163] The bus 1202 may include a path to transmit information between the above components. The bus 1202 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1202 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0164] The memory 1203 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.
[0165] The memory 1203 is used to store the computer program code for executing the solution of the present application, and the execution is controlled by the processor 1201. The processor 1201 is used to execute the computer program code stored in the memory 1203 to implement the contents shown in the above method embodiment.
[0166] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.12 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0167] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program, wherein the computer program is configured to execute any one of the above-mentioned embodiments of the method for constructing a topic analysis model based on journal article big data when running.
[0168] Based on the same inventive concept, an embodiment of the present application also provides a computer program product, including a computer program, which is configured to execute the topic analysis model construction method based on journal article big data of any of the above embodiments when running.
[0169] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.
[0170] Those skilled in the art can understand that the technical solution of the present application can be essentially or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, which includes a number of program instructions, so that an electronic device (such as a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application when running the program instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0171] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as electronic devices such as personal computers, servers, or network devices), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of an electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of the present application.
[0172] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present application, they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate from the protection scope of the present application.
Claims
1. A method for constructing a topic analysis model based on journal article big data, characterized in that: The method comprises: Based on the big data of journal articles collected by the preset data platform, extract the journal article data of the set time interval; Preprocessing the extracted journal paper data to obtain preprocessed paper data; By deeply processing the preprocessed paper data, features including the paper's subject keywords, citation times, and citation frequency are extracted; Utilize the extracted features and combine them with the preset machine learning algorithm to build a topic analysis model.
2. The method according to claim 1, characterized in that The extracting of journal article data within a set time interval based on the journal article big data collected by the preset data platform includes: Determine the goals and needs of data collection; Develop a data collection plan; According to the data collection plan, data collection goals and requirements, data collection is carried out to extract journal article data within a set time period collected by the preset data platform; Implement data collation, organize the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
3. The method according to claim 1, characterized in that The pre-processing includes one or more of the following: Classify the journal article data according to subject information and / or industry information; Handling missing values and outliers in the journal article data; Deleting duplicate rows or records in the journal article data to ensure that each record of the journal article data is unique; The data range, dimension, format and type of the journal article data are unified to facilitate subsequent data calculation and analysis.
4. The method according to claim 1, characterized in that: The extracted features are combined with a preset machine learning algorithm to construct a topic analysis model, including: Select a preset machine learning algorithm; Divide the extracted features into training set and validation set; Use the training set to train the preset machine learning algorithm to obtain a preliminary topic analysis model; The validation set is used to optimize and adjust the parameters of the preliminary topic analysis model to obtain the optimized topic analysis model.
5. The method according to claim 1, characterized in that After constructing the topic analysis model by using the extracted features in combination with a preset machine learning algorithm, the method further includes: Deploy the topic analysis model based on actual application scenarios; Acquire publishing hotspots within a set time period based on subject information; Select a publishing hotspot, pass it into the topic analysis model, and obtain recommended topics; Select the recommended topic, pass it into the topic analysis model, and obtain the paper title; Select a paper topic and generate a paper outline.
6. A device for constructing a topic analysis model based on journal article big data, characterized in that: The device comprises: A data collection unit, used to extract journal article data within a set time interval based on the journal article big data collected by a preset data platform; A data preprocessing unit, used for preprocessing the extracted journal paper data to obtain preprocessed paper data; A feature extraction unit is used to extract features including the subject keywords, citation times, and citation frequency of the paper by performing deep processing on the preprocessed paper data; The model building unit is used to build a topic analysis model by utilizing the extracted features in combination with a preset machine learning algorithm.
7. The device according to claim 6, characterized in that The data acquisition unit is also used for: Determine the goals and needs of data collection; Develop a data collection plan; According to the data collection plan, data collection goals and requirements, data collection is carried out to extract journal article data within a set time period collected by the preset data platform; Implement data collation, organize the collected data into a set standardized format according to the resource type, and select a set storage device or database for storage to facilitate subsequent data analysis and application.
8. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing a topic analysis model based on journal article big data as described in any one of claims 1 to 5.
9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method for constructing a topic analysis model based on journal article big data according to any one of claims 1 to 5 when running.
10. A computer program product, comprising a computer program, characterized in that The computer program is configured to execute the method for constructing a topic analysis model based on journal article big data as described in any one of claims 1 to 5 when running.
Citation Information
Cited By
AI topic selection ability evaluation system based on adaptive evaluation
CN120929348A
An AI-based topic selection ability assessment system based on adaptive evaluation
CN120929348B
