Agricultural subject classification method for papers

Abstract

The invention discloses an agricultural subject classification method for papers, and solves the technical problems of insufficient data specificity and low classification precision in the prior art. Comprising the steps of obtaining to-be-classified paper content, and performing preliminary data preprocessing to form complete text content; inputting text content, and converting the text content into a format required by the model; training and optimizing the agricultural subject classification model; inputting the text content subjected to format conversion into the trained models, wherein each model gives an independent classification judgment result for the same text; and integrating a classification model result, and outputting a final classification result of the papers by adopting a weighted voting method. According to the method, a fine data preprocessing and optimized training strategy is adopted, so that the model can better capture features and modes in the agricultural field, and the ability to understand and classify complex texts is improved.

Description

Technical Field

[0001] The invention relates to the field of agricultural information technology, and in particular to an agricultural subject classification method for papers. Background Art

[0002] In the process of academic research and technological innovation, subject classification is a key step. Traditional subject classification methods mainly rely on manual annotation by experts based on rules and domain knowledge. Although this method can utilize the experience of experts, it is inefficient when faced with massive paper data, and is prone to subjective bias due to human factors. In addition, existing classification systems are mostly based on general subject classification standards, which are difficult to meet the needs of refined classification in specific fields such as agricultural subjects. With the development of science and technology, the application of deep learning and natural language processing technology has brought new opportunities and challenges to subject classification. Pre-trained models such as BERT, RoBERTa and XLNet have demonstrated superior performance in natural language processing tasks, especially in text classification tasks. However, for research papers in the agricultural field, these models still face problems such as insufficient data specificity and low classification accuracy.

[0003] Therefore, how to take advantage of the pre-trained model and combine the knowledge of field experts to develop a method that can automatically, efficiently and accurately classify papers in agricultural disciplines has become a technical problem that needs to be solved urgently. It is in this context that the present invention is proposed, aiming to develop a classification method that can adapt to the characteristics of agricultural disciplines by comprehensively utilizing multiple deep learning models and their classification results, improve the classification accuracy and efficiency of agricultural field paper data, and provide strong data support for agricultural scientific research and policy making. Summary of the invention

[0004] The purpose of the present invention is to provide a method for agricultural subject classification of papers to solve the technical problems of insufficient data specificity and low classification accuracy in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides a method for classifying agricultural disciplines of papers, comprising:

[0007] Obtain the content of the papers to be classified, perform preliminary data preprocessing, and form complete text content;

[0008] Input text content and convert it into the format required by the model;

[0009] Use BERT, RoBERTa, and XLNet as basic models, load pre-trained weights, build a binary classification model and a twelve-subject agricultural classification model, and train and optimize the models;

[0010] The converted text content is input into the trained model, and each model gives independent classification judgment results for the same text;

[0011] The results of the comprehensive classification model are used, and a weighted voting method is adopted to output the final classification results of the paper.

[0012] Furthermore, the paper content to be classified includes title and abstract fields. The Pandas data processing library in Python is used to merge the title and abstract fields of each paper through a simple string concatenation operation to form a complete text content and save it as a new file.

[0013] Furthermore, the input text content and converting the text content into the format required by the model include:

[0014] Use BertTokenizer, RobertaTokenizer, and XLNetTokenizer tokenizers to split the text into corresponding tokens, and generate the token encoding and attention mask required by the model. The token encoding is a numerical representation of the text content, and the attention mask marks the difference between the actual text and the filling part, which is used to guide the model to focus on the valid part as the input of the subsequent text classification task.

[0015] Furthermore, using BERT, RoBERTa and XLNet as basic models, loading pre-trained weights, and constructing a binary classification model and an agricultural twelve-subject classification model include:

[0016] Get a collection of papers to be trained, where each paper includes a title and an abstract field;

[0017] Use the data processing library to merge the title and abstract fields of each paper to form a complete text content, and assign a corresponding real subject classification label to each paper;

[0018] Convert the merged text content into the format required by the model, use at least one of BERT, RoBERTa and XLNet as the basic model, and build a binary classification model and an agricultural twelve-discipline classification model for the paper classification task by loading pre-trained weights;

[0019] The cross entropy loss function is used as the optimization target, and the AdamW optimizer is used for model training and optimization. Batch gradient descent is used during training, and mixed precision training technology is used to accelerate training and avoid memory overflow.

[0020] Use the validation set to evaluate the trained model. The evaluation indicators include accuracy, precision, recall, and F1 score. Adjust the model structure and hyperparameters based on the evaluation results.

[0021] Select the model that performs best on the validation set and perform performance testing on the test set to confirm the generalization ability of the model.

[0022] Furthermore, the output layer of the binary classification model is a fully connected layer of two neurons, which is used to predict the probability of a paper belonging to agriculture or non-agriculture; the output layer of the agricultural twelve-subject classification model is a fully connected layer containing twelve neurons, which is used to predict a specific agricultural subject category.

[0023] Furthermore, the calculation formula of the cross entropy loss function is as follows:

[0024]

[0025] Among them, y ij represents the true label of the i-th sample, represents the predicted probability of the model output, C is the total number of categories, and N is the number of samples.

[0026] Furthermore, the AdamW optimizer controls the model convergence speed and prevents overfitting by dynamically adjusting the learning rate and regularization parameters. The initial learning rate is set to 5e-5 and is gradually reduced through a linear decay strategy to ensure stable convergence of the model in the later stages of training. During training, batch gradient descent is used, and the batch size is set to 16 or 32, which is dynamically adjusted according to specific hardware conditions.

[0027] Furthermore, the comprehensive classification model results adopt a weighted voting method to output the final classification results of the paper, including:

[0028] Compare the classification results of the BERT, RoBERTa, and XLNet binary classification models. If the prediction results of the three models are all agriculture, the text is judged to be in the agricultural field and enters the next step of the agricultural twelve-subject classification model. If a model judges it as non-agriculture, it is directly output as non-agriculture without further processing;

[0029] For the texts judged to be agricultural, the classification results of the BERT, RoBERTa and XLNet agricultural twelve-subject classification models were combined, and a weighted voting method was used to obtain the final classification results of the paper.

[0030] Based on the above technical solution, the embodiments of the present invention can at least produce the following technical effects:

[0031] (1) The agricultural subject classification method for papers provided by the present invention effectively improves the accuracy of subject classification by integrating the results of multiple pre-trained models. By integrating the judgments of multiple models, it is possible to avoid the deviations and misjudgments that may exist in a single model, thereby improving the reliability of the classification results. The multi-model voting strategy further enhances the stability of the classification, making the final output results more convincing. In addition, the present invention adopts sophisticated data preprocessing and optimized training strategies, so that the model can better capture the characteristics and patterns of the agricultural field, and improve the understanding and classification capabilities of complex texts.

[0032] (2) The agricultural subject classification method for papers provided by the present invention realizes the automated subject classification of paper data, greatly reducing the necessity of manual intervention. Traditional manual classification methods are often time-consuming, labor-intensive and error-prone, while the present invention is based on deep learning technology and can quickly process massive amounts of data, significantly improving classification efficiency. The automated classification process not only improves work efficiency, but also reduces the occurrence of human errors, ensuring the consistency and objectivity of classification. This automation capability is particularly suitable for the processing and analysis of large-scale agricultural data, and can save a lot of time and energy for scientific researchers.

[0033] (3) The agricultural subject classification method for papers provided by the present invention is applicable to subject classification tasks in various agricultural fields and has strong universality. By combining the feature extraction and classification capabilities of multiple deep learning models, the system can flexibly respond to subtle differences between different agricultural subjects and is applicable to classification needs under various agricultural themes. Whether it is basic research, applied research, or research in interdisciplinary fields, this method can be effectively applied for classification. At the same time, the method has good scalability and portability, and the model can be further trained and optimized according to specific needs, so that it can be applied to more academic classification fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0035] Figure 1 It is a flow chart of the agricultural subject classification method of the present invention;

[0036] Figure 2 It is a training flow chart of the agricultural subject classification model of the present invention;

[0037] Figure 3 It is a table diagram of the data input of the paper of the present invention;

[0038] Figure 4 It is the subject classification output data diagram of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0040] Embodiment 1

[0041] like Figure 2 As shown, a training method for an agricultural subject classification model of the paper includes:

[0042] Get the set of papers to be trained. The papers to be trained should include the title and abstract fields. Use the Pandas data processing library in Python to merge the title and abstract fields of each paper through simple string concatenation operations to form a complete text content. At the same time, each paper corresponds to a real subject classification label.

[0043] Convert the input text into the format required by the model. To adapt to the needs of different pre-trained language models, you need to use BertTokenizer, RobertaTokenizer, and XLNetTokenizer to split the text into corresponding tokens and generate the token encoding (input_ids) and attention mask (attention_mask) required by the model. Input_ids is the numerical representation of the text content, and attention_mask marks the difference between the actual text and the filling part, which is used to guide the model to focus on the effective part as the input of the subsequent text classification task.

[0044] Perform model training and optimization. Use BERT, RoBERTa, and XLNet as the basic models, and build a binary classification model and an agricultural twelve-subject classification model for the two classification tasks by loading pre-trained weights. For the agricultural / non-agricultural binary classification task, define the output layer as a fully connected layer of two neurons to predict the probability that the paper belongs to "agriculture" or "non-agriculture"; for the agricultural twelve-subject classification task, define the output layer as a fully connected layer containing twelve neurons to predict the specific agricultural subject category. Model training uses an optimization objective based on the cross-entropy loss function. For classification problems, the cross-entropy loss function can effectively measure the deviation between the predicted probability distribution and the true label. The formula is as follows:

[0045]

[0046] Among them, y ij represents the true label of the i-th sample, represents the predicted probability of the model output, C is the total number of categories, and N is the number of samples.

[0047] In this embodiment, the optimization process uses the AdamW (Adaptive Moment Estimation with Weighted Decay) optimizer, which controls the model convergence speed and prevents overfitting by dynamically adjusting the learning rate and regularization parameters. The initial learning rate is set to 5e-5 and is gradually reduced through a linear decay strategy to ensure stable convergence of the model in the later stages of training. During training, Mini-Batch Gradient Descent is used, and the batch size is usually set to 16 or 32, which is dynamically adjusted according to specific hardware conditions.

[0048] In this embodiment, at each iteration, the segmented input data and its true label are passed to the model, the loss value is calculated through forward propagation, and the model parameters are updated through back propagation. In order to speed up training and avoid memory overflow, mixed precision training technology is used.

[0049] Perform model evaluation. After training, use the validation set to evaluate model performance. Evaluation indicators include accuracy, precision, recall, and F1-Score. For the agricultural / non-agricultural binary classification task, the evaluation indicators focus on the balance between precision and recall; for the agricultural twelve-subject classification task, the focus is on the classification accuracy of each subject and the overall F1 score.

[0050] In this embodiment, the model structure and hyperparameters are further adjusted based on the model weaknesses found through the validation set evaluation. Specific optimization methods include: adjusting the learning rate and batch size to optimize the model convergence speed; using data enhancement strategies (such as randomly disrupting the order of sentences and replacing synonyms) to increase the diversity of training samples; using the early stopping mechanism to prevent the model from overfitting, that is, terminating the training when the validation set performance no longer improves.

[0051] The model with the best performance on the validation set was selected, and performance was tested on the test set. The model's prediction results were compared with the true labels to confirm the model's generalization ability. After the above training and optimization process, the obtained model can stably and efficiently complete the agricultural / non-agricultural binary classification and agricultural twelve-subject classification tasks, providing reliable technical support for practical applications.

[0052] Embodiment 2

[0053] like Figure 1 As shown, a method for classifying agricultural subjects of papers, using the trained model of Example 1 for classification, includes:

[0054] Obtain the content of the papers to be classified and perform preliminary data preprocessing. The papers to be classified should include the title and abstract fields. Use the Pandas data processing library in Python to merge the title and abstract fields of each paper through simple string concatenation operations to form a complete text content and save it as a new file to facilitate the subsequent classification model call and processing.

[0055] In this embodiment, the paper data input0.xlsx is input, and the TI (title) and AB (abstract) fields are used, such as Figure 3 shown.

[0056] Convert the input text into the format required by the model. To adapt to the needs of different pre-trained language models, you need to use BertTokenizer, RobertaTokenizer, and XLNetTokenizer to split the text into corresponding tokens and generate the token encoding (input_ids) and attention mask (attention_mask) required by the model. Input_ids is the numerical representation of the text content, and attention_mask marks the difference between the actual text and the filling part, which is used to guide the model to focus on the effective part as the input of the subsequent text classification task.

[0057] Model prediction is performed to achieve text classification. The processed data will be input into the trained BERT, RoBERTa and XLNet models, including the agricultural / non-agricultural binary classification and the agricultural twelve-subject classification models.

[0058] In this embodiment, the architecture of each model extracts contextual features from the input sequence based on its self-attention mechanism. After multiple layers of Transformer encoding, the model outputs a set of logits, indicating the confidence score that the text belongs to each category (such as "agriculture" or "non-agriculture"). The logits are normalized by the softmax function, and the index corresponding to the maximum value is taken as the classification result. By analogy, each model gives its independent classification judgment result for the same text.

[0059] The final classification is output after comprehensive consideration of the classification model results. The system will adopt a two-stage voting strategy. The first stage is for the agriculture / non-agriculture binary classification, comparing the classification results of the BERT, RoBERTa and XLNet agriculture / non-agriculture binary classification models. If the prediction results of the three models are all "agriculture", the text is judged to be in the agricultural field and enters the next step of the agricultural twelve-discipline classification process; if any model is judged to be "non-agriculture", "non-agriculture" is directly output without further processing. The second stage is for the text judged as "agriculture", combining the classification results of the BERT, RoBERTa and XLNet agriculture twelve-classification models, and using a weighted voting method to obtain the final classification result of the paper.

[0060] In this embodiment, the Python program is run to implement the subject classification of the paper and output the classification results (category_cn column), such as Figure 4 shown.

[0061] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of the present invention to be protected is defined by the attached claims and their equivalents.

Claims

1. A method for classifying agricultural subjects of papers, characterized in that: include: Obtain the content of the papers to be classified, perform preliminary data preprocessing, and form complete text content; Input text content and convert it into the format required by the model; Using BERT, RoBERTa, and XLNet as the basic models, we built a binary classification model and a twelve-discipline agricultural classification model for the paper classification task by loading pre-trained weights, and trained and optimized the models. The converted text content is input into the trained model, and each model gives independent classification results for the same text; The results of the comprehensive classification model are used, and a weighted voting method is adopted to output the final classification results of the paper.

2. The agricultural subject classification method of the paper according to claim 1 is characterized in that: The paper content to be classified includes title and abstract fields. The Pandas data processing library in Python is used to merge the title and abstract fields of each paper through a simple string concatenation operation to form a complete text content and save it as a new file.

3. The agricultural subject classification method of the paper according to claim 1 is characterized in that: The input text content is converted into the format required by the model, including: Use BertTokenizer, RobertaTokenizer, and XLNetTokenizer tokenizers to split the text into corresponding tokens, and generate the token encoding and attention mask required by the model. The token encoding is a numerical representation of the text content, and the attention mask marks the difference between the actual text and the filling part, which is used to guide the model to focus on the valid part as the input of the subsequent text classification task.

4. The agricultural subject classification method of the paper according to claim 1 is characterized in that: The use of BERT, RoBERTa and XLNet as the basic models, by loading pre-trained weights, to build a binary classification model and an agricultural twelve-subject classification model includes: Get a collection of papers to be trained, where each paper includes a title and an abstract field; Use the data processing library to merge the title and abstract fields of each paper to form a complete text content, and assign a corresponding real subject classification label to each paper; Convert the merged text content into the format required by the model, use at least one of BERT, RoBERTa and XLNet as the basic model, and build a binary classification model and an agricultural twelve-discipline classification model for the paper classification task by loading pre-trained weights; The cross entropy loss function is used as the optimization target, and the AdamW optimizer is used for model training and optimization. Batch gradient descent is used during training, and mixed precision training technology is used to accelerate training and avoid memory overflow. Use the validation set to evaluate the trained model. The evaluation indicators include accuracy, precision, recall, and F1 score. Adjust the model structure and hyperparameters based on the evaluation results. Select the model that performs best on the validation set and perform performance testing on the test set to confirm the generalization ability of the model.

5. The agricultural subject classification method of the paper according to claim 4 is characterized in that: The output layer of the binary classification model is a fully connected layer of two neurons, which is used to predict the probability that the paper belongs to agriculture or non-agriculture; the output layer of the agricultural twelve-discipline classification model is a fully connected layer containing twelve neurons, which is used to predict the specific agricultural discipline category.

6. The agricultural subject classification method of the paper according to claim 4 is characterized in that: The calculation formula of the cross entropy loss function is as follows: Among them, y ij represents the true label of the i-th sample, represents the predicted probability of the model output, C is the total number of categories, and N is the number of samples.

7. The agricultural subject classification method of the paper according to claim 4 is characterized in that: The AdamW optimizer controls the model convergence speed and prevents overfitting by dynamically adjusting the learning rate and regularization parameters. The initial learning rate is set to 5e-5 and is gradually reduced through a linear decay strategy to ensure stable convergence of the model in the later stages of training. During training, batch gradient descent is used, and the batch size is set to 16 or 32, which is dynamically adjusted according to specific hardware conditions.

8. The agricultural subject classification method of the paper according to claim 1 is characterized in that: The comprehensive classification model results adopt a weighted voting method to output the final classification results of the paper, including: Compare the classification results of the BERT, RoBERTa, and XLNet binary classification models. If the prediction results of the three models are all agriculture, the text is judged to be in the agricultural field and enters the next step of the agricultural twelve-subject classification model. If a model judges it as non-agriculture, it is directly output as non-agriculture without further processing; For the texts judged to be agricultural, the classification results of the BERT, RoBERTa and XLNet agricultural twelve-subject classification models were combined, and a weighted voting method was used to obtain the final classification results of the paper.

Images