Automatic classification method of citation text based on large language model

Abstract

The present invention discloses a method for automatic classification of citation texts based on a large language model, comprising: obtaining the full text of the cited document to be analyzed and the full text of its citing document, and performing data preprocessing on them; using the large language model to extract key information from the cited document; inputting the extracted key information and the obtained citation text information into a BERT model to extract corresponding features, aggregating the extracted features and the citation position features obtained in the data preprocessing, and using the aggregated features to train a multi-layer perceptron classification model for the classification task; after extracting features from the citation text to be classified through the above steps, aggregating the features and inputting them into the trained multi-layer perceptron classification model to predict the citation object label and citation function label of the citation text, and classifying the citation text according to the citation object label and citation function label. The present invention greatly improves the accuracy of citation text classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of citation analysis, and in particular relates to a method for automatic classification of citation texts based on a large language model. Background Art

[0002] Traditional citation analysis methods based on the frequency of citations have been widely used in academic evaluation, scientometrics, and research performance management, resulting in influential metrics such as the Journal Impact Factor (IF) and the H-index. However, these methods treat all citations equally and fail to effectively distinguish the specific differences between different citation contents. With the widespread use of full-text document databases and the development of text processing technology, fine-grained analysis and processing of citation text has become a general trend. Attempts to automatically classify citation text based on multiple aspects, such as the citation object and citation function, have gradually become a focus of researchers. However, current automatic citation content identification methods are not ideal in terms of accuracy, and are particularly disconnected from the content of the cited documents themselves. Therefore, how to effectively solve the problem of citation text classification and thus more fine-grainedly evaluate the influence of cited documents has become an important issue. Summary of the Invention

[0003] The purpose of the present invention is to address the shortcomings of the existing technology and provide a method for automatic classification of citation texts based on a large language model. This method reveals the essence of citation behavior in a more fine-grained manner from the two levels of citation objects and citation functions, thereby improving the accuracy of citation text classification.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for automatic classification of citation text based on a large language model includes the following steps:

[0006] Step 1: Obtain the full text of the cited document to be analyzed and the full text of its citing document, and perform data preprocessing on them;

[0007] Step 2: Use the large language model to extract key information from the cited literature;

[0008] Step 3: Input the key information extracted in step 2 and the citation text information obtained in step 1 into the BERT model to extract corresponding features. The extracted features and the citation location features obtained in step 1 are aggregated and the aggregated features are used to train the multi-layer perceptron classification model (MLP) for the classification task.

[0009] Step 4: After the citation text to be classified is processed through steps 1 and 2 and features are extracted through step 3, the features are aggregated and input into the trained multi-layer perceptron classification model MLP to predict the reference object label and reference function label of the citation text, and classify the citation text according to the reference object label and reference function label.

[0010] Furthermore, step 1 specifically includes:

[0011] S1.1. Obtain the full text of the cited articles to be analyzed and their corresponding citing articles from the database, and delete the articles for which the full text is unavailable.

[0012] S1.2. Convert the full-text PDF of the cited article to TXT format, preserving the citation marks;

[0013] S1.3. Use regular expression to match citation tags to obtain the citation content and citation location in the citing literature, and obtain the citation content set Among them, x i Represents the i-th sample (x i , z i ) in the cited text; i Represents the i-th sample (x i , z i ) in the cited literature; d i The citation position feature vector of the cited text in the citing document; i Represents the reference object label, y′ i Represents the reference function label, all labels are manually annotated; i = 1, 2, ..., N, where N represents the total number of samples.

[0014] Furthermore, the key information extracted in step 2 includes the background, methods and conclusions of the cited literature.

[0015] Furthermore, in step 2, the full text of the cited literature is input into the Large Language Model (LLM), and the background, method, and conclusion extracted are:

[0016] s i =LLM(x i , prompt1);

[0017] s′ i =LLM′(x i , prompt2);

[0018] s″ i =LLM″(x i , prompt3);

[0019] Among them, LLM is a large language model for refining the background of cited documents, prompt1 is a prompt word for refining the background of cited documents, and s i is the background text after the i-th cited document is extracted; LLM′ is the large language model of the method for extracting cited documents, prompt2 is the prompt word for the method of extracting cited documents, and s′ i is the method text extracted from the cited article i; LLM″ is the large language model for extracting the conclusion of the cited article; prompt3 is the prompt word for extracting the conclusion of the cited article, s″ i It is the conclusion text after extraction of the i-th cited document.

[0020] Furthermore, step 3 specifically includes:

[0021] Input the key information of the cited literature extracted in step 2 into the BERT model to obtain the feature vector of the key information of the cited literature;

[0022] Input the citation text information obtained in step 1 into the BERT model to obtain the feature vector of the citation text;

[0023] The training set is obtained by concatenating the feature vector of the key information of the cited document, the feature vector of the citation text, and the feature vector of the citation position obtained in step 1. The multi-layer perceptron classification model MLP is trained using the training set to obtain the model parameters of the multi-layer perceptron classification model MLP.

[0024] Furthermore, in step 3, during the training process, a loss function is established and the model parameters are optimized by back propagation until the loss function converges. The loss function is the cross entropy loss function L:

[0025]

[0026] Where, is the reference object label output by the multi-layer perceptron classification model MLP, is the output reference function label; y i is the reference object label manually annotated in the training set, y′ i is the manually annotated reference function label in the training set; i = 1, 2, ..., N, where N represents the total number of samples; λ is the parameter that optimizes the model effect, and its value range is [0.1, 5].

[0027] Furthermore, in step 4, a citation triple is constructed based on the predicted reference object label and reference function label. The citation triple includes a reference object inclusion relation triple, a reference object category triple, and a citation function triple.

[0028] Furthermore, the reference object inclusion relation triple is represented as (reference object, Part-of, cited document).

[0029] Furthermore, the reference object category triple is represented as (reference object, ISA, reference object category).

[0030] Furthermore, the citation function triple is represented as (citing document, citation function, cited document).

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention achieves better processing of long texts through a large language model, which can effectively extract key content information of the full text of the cited document and convert it into feature vectors for training the integrated language feature classifier, thereby achieving effective matching between the citation content and the content of the cited document itself, thereby improving the classification accuracy of the cited text;

[0033] 2. This invention further improves the classification accuracy of citation text by identifying the citation position feature vectors of the citation text in the citing document, including Introduction, Data & Methods, Results, Conclusion, etc., and taking advantage of the fact that citations in different positions usually have different citation functions;

[0034] 3. This paper uses manually annotated reference objects and reference function labels to fine-tune the BERT short text classification model, improving the BERT model's feature extraction capability for specific text classification tasks and obtaining a more accurate semantic feature vector representation;

[0035] 4. The present invention achieves more accurate citation text classification by integrating the content features of the cited document itself and the citation features in the citing document, and using manually annotated labels to fine-tune the pre-trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for automatic classification of citation texts based on a large language model according to an embodiment of the present invention;

[0037] Figure 2 This is a block diagram of an implementation method for automatic classification of citation texts based on a large language model according to an embodiment of the present invention;

[0038] Figure 3 This is a block diagram of an implementation of using a large language model to extract key information of cited documents in an embodiment of the present invention;

[0039] Figure 4 This is a block diagram of an implementation of multi-target classification using integrated language features in an embodiment of the present invention; DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0042] The present invention will be further described below with reference to specific examples, but they are not intended to limit the present invention.

[0043] like Figure 1 and Figure 2 As shown, the embodiment of the present invention discloses a method for automatic classification of cited texts based on a large language model. This method obtains the original text information of the cited document through the large language model and an integrated language feature classifier, and adds a citation position feature vector to improve the accuracy of cited text classification. This achieves the technical effects of assisting in the evaluation of the influence of cited documents, exploring the knowledge transfer process between documents, and building a knowledge dissemination network and academic ontology. Specifically, the method includes the following steps:

[0044] Step 1: Obtain the full text of the cited document to be analyzed and its citing document, and perform data preprocessing:

[0045] S1.1. Obtain the full text of the cited articles to be analyzed from literature databases, such as Web of Science, Scopus, and PubMed. Use the "View Citing Articles" function provided by the database to obtain the full text of their corresponding citing articles. Download the cited article and its corresponding citing article in PDF format to form a document pair. Eliminate any document pairs for which the full text PDF is unavailable due to access permissions or other reasons.

[0046] S1.2. When a cited article appears in a citing article, it is usually marked in the format of "[number]". For example, the introduction section of the cited article "SARS-CoV-2 variants of concern and vaccine escape, from Alpha to Omicron and beyond" in its citing article "Antibody persistence and safety after heterologous boosting with orally aerosolised Ad5-nCoV in individuals primed with two-dose CoronaVac previously: 12-month analyses of a randomized controlled trial" is marked as "[3]", where "[3]" is the citation mark and the sentence containing "[3]" is the citation sentence. Use Python language to convert all the PDFs of the citing articles obtained in S1.1 into TXT format text, and retain all the citation marks "[]" in the text;

[0047] S1.3. Use the regular expression in Python to match the reference mark "[]" to obtain the citation sentence and citation position in the citing article. The citation position is divided into four categories: "Introduction", "Data & Methods", "Results" and "Discussion & Conclusion". The specific classification system is shown in Table 1. For example, in the article "Antibody persistence and safety after heterologous boosting with orally aerosolisedAd5-nCoV in individuals primed with two-dose CoronaVac previously: 12-month analyses of a randomized controlled trial" mentioned in S1.2, the reference mark "[3]" can be matched to extract the citation sentence and the citation position "Introduction".

[0048] Table 1 is the classification system of citation positions

[0049]

[0050] Thus, we can get the citation text set:

[0051]

[0052] Among them, x i Represents the i-th document pair sample (x i , z i ) in the citation text. Since the citation context plays an important role in understanding the semantics of the citation, it is defined here as the citation sentence and its first two sentences and last two sentences in the citing document, a total of 5 sentences; i Represents the i-th document pair sample (x i , z i ) in the cited literature; d i The feature vector representing the citation position of the cited text in the citing document has a feature dimension of 4; i Represents the reference object label, y′ i = citation function labels, all of which are manually annotated; i = 1, 2, ..., N, where N represents the total number of samples. The citation object labels are mainly divided into three categories: "background," "method," and "conclusion." The specific classification system is shown in Table 2. The citation function labels are mainly divided into five categories: "background," "based on," "support," "criticism," and "different." The specific classification system is shown in Table 3.

[0053] Table 2 is the classification system of reference objects

[0054]

[0055]

[0056] Table 3 is the classification system of reference functions

[0057]

[0058] Step 2: Use the large language model to extract key information from the cited literature;

[0059] like Figure 3 As shown, in this embodiment, the full text of the cited document is input into the large language model LLM, and three groups of prompt words are set to respectively extract the background, method and conclusion of the cited document:

[0060] s i =LLM(x i ,prompt1); (2)

[0061] s′ i =LLM′(x i ,prompt2); (3)

[0062] s″ i =LLM″(x i , prompt3); (4)

[0063] Among them, LLM is a large language model for refining the background of cited documents, prompt1 is a prompt word for refining the background of cited documents, and s i is the background text after the i-th cited document is extracted; LLM′ is the large language model of the method for extracting cited documents, prompt2 is the prompt word for the method of extracting cited documents, and s′ i is the method text extracted from the cited article i; LLM″ is the large language model for extracting the conclusion of the cited article, prompt3 is the prompt word for extracting the conclusion of the cited article, and s″ i It is the conclusion text after extraction of the i-th cited document.

[0064] Step 3: Input the key information of the cited literature extracted in step 2 and the citation text information obtained in step 1 into the BERT model to extract corresponding features. The extracted features and the citation location features obtained in step 1 are used to train the multi-layer perceptron classification model MLP for the classification task. This step specifically includes:

[0065] S3.1, BERT model is a pre-trained model that can integrate deep contextual bidirectional language features, suitable for tasks such as text classification and semantic understanding. i , method text s′ i and conclusion text s″ i By inputting different BERT models respectively, we can get their feature vector representations:

[0066] q i =BERT(s i ); (5)

[0067] q′ i =BERT′(s′ i ); (6)

[0068] q″ i =BERT″(s″ i ); (7)

[0069] Among them, BERT is the BERT model used to extract the background features of cited documents, BERT′ is the BERT model used to extract the method features of cited documents, and BERT″ is the BERT model used to extract the conclusion features of cited documents; q i is the obtained background text feature vector of the cited document. In this embodiment, the feature dimension is 768; q′ i is the feature vector of the cited literature method text, with a feature dimension of 768; q″ i It is the feature vector of the conclusion text of the cited literature, and its feature dimension is 768.

[0070] S3.2、Transfer the citation text to x i Input the BERT model to obtain the citation text feature vector:

[0071] P i =BERT″′(x i ) (8)

[0072] Among them, BERT″′ is the BERT model for extracting citation text features, P i is the output citation text feature vector. In this embodiment, the feature dimension is 768.

[0073] S3.3. Feature vector q of the background text of the cited document i , method text feature vector q′ i , Conclusion text feature vector q″ i , the citation text feature vector Pi and the citation position vector d obtained in step 1 i After splicing, we get the training set; input the training set into the multi-layer perceptron classification model MLP, and output the predicted value:

[0074]

[0075]

[0076] Among them, MLP is a classifier for classifying reference target labels, with a feature dimension of [128, 16]; MLP′ is a classifier for classifying reference function labels, with a feature dimension of [128, 16]; is the output reference object label, Is the reference function tag.

[0077] S3.4. Use the training set to train the multi-layer perceptron classification model. Set the loss function during the training process and optimize the model parameters through back propagation until the loss function converges. The loss function is the cross entropy loss function L.

[0078]

[0079] Where, is the reference object label output by the multi-layer perceptron classification model, is the output reference function label; y i is the reference object label manually annotated in the training set, y′ i is the manually annotated reference function label in the training set; i = 1, 2, ..., N, where N represents the total number of samples; λ is the parameter that optimizes the multilayer perceptron classification model, and its value range is [0.1, 5].

[0080] Step 4: After the citation text to be classified is processed in steps 1 and 2 and features are extracted in step 3, the features are aggregated and input into the trained multi-layer perceptron classification model MLP to predict the reference object label and reference function label of the citation text, and the citation text is classified according to the reference object label and reference function label;

[0081] In this embodiment, if Figure 4 As shown in the figure, a trained multi-layer perceptron (MLP) classification model is used to predict the citation object label and citation function label of the citation text to be classified. Citation triples are then constructed based on the predicted citation object label and citation function label of the citing document. This citation triple consists of a citation object inclusion relationship triple, a citation object category triple, and a citation function triple. The citation object inclusion relationship triple, represented as (citation object, part-of, cited document), expresses the inclusion relationship between the citation object and the cited document, linking the citation object to the cited document to which it originally belongs. The citation object category triple, represented as (citation object, ISA, citation object category), expresses the categorical attributes of the citation object, namely, the content of the cited document cited by the citing document. The citation function triple, represented as (citing document, citation function, cited document), expresses the functional role played by the cited document within the citing document. The three triplets reveal the essence of the citation relationship from multiple dimensions and jointly express the complete connotation of the citation. They serve as the three attributes of the citation text to achieve the classification purpose. Examples are shown in Table 4.

[0082] Table 4 shows examples of citation text triples.

[0083]

[0084] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the present invention specification should be included in the protection scope of the present invention.

Claims

1. A method for automatic classification of citation text based on a large language model, characterized in that: The steps include: Step 1: Obtain the full text of the cited document to be analyzed and the full text of its citing document, and perform data preprocessing on them; Step 2: Use the large language model to extract key information from the cited literature; Step 3: Input the key information extracted in step 2 and the citation text information obtained in step 1 into the BERT model to extract corresponding features. The extracted features and the citation location features obtained in step 1 are aggregated and the aggregated features are used to train the multi-layer perceptron classification model (MLP) for the classification task. Step 4: After the citation text to be classified is processed in steps 1 and 2 and features are extracted in step 3, the features are aggregated and input into the trained multi-layer perceptron classification model MLP to predict the reference object label and reference function label of the citation text, and the citation text is classified according to the reference object label and reference function label; Wherein, step 1 specifically includes: S1.

1. Obtain the full text of the cited articles to be analyzed and their corresponding citing articles from the database, and delete the articles for which the full text is unavailable. S1.

2. Convert the full-text PDF of the cited article to TXT format, preserving the citation marks; S1.

3. Use regular expression to match citation tags to obtain the citation content and citation location in the citing literature and obtain the citation text set ,in, Indicates the Sample Citation text in ; Indicates the Sample The content of the cited literature in The feature vector representing the citation position of the cited text in the citing document; Indicates reference function labels, all labels are manually annotated; represents the total number of samples; The key information extracted in step 2 includes the background, methods, and conclusions of the cited literature.

2. The method for automatic classification of citation text based on a large language model according to claim 1 is characterized in that: In step 2, the full text of the cited literature is input into the Large Language Model (LLM), and the background, method, and conclusion extracted are: in, It is a large language model for refining the background of cited documents, prompt1 is a prompt word for refining the background of cited documents, It is Background text extracted from cited articles; It is a large language model for refining cited literature methods, prompt2 is the prompt word for refining cited literature methods, It is Method text after extracting the cited articles; It is a large language model for refining the conclusions of cited documents; prompt3 is the prompt word for refining the conclusions of cited documents. It is The conclusion text after extracting the cited literature.

3. The method for automatic classification of citation text based on a large language model according to claim 1 is characterized in that: Step 3 specifically includes: Input the key information of the cited literature extracted in step 2 into the BERT model to obtain the feature vector of the key information of the cited literature; Input the citation text information obtained in step 1 into the BERT model to obtain the feature vector of the citation text; The feature vector of the key information of the cited document is concatenated with the feature vector of the citation text and the citation position obtained in step 1 to obtain a training set. The multi-layer perceptron classification model MLP is trained using the training set to obtain the model parameters of the multi-layer perceptron classification model MLP.

4. The method for automatic classification of citation text based on a large language model according to claim 3 is characterized in that: In step 3, during the training process, a loss function is established and the model parameters are optimized through back propagation until the loss function converges. The loss function is the cross entropy loss function L: Where, is the reference object label output by the multi-layer perceptron classification model MLP, The reference function label for the output; is the reference object label manually annotated in the training set, Reference function labels manually annotated in the training set; Represents the total number of samples; λ is the parameter that optimizes the model effect.

5. The method for automatic classification of citation text based on a large language model according to claim 1 is characterized in that: In step 4, a citation triple is constructed based on the predicted reference object label and reference function label. The citation triple includes a reference object inclusion relation triple, a reference object category triple, and a citation function triple.

6. The method for automatic classification of quotation text based on a large language model according to claim 5 is characterized in that: The reference object inclusion relationship triple is expressed as (reference object, Part-of, cited document).

7. The method for automatic classification of citation text based on a large language model according to claim 5 is characterized in that: The reference object category triple is represented as (reference object, ISA, reference object category).

8. The method for automatic classification of citation text based on a large language model according to claim 5, characterized in that: The citation function triple is represented as (citing document, citation function, cited document).

Images