Citation semantic function classification method, apparatus, device, medium, and product

Abstract

Embodiments of the present disclosure provide a citation semantic function classification method, device, equipment, medium and product, and relate to the technical field of data analysis. The method comprises: obtaining a sentence sequence of a to-be-analyzed literature, inputting the sentence sequence into a citation analysis model, obtaining a first semantic vector of each sentence in the sentence sequence through the context dependency relationship between each sentence in the sentence sequence by the citation analysis model, identifying the sentences belonging to citations according to the first semantic vectors of all the sentences, for each citation, obtaining a second semantic vector of the citation through the context dependency relationship between each vocabulary in the citation by the citation analysis model, classifying the semantic function of the citation according to the second semantic vector of the citation, and obtaining the semantic function category corresponding to the citation. The citation analysis model processes two subtasks of citation sentence recognition and semantic function category classification in series, and realizes end-to-end automatic analysis from the original to-be-analyzed literature to the semantic function category of the citation.

Description

Technical Field

[0001] This disclosure relates to the field of data analysis technology, and more specifically, this application relates to a method, apparatus, device, medium, and product for citation semantic function classification. Background Technology

[0002] Citations refer to sentences, paragraphs, and viewpoints extracted or cited from other books, articles, documents, and other materials in writing or academic research. As a core representation of knowledge connections within literature, the analytical methods of academic citations are an important foundation for scientometric research and the construction of academic evaluation systems.

[0003] Existing citation analysis work, both domestically and internationally, focuses on fine-grained citations, including classifying citations according to their semantic meaning in documents. This involves dividing citations into multiple semantic function categories to characterize the role of different types of citations in academic argumentation and to serve application scenarios such as academic impact assessment, knowledge graph construction, and document retrieval optimization.

[0004] Citation semantic function classification methods often rely on manual preprocessing or structured annotation from commercial databases. They typically use segmented citation sentences or annotated (Extensible Markup Language, XML) files as input. However, manually constructing annotations is extremely costly and susceptible to subjective bias and domain limitations. Therefore, existing citation semantic function classification methods are not applicable to unannotated original documents and lack scalability on large-scale open corpora. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, medium, and product for citation semantic function classification, which can solve the above-mentioned problems of the prior art. The technical solution is as follows: According to one aspect of the embodiments of this disclosure, a citation semantic function classification method is provided, including: Obtain the sentence sequence of the document to be analyzed; The sentence sequence is input into the citation analysis model. The citation analysis model obtains the first semantic vector of each sentence based on the contextual dependencies between each sentence in the sentence sequence. Based on the first semantic vectors of all sentences, the sentences belonging to the citation are identified. For each citation, the second semantic vector of the citation is obtained by using the citation analysis model to understand the contextual dependencies between the words in the citation. Based on the second semantic vector of the citation, the citation is classified into semantic functions to obtain the corresponding semantic function category.

[0006] According to another aspect of the present disclosure, a citation semantic function classification apparatus is provided, comprising: The sentence sequence acquisition module is used to obtain the sentence sequence of the document to be analyzed; The citation statement recognition module is used to input the statement sequence into the citation analysis model. The citation analysis model obtains the first semantic vector of each statement based on the contextual dependencies between each statement in the statement sequence. Based on the first semantic vectors of all statements, the module identifies the statements that belong to the citation. The functional category classification module is used to obtain the second semantic vector of each citation by using the citation analysis model to understand the contextual dependencies between the words in the citation. Based on the second semantic vector, the citation is classified into semantic functions to obtain the corresponding semantic function category.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device including a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the above-described method.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method.

[0009] According to one aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0010] The beneficial effects of the technical solutions provided in this disclosure are: The citation analysis model handles two subtasks: citation sentence identification and semantic function category classification. It achieves end-to-end automatic analysis from the original document to be analyzed to the semantic function category of the citation, without relying on manual preprocessing or structured input. While improving the efficiency and scalability of fine-grained citation analysis, it can be transferred to large-scale open corpora. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0012] Figure 1 A flowchart illustrating a citation semantic function classification method provided in this embodiment of the disclosure; Figure 2 This is one of the structural schematic diagrams of a citation analysis model provided in an embodiment of this disclosure; Figure 3 This is a second schematic diagram of the structure of a citation analysis model provided in an embodiment of the present disclosure; Figure 4 A schematic diagram of a citation semantic function classification process framework provided in this disclosure embodiment; Figure 5 A schematic diagram of the structure of a citation semantic function classification device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0013] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0014] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0015] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0016] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0017] The following explains the terminology and related technologies involved in this application: In the digital research environment, the scale of academic literature continues to expand, highlighting the growing need for systematic quantitative analysis of large-scale research results. Academic citations, as the core representation of knowledge relationships within literature, serve as a crucial foundation for scientometric research and the construction of academic evaluation systems.

[0018] Current citation analysis is undergoing a transformation from macro-level quantitative statistics to micro-level semantic analysis. This transformation is mainly reflected in the in-depth semantic characterization of citations through fine-grained citation context analysis, thereby providing strong support for accurately understanding academic dialogue mechanisms, tracing knowledge evolution paths, and constructing a refined scientific research evaluation framework. However, how to develop scalable methods that can automatically extract citation information from full-text academic papers and achieve fine-grained annotation has become a key technical problem that urgently needs to be solved to overcome the bottlenecks in the field's development.

[0019] In existing citation analysis work, domestic and foreign research on fine-grained citations has accumulated relatively systematically, mainly focusing on the semantic function categories of citations, and continuously advancing and deepening in the construction of theoretical frameworks and the expansion of technical paths.

[0020] At the theoretical level, existing work mostly divides citations into five semantic function categories to characterize the role of different citations in academic argumentation and to serve application scenarios such as academic impact assessment, knowledge graph construction and literature retrieval optimization, providing a theoretical basis for subsequent research.

[0021] The five semantic function categories include Background, Basis, Discussion, Support, and Contrast. Background refers to the research context, which outlines the research background, context, or points out knowledge gaps. Basis refers to the theoretical foundation, which identifies the basis of the current research theory, method, or hypothesis. Discussion refers to comparative discussion, which is used to compare, analyze, explore in depth, or critique the cited viewpoints. Support refers to corroboration and support, which is cited to support and strengthen the author's own arguments or results. Contrast refers to rebuttal and questioning, which clearly refutes, questions, or points out the limitations of the cited work, emphasizing the differences or contradictions between studies.

[0022] At the methodological level, the mainstream solutions can be divided into the following three categories: 1. Adopting a rule-based and dictionary-based technical approach, relying on expert-designed sentence templates and sentiment lexicons, and combining part-of-speech tagging, dependency parsing and other methods, it can automatically identify the semantic function category of citations.

[0023] 2. Citation classification is modeled as a supervised learning problem. By constructing multiple features such as bag-of-words, term frequency-inverse document frequency (TF-IDF), syntax, and position, traditional machine learning models such as support vector machines, Naive Bayes, and conditional random fields are used to complete the semantic function category classification of citations.

[0024] 3. With the development of deep learning and pre-trained language models, models such as Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) have been gradually introduced into citation semantic function category recognition tasks. Pre-trained language models based on Transformer (Bidirectional Encoder Representations from Transformers, BERT) and SciBERT have shown superior performance to traditional methods in tasks such as scientific text classification, named entity recognition, and citation function recognition. Using pre-trained models, citation semantic function categories can be automatically labeled on pre-segmented citation corpora, achieving high F1 scores on multiple public datasets.

[0025] In practice, manually constructing large-scale citation semantic function category (SFC) annotated corpora is extremely costly, and the annotation process is easily affected by subjective biases and domain limitations, making it difficult to continuously expand with the development of the discipline and the increase in the scale of literature. Meanwhile, literature in real-world research environments is primarily in Portable Document Format (PDF) format, with inconsistent text parsing quality, diverse citation formats, and complex page structures. These factors all interfere with citation location identification and context extraction, making it difficult to directly apply many methods based on "idealized input" in real-world scenarios.

[0026] The above studies provide a solid theoretical foundation and methodological reserves for fine-grained citation analysis, but they have not yet solved the problem of how to automatically complete the end-to-end processing of citation location identification, context extraction and citation semantic function category annotation from the original full-text PDF, and it is also difficult to achieve scalable application on massive open full-text corpora.

[0027] In other words, although fine-grained citation analysis has made some progress in the existing research mentioned above, there are still problems such as over-reliance on existing annotations in commercial databases, difficulty in automatically processing raw full texts, and insufficient scalability on large-scale open corpora.

[0028] How to fully utilize existing high-quality annotation resources while minimizing human intervention, and combine the latest advancements in text classification and pre-trained language models in recent years to build an automated processing workflow for original full-text PDFs, has become a key issue that fine-grained citation analysis must address in order to move towards engineering and large-scale applications.

[0029] Therefore, this disclosure provides a citation semantic function classification method to solve or partially solve the above-mentioned technical problems. It constructs an automatic identification method for academic citation context and citation semantic function category annotation based on full-text PDFs, which balances recognition accuracy and processing efficiency. In internal testing, it can reproduce expert annotation rules well and shows a certain generalization ability on external corpora, providing a feasible technical path for large-scale, automated fine-grained citation analysis.

[0030] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0031] It is understood that in the citation semantic function classification method provided in the embodiments of this disclosure, any method step can be executed by the terminal and / or the server, and all steps in the method can be executed independently by the terminal or the server, or jointly by the terminal and the server.

[0032] The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart voice interaction device (such as a smart speaker), wearable electronic device (such as a smartwatch), in-vehicle terminal, smart home appliance (such as a smart TV), AR / VR device, etc., but is not limited to these.

[0033] The embodiments of this disclosure will be described below with the server as the execution subject; however, this does not constitute a limitation on the embodiments of this disclosure.

[0034] Figure 1 A flowchart illustrating a citation semantic function classification provided in this disclosure is shown below. Figure 1 As shown, this disclosure provides a citation semantic function classification method, including: Step S101: Obtain the sentence sequence of the document to be analyzed.

[0035] Specifically, in step S101, the sentence sequence of the document to be analyzed is obtained.

[0036] For example, natural language processing tools such as NLTK, Stanford CoreNLP, or rule-based regular expressions can be used to segment the document to be analyzed into sentences, remove stop words and special symbols to complete preprocessing, and obtain a sentence sequence.

[0037] Step S102: Input the statement sequence into the citation analysis model. Using the contextual dependencies between each statement in the statement sequence, the citation analysis model obtains the first semantic vector of each statement. Based on the first semantic vectors of all statements, identify the statements that belong to the citation.

[0038] Specifically, in this embodiment of the disclosure, the citation analysis model is used to perform two sub-tasks. Sub-task 1 is the citation statement recognition task, which realizes citation context segmentation and positioning, aiming to accurately identify and extract citation statements that mention specific references from the full text of the document. Sub-task 2 is the semantic function category classification task, which realizes the semantic function category classification of citations, and performs fine-grained semantic function category classification on each citation extracted from sub-task 1.

[0039] In step S102, the statement sequence is input into the citation analysis model, and subtask 1 is executed through the citation analysis model to obtain the first semantic vector of each statement based on the contextual dependencies between each statement in the statement sequence. Based on the first semantic vectors of all statements, the statements belonging to the citation are identified.

[0040] Understandably, citation analysis models are based on the Transformer architecture and can use pre-trained language models as sentence encoders, such as BERT, RoBERTa, or SciBERT. They capture long-distance contextual dependencies between sentences through self-attention mechanisms and output the first semantic vector of each sentence.

[0041] The first semantic vector is input into a Conditional Random Field (CRF) or a fully connected layer classifier to identify sentences that belong to the citation, for example, by marking the start, middle and non-citation positions of the citation using BIO sequence labeling.

[0042] Step S103: For each citation, the second semantic vector of the citation is obtained by using the citation analysis model to determine the contextual dependencies between the words in the citation. The citation is then classified into semantic functions based on the second semantic vector to obtain the corresponding semantic function category.

[0043] Specifically, in step S103, for each citation, subtask 2 is executed through the citation analysis model to obtain the second semantic vector of the citation based on the contextual dependencies between the words in the citation. The citation is then classified into semantic functions based on the second semantic vector to obtain the corresponding semantic function category.

[0044] For each identified citation, its word sequence is input into the word-level encoding layer of the citation analysis model. The word-level encoding layer can use a bidirectional long short-term memory network (BiLSTM) or a Transformer encoder based on a multi-head attention mechanism. By calculating the attention weights between words, the main idea information of the citation is aggregated to obtain a vector representation of the deep semantic features of the citation, thus obtaining a second semantic vector representing the contextual dependencies between words in the citation.

[0045] Based on the second semantic vector, a softmax classifier or a support vector machine (SVM) is used to classify the citation into semantic function categories. For example, if the citation's second semantic vector, after being calculated by the classifier, corresponds to the category "Background" with the highest probability, then the citation is classified into this semantic function category.

[0046] It is understood that, in the embodiments of this disclosure, the specific classification method of semantic function categories can be determined according to actual needs, including but not limited to the classification methods of the five semantic function categories: Background, Basis, Discussion, Support, and Contras.

[0047] The method provided in this embodiment handles two sub-tasks: citation analysis model and semantic function category classification. It realizes end-to-end automatic analysis from the original document to be analyzed to the semantic function category of the citation, without relying on manual preprocessing or structured input. While improving the efficiency and scalability of fine-grained citation analysis, it can be transferred to large-scale open corpora.

[0048] In one possible implementation, the first semantic vector of each statement is obtained based on the contextual dependencies between each statement in the statement sequence, including: For each statement, determine the paragraph to which the statement belongs in the document to be analyzed, and obtain the corresponding sequence segment from the statement sequence with the statement as the center. The sequence segment includes the statement and adjacent statements that belong to the same paragraph as the statement. The sequence fragment is input into the first encoder to obtain the first semantic vector of the statement. The first semantic vector is used to represent the semantic information of the corresponding statement and the semantic information of the adjacent statements of the corresponding statement.

[0049] Specifically, for the document to be analyzed, natural language processing tools (such as NLTK, spaCy, or rule-based regular expressions) are used to segment the document into sentences and identify paragraph boundaries to obtain the sentence sequence and record the paragraph number to which each sentence belongs.

[0050] For each statement, determine the paragraph to which the statement belongs in the document to be analyzed; obtain the corresponding sequence segment from the statement sequence with the statement as the center, the sequence segment includes the statement and adjacent statements in the same paragraph as the statement.

[0051] For example, for each statement in the statement sequence, its paragraph number determines the paragraph it belongs to in the document to be analyzed. Then, using that statement as the center, extract the k adjacent statements (e.g., k is 1 or 2) from the same paragraph to form a sequence segment. The sequence segment includes the statement itself and its adjacent statements within the same paragraph. For instance, if a paragraph contains statements A1 to A5, the sequence segment extracted for statement A3 could be {A2, A3, A4}.

[0052] The number of adjacent statements that make up a sequence segment and their positional relationship and number with the central statement can be set according to actual needs. For statements located at the beginning or end of each paragraph, if they do not have adjacent statements in the preceding or following positions, the central statement can be copied as an adjacent statement.

[0053] The sequence fragment is input into the first encoder, which captures the contextual dependencies between statements within the fragment and outputs the first semantic vector of the statement. The first semantic vector not only represents the semantic information of the corresponding statement itself, but also incorporates the semantic information of adjacent statements in the same paragraph. The specific structure of the first encoder can be set according to actual needs.

[0054] The method provided in this embodiment uses the paragraph to which a statement belongs as a constraint to extract a sequence fragment containing adjacent statements and input it into an encoder. This allows the first semantic vector to fuse the semantic information of the statement itself and its neighboring sentences in the same paragraph, thereby enhancing the local contextual coherence of the statement representation, effectively capturing the contextual features of quotations within a paragraph, and improving the accuracy of distinguishing between quotations and non-quotations.

[0055] In one possible implementation, for each citation, a second semantic vector of the citation is obtained based on the contextual dependencies between the words in the citation, including: For each citation, the citation is segmented to obtain a word sequence, where the starting position of the word sequence is the classification tag CLS; The word sequence is encoded in multiple rounds by the second encoder, and the target vectors of CLS output by each encoding operation in the last N rounds of encoding operations are combined into the first vector sequence. The first vector sequence is pooled to compress it into a second vector sequence, the length of which is less than N. The second vector sequence is linearly interpolated to obtain the third vector sequence, the length of which is N. The first vector sequence and the third vector sequence are added element by element and averaged to obtain the fused vector sequence. The second semantic vector of the citation is obtained by calculating the average of all vectors in the fused vector sequence.

[0056] Specifically, in the step of obtaining the second semantic vector of each citation, unlike the traditional method of only using the top-level classification token (CLS) vector of the encoder for classification, the embodiments of this disclosure integrate the multi-level semantic representation of the CLS vector in the last N rounds of encoding operations of the encoder, and construct a richer citation semantic vector through pooling dimensionality reduction, linear interpolation restoration and vector fusion.

[0057] For each citation, the citation is segmented using a word segmentation tool such as jieba, LTP, or BERT Tokenizer to obtain a word sequence. A classification tag [CLS] is added to the beginning position of the word sequence to obtain an updated word sequence. The beginning position of the updated word sequence is the classification tag [CLS].

[0058] The word sequence is input into the second encoder, which performs multiple rounds of encoding operations on the word sequence. The target vectors [CLS] output by each encoding operation in the last N rounds of encoding operations are combined to form the first vector sequence. The second encoder includes, but is not limited to, BERT, RoBERTa, ALBERT, or SciBERT. The number of rounds of encoding operations of the second encoder and the specific number of N rounds of encoding extracted can be determined according to actual needs.

[0059] For example, the second encoder performs 24 rounds of encoding operations, extracts the [CLS] target vector output from each of the last 6 rounds, and forms a first vector sequence of length 6.

[0060] The first vector sequence is pooled to compress it into a second vector sequence. The length of the second vector sequence is less than N to reduce feature redundancy and preserve the core semantic pattern. The pooling process includes, but is not limited to, average pooling, max pooling, or one-dimensional convolutional pooling (kernel size 3, stride 2). It is understood that the length of the second vector sequence is related to the stride and window during pooling.

[0061] Linear interpolation is performed on the second vector sequence to restore it to a third vector sequence of length N, in order to reconstruct the dimension aligned with the first vector sequence. The linear interpolation process includes, but is not limited to, nearest neighbor interpolation or bilinear interpolation.

[0062] The first vector sequence and the third vector sequence are added element by element and averaged to obtain the fused vector sequence, which achieves the complementary fusion of the original hierarchical features and the compressed reconstructed features. The average value of all vectors in the fused vector sequence is calculated to obtain the second semantic vector of the citation.

[0063] The method provided in this embodiment inserts CLS tags into citations and extracts the final N-round CLS vectors through multiple rounds of encoding. This fully explores the deep semantic features of citation words, enhances the richness of semantic representation, and obtains a second semantic vector through pooling, interpolation recovery, and fusion averaging. By using a fusion mechanism of deep multi-round representation and compression recovery, it aggregates low-level syntactic features and high-level abstract semantic information, and reduces the redundancy of deep features, significantly improving the accuracy of subsequent citation semantic function classification.

[0064] In one possible implementation, each round of encoding operations includes: For this round of encoding operation, the initial vector of each element in the word sequence is determined. The elements include CLS and words. For the first round of encoding operation, the initial vector of each element is obtained by the encoder in advance. For subsequent rounds of encoding operation, the initial vector of each element is the target vector output by the previous round of encoding operation. For each element in the word sequence, the attention weight between each element and other elements is obtained based on the similarity between the initial vector of the element and the initial vectors of other elements in the word sequence. For each element in the word sequence, the initial vectors of all elements in the word sequence are weighted and aggregated according to the attention weight between the element and other elements to obtain the aggregated vector corresponding to the element. The initial vector of the element is updated with the aggregated vector to obtain the target vector of the element.

[0065] Specifically, in the citation semantic encoding process, each round of encoding operations includes: For this round of encoding operations, the initial vector of each element in the word sequence is determined. The elements include CLS and words. The initial vector of each element is obtained by adding word embedding, position embedding and segment embedding. The specific representation of word embedding includes, but is not limited to, WordPiece, Byte Pair Encoding (BPE).

[0066] It is understandable that for the first round of encoding, the initial vector of each element is obtained by the encoder in advance, while for subsequent rounds of encoding, the initial vector of each element is the target vector output by the previous round of encoding.

[0067] For each element in the word sequence, the attention weight between each element and other elements is obtained based on the similarity between the initial vector of the element and the initial vectors of other elements in the word sequence.

[0068] For example, for each element in the word sequence, its initial vector is input into three learnable linear transformation matrices to generate a query vector, a key vector, and a value vector.

[0069] Based on the similarity between the query vector of an element and the key vectors of all other elements in the vocabulary sequence (e.g., similarity calculated by scaling the dot product), the attention weight between each element and other elements is obtained by normalization using the Softmax function.

[0070] For each element in the word sequence, the value vectors of all elements in the word sequence are weighted and summed using the attention weights between that element and other elements to obtain the aggregate vector corresponding to that element. This aggregate vector incorporates global contextual information related to the semantics of the current element.

[0071] The initial vector of the element is updated using the aggregate vector to obtain the target vector of the element.

[0072] For example, the aggregated vector is residually connected to the initial vector, then processed by layer normalization, and then input into a feed-forward network. Two layers of linear transformation are used in conjunction with ReLU or GELU activation functions for further nonlinear transformation. After residual connection and layer normalization again, the target vector of the element in this round of encoding operation is obtained.

[0073] The method provided in this embodiment refines the citation semantic encoding process into a multi-round layer-by-layer vector update based on a self-attention mechanism. This allows the CLS tag to gradually aggregate global semantic information through dynamic interaction with citation words in each round. Compared to simple single-layer encoding, this multi-round iterative attention aggregation can capture long-distance dependencies and deep semantic associations between words, significantly improving the representation ability of citation semantic vectors and thus improving the accuracy of semantic function classification.

[0074] In one possible implementation, the first encoder and the second encoder are the same encoder; The first semantic vector of the statement corresponding to the sequence fragment is obtained in the following way: The encoder obtains the second semantic vector of each statement in the sequence segment; The weight of each statement in the sequence segment is determined based on the positional relationship between the statements in the sequence segment. The second semantic vectors of the corresponding statements are weighted and fused according to the weight of each statement to obtain the first semantic vector of the statement corresponding to the sequence segment.

[0075] Specifically, in this embodiment, the first encoder and the second encoder are the same encoder, which uniformly processes the semantic representations at the statement and word levels, avoiding the problems of parameter redundancy and semantic space inconsistency between the two encoders; at the same time, a dynamic weighted fusion mechanism based on positional relationship is introduced, so that the central statement can obtain differentiated contextual information contributions based on the distance between adjacent statements, thereby improving the accuracy of citation recognition and reducing computational overhead.

[0076] The first semantic vector of the statement corresponding to the sequence fragment is obtained in the following way: The sequence segment (containing the central statement and its adjacent statements in the same paragraph) is input into the encoder. Each statement in the segment is independently lexical encoded. For each statement, the dependencies between words in the statement are captured through a multi-layer self-attention mechanism. The top-level [CLS] vector corresponding to the statement output by the encoder or the vector obtained by fusing the [CLS] vectors of multiple rounds of encoding operations is extracted as the second semantic vector of the statement.

[0077] Based on the positional relationship of each statement in the sequence segment, the weight of each statement in the sequence segment is determined. For example, the position index of the central statement is set to 0, and the position indices of adjacent statements are ±1, ±2, etc. The weight of each statement is calculated using a Gaussian kernel function (such as w=exp(-pos² / 2σ²)), an exponential decay function (such as w=exp(-λ|pos|)) or a learnable position embedding vector. The central statement usually has the highest weight, and the weight of adjacent statements that are farther away is lower, in order to suppress long-distance noise interference.

[0078] The second semantic vectors of the corresponding statements are weighted and fused according to the weight of each statement to obtain the first semantic vector of the statement corresponding to the sequence segment. The first semantic vector retains the semantic features of the central statement itself and also incorporates position-aware contextual semantic information. The specific methods of weighted fusion include, but are not limited to, weighted averaging or weighted summation based on attention mechanisms.

[0079] In the method provided by the embodiments of this disclosure, the first encoder and the second encoder share the same encoder. After encoding each sentence in the sequence segment, they are weighted and fused according to the positional relationship to obtain the first semantic vector. While reducing the number of model parameters and computational overhead, the encoders collaboratively learn sentence-level and word-level features, thereby improving the citation recognition efficiency and the overall generalization performance of the model.

[0080] In one possible implementation, the citation analysis model is obtained as follows: The initial first model is trained iteratively for multiple rounds until the iteration stopping condition is met. The first model that meets the iteration stopping condition is used as the citation analysis model. The first model includes a first sub-network and a second sub-network. The first sub-network and the second sub-network use a shared encoder. In this process, the first and second subnetworks are trained alternately in two adjacent training iterations. Each round of training iterations includes: From the pre-built sample dataset, obtain the sample set corresponding to this round of training. The sample set includes multiple sample documents. Each sentence in each sample document is labeled with a tag, which includes citations or non-citations. Each citation is labeled with the true semantic function category. Input the sample set into the sub-network trained in this round to obtain the prediction results of the sub-network's output; For all sample documents in the sample set, the total loss of this iteration is obtained based on the difference between the prediction results of the subnetwork output and the corresponding annotations of the sample documents. The parameters of the subnetwork and the encoder of this iteration are adjusted with the total loss to obtain the citation analysis model for the next iteration. When the subnetwork trained in this round is the first subnetwork, the prediction results output by the subnetwork include the label of each sentence in the sample document; When the subnetwork used in this round of training is the second subnetwork, the prediction results output by the subnetwork include the probability that each citation in the sample document belongs to each semantic function category.

[0081] Specifically, the embodiments disclosed herein employ an architecture in which the first sub-network and the second sub-network share an encoder, and perform joint optimization through an alternating training mechanism in adjacent rounds. On the one hand, this avoids gradient conflicts caused by the simultaneous backpropagation of two tasks, citation recognition and semantic function classification, reducing the optimization difficulty of multi-task learning; on the other hand, it enables the shared encoder to learn sentence-level contextual representations and lexical-level deep semantics separately in a unified semantic space, reducing the number of model parameters and improving training efficiency.

[0082] The specific training method for the citation analysis model is as follows: The initial first model is trained iteratively for multiple rounds until the iteration stopping condition is met. The first model that meets the iteration stopping condition is used as the citation analysis model. The first model includes a first sub-network and a second sub-network. The first sub-network and the second sub-network use a shared encoder.

[0083] The iteration stopping condition can be set according to actual needs, including but not limited to setting it to the total loss converges to below the threshold, the validation set F1-score no longer improves for multiple consecutive rounds, or the preset maximum number of iteration rounds is reached (such as 50 rounds).

[0084] In two adjacent training iterations, the first subnetwork and the second subnetwork are trained alternately. That is, if the first subnetwork is trained in the current round, the parameters of the second subnetwork are frozen; and vice versa.

[0085] Each round of iterative training includes: The sample set corresponding to this round of training is obtained from the pre-constructed sample dataset. The sample set includes multiple sample documents. Each sentence in each sample document is labeled with a tag, which includes citations or non-citations. Each citation is labeled with the true semantic function category.

[0086] The sample set corresponding to this round of training is obtained by randomly sampling from the sample dataset or by batching. The sample set is then input into the sub-network to be trained in this round to obtain the prediction result of the sub-network's output.

[0087] Understandable, Figure 2 This is one of the structural diagrams of a citation analysis model provided in an embodiment of this disclosure. The first sub-network is as follows: Figure 2 As shown, the first sub-network is used to perform subtask 1, realizing citation context segmentation and localization. The first sub-network includes an encoder and a sequence labeling module. When the sub-network trained in this round is the first sub-network, the encoder represents all sentences (sentence 1-sentence n) corresponding to the document as their respective first semantic vectors, and the sequence labeling module labels each sentence with corresponding tags (tag 1-tag n), obtaining a tag sequence. The prediction result output by the sub-network includes the tag of each sentence in the sample document, that is, the predicted tag of each sentence belonging to citation / non-citation.

[0088] Figure 3 This is a second schematic diagram of the structure of a citation analysis model provided in an embodiment of this disclosure. The second sub-network is as follows: Figure 3 As shown, the second sub-network is used to perform subtask 2, which is used to classify the semantic function categories of citations. The second sub-network includes an encoder and a classification layer. When the sub-network trained in this round is the second sub-network, the sentences whose labels are determined to be citations in the first sub-network are taken as citations, and m citations are obtained (citation 1 - citation m, m is less than n). The encoder represents each citation as its corresponding second semantic vector, and the classification layer obtains the category probability distribution corresponding to each citation. Figure 3 The probability distribution of the category m corresponding to the reference m is given by the middle category. The prediction results output by the sub-network include the probability that each reference in the sample document belongs to each semantic function category.

[0089] For all sample documents in the sample set, the total loss of this iteration is obtained based on the difference between the prediction results of the sub-network output and the corresponding annotations of the sample documents. It is understandable that the loss function used to calculate the loss value is different when training different sub-networks, and the specific type of loss function applied during the training of each sub-network can be determined according to actual needs.

[0090] For example, when training a sub-network, cross-entropy loss or conditional random field (CRF) loss function is used to calculate the difference between the predicted label and the true label; when training a second sub-network, the shared encoder extracts citation vocabulary features and cross-entropy loss or focal loss is used to calculate the difference between the predicted probability and the true semantic function category.

[0091] The parameters of the sub-network and the encoder in this iteration are adjusted using the total loss to obtain the citation analysis model for the next iteration.

[0092] During training, the AdamW, SGD, or RMSprop optimizer is used to jointly adjust the parameters of the training subnetwork and the shared encoder through the backpropagation algorithm, while keeping the parameters of the non-training subnetwork unchanged.

[0093] The method provided in this embodiment employs an alternating training strategy to perform multiple rounds of iterative training on the dual-task sub-network of the shared encoder, enabling citation recognition and semantic function classification tasks to mutually promote and synergistically optimize each other. This allows the encoder to simultaneously learn sentence boundary awareness and citation semantic understanding capabilities, enhancing the model's adaptability and generalization ability in different document scenarios.

[0094] In one possible implementation, when the subnetwork trained in this round is the second subnetwork, the total loss is the sum of the losses corresponding to the citations of all sample documents in the sample set; The loss for each citation is determined as follows: Obtain the confusion probability matrix of the application in this round of training. The confusion probability in the i-th row and j-th column of the confusion probability matrix is ​​used to indicate the probability that a citation in the pre-set validation set that is actually of the i-th semantic function category is predicted as the j-th semantic function category by the second sub-network. For citations, the probability of a citation belonging to a true semantic function category is predicted based on the second sub-network, and the basic loss is obtained. For each non-real category, calculate the product of the probability that the second sub-network predicts the citation belongs to the non-real category and the corresponding confusion probability, and sum the products corresponding to all non-real categories to obtain the comprehensive weight of the citation being identified as a non-real semantic function category. The base loss is updated based on the comprehensive weights to obtain the loss corresponding to the citation.

[0095] Specifically, in the training process of the second sub-network, this embodiment introduces a confusion probability matrix to dynamically adjust the loss function, and updates the base loss based on the historical confusion level between each category. This approach can adaptively increase the penalty for easily confused categories in citation semantic function classification, guiding the model to focus on semantic boundaries that are difficult to distinguish, and significantly improving classification accuracy and robustness.

[0096] When the subnetwork trained in this round is the second subnetwork, the total loss is the sum of the losses corresponding to the citations of all sample documents in the sample set; The loss for each citation is determined as follows: Obtain the confusion probability matrix for this round of training. The confusion probability in the i-th row and j-th column of the confusion probability matrix is ​​used to indicate the probability that a citation in the pre-set validation set that is actually of the i-th semantic function category is predicted by the second sub-network to be of the j-th semantic function category.

[0097] Understandably, the confusion probability matrix can be statistically generated on the validation set using scikit-learn or PyTorch / TensorFlow; the basic loss can be achieved using the cross-entropy loss function; the comprehensive weight calculation can be implemented using matrix multiplication; and the training framework supports PyTorch, TensorFlow, or PaddlePaddle, etc.

[0098] Before each round of training of the second sub-network, a confusion probability matrix is ​​calculated based on a pre-set validation set. The confusion probability matrix is ​​obtained based on the validation set. Multiple rounds (such as 10 iterations) can be considered as a stage. After the training of each stage, the confusion probability matrix is ​​updated with the updated model and validation set for the training of the next stage.

[0099] For a citation, the probability of the citation belonging to the true semantic function category is predicted based on the second sub-network to obtain the basic loss. The loss function for calculating the basic loss can be determined according to actual needs, including but not limited to negative log-likelihood loss (-log(p_true)) or cross-entropy loss.

[0100] Iterate through all non-real semantic function categories, multiply the probability that the second sub-network predicts the citation belongs to each non-real category by the confusion probability at the corresponding position (real category row, non-real category column) in the confusion probability matrix, and sum the multiplications corresponding to all non-real categories to obtain the comprehensive weight of the citation being identified as a non-real semantic function category.

[0101] The base loss is updated based on the comprehensive weights to obtain the loss corresponding to the citation. For example, the base loss is multiplied by (1 + comprehensive weights) or the weighted comprehensive weights are added to obtain the loss corresponding to the citation.

[0102] For example, the formula for calculating the loss of a single sample is: ; In the formula, pc is the predicted probability of the true semantic function category c of the sample citation after the model outputs Softmax; Mc,j is the confusion probability that the sample citation of the true semantic function category c is predicted to be of semantic function category j; pj is the predicted probability of the model predicting that the current sample citation belongs to semantic function category j.

[0103] The comprehensive weight, which is the sum of the products of the confusion probability and the prediction probability of all non-real category j, represents the weighted probability that the citation is identified as a non-real semantic function category. λ is the adaptive penalty term, and λ is a hyperparameter that controls the overall weight of the confusion probability penalty term.

[0104] The method provided in this embodiment calculates the comprehensive weight of the non-true categories based on the confusion probability matrix of the validation set and updates the basic loss, enabling the model to dynamically focus on easily confused categories during training, specifically suppressing high-frequency misjudgment patterns, effectively alleviating the problems of category imbalance and similar semantic interference, and significantly improving the robustness and accuracy of citation semantic function classification.

[0105] In one possible implementation, based on the first semantic vector of all statements, the statements belonging to the quotation are identified, including: Based on the first semantic vector of each statement and the first information of pre-training, a label is assigned to each statement in the statement sequence, and the label of each statement is either a citation or a non-citation. Each statement in the statement sequence that is tagged as a quotation is identified as a quotation. The first piece of information is used to represent the transfer pattern between labels of any adjacent statements, and is obtained by training the citation analysis model through a sequence of labeled statements.

[0106] Specifically, unlike classifying individual sentences in isolation, this embodiment of the disclosure introduces semantic coherence constraints between adjacent sentences and assigns labels to sentence sequences through global structured prediction.

[0107] Based on the first semantic vector of all statements, identify the statements belonging to the citation, including: After preprocessing the sentences in the document to be analyzed, the data is input into the citation analysis model to obtain the first semantic vector for each sentence. See again. Figure 2 The first sub-network includes a sequence labeling module. The sequence labeling module takes a sequence of sentences as input and assigns labels based on pre-trained information. When predicting the label for each sentence, the sequence labeling module considers not only the sentence's first semantic vector but also its semantic relevance to adjacent sentences and the global consistency of the entire sequence label distribution.

[0108] Understandably, the first piece of information is used to represent the transition pattern between labels of any adjacent statements. It is obtained by training on a large number of labeled statement sequences and is used to constrain the rationality of the label configuration of adjacent statements.

[0109] The sequence labeling module can use a Conditional Random Field (CRF), a Bidirectional Long Short-Term Memory Network combined with sequence labeling (BiLSTM-Tagging), or a Transformer-based sequence labeling model; global decoding can be implemented using the Viterbi algorithm or Beam Search.

[0110] For example, when adjacent statements have highly similar semantic vectors and are in the same paragraph, the sequence labeling layer tends to assign them the same label to maintain the integrity of the citation fragment; when a semantic mutation is detected, label switching is triggered.

[0111] The optimal label sequence that satisfies the contextual coherence constraint is obtained by using a global optimal decoding algorithm. Each sentence is assigned a citation or non-citation label, and sentences labeled as citations are identified as citation sentences.

[0112] The method provided in this embodiment introduces semantic coherence constraints between adjacent sentences, assigns labels to sentence sequences through global structured prediction, and makes full use of the distribution characteristics of citations that usually appear in paragraphs in the literature. This effectively suppresses the label fragmentation problem caused by semantic ambiguity of single sentences and significantly improves the coherence and accuracy of citation boundary recognition.

[0113] In one possible implementation, the first information includes the transition scores of all tag groups corresponding to adjacent statement tags, and the transition score of each tag group is used to indicate the probability of the previous tag in the tag group transitioning to the next tag; Based on the first semantic vector of each statement and the pre-trained first information, a label is assigned to each statement in the statement sequence, including: For each statement, based on the first semantic vector of the statement, determine the emission score corresponding to each label. The emission score corresponding to each label is used to indicate the confidence that the statement belongs to the label. Based on the emission score corresponding to each statement and the transition score between adjacent statement labels, the label sequence with the highest global total score is searched among all possible label sequences in the statement sequence. The label sequence is a sequence composed of the labels assigned to each statement in the statement sequence. The global total score is the sum of the emission scores and the transition scores under the corresponding label sequence.

[0114] Specifically, unlike the fragmentation problem caused by the traditional independent classification of single sentences, this embodiment introduces the transfer score constraint between adjacent sentence tags, transforming citation recognition into a global sequence labeling task.

[0115] The first information includes the transition scores of all tag groups corresponding to adjacent statement tags. The transition score of each tag group is used to indicate the probability of the previous tag moving to the next tag in the tag group. It includes the transition scores corresponding to the four tag groups: non-quote → non-quote, non-quote → quotation, quotation → non-quote, and quotation → quotation.

[0116] Based on the first semantic vector of each statement and the pre-trained first information, a label is assigned to each statement in the statement sequence, including: The literature to be analyzed is segmented and preprocessed before being input into the citation analysis model to obtain the first semantic vector for each sentence. Subsequently, the first semantic vector is mapped to the emission score corresponding to each sentence and each label (citation / non-citation). This score indicates the confidence that the sentence belongs to the corresponding label in the form of unnormalized log probability. The emission score can be calculated using structures such as fully connected layers or multilayer perceptrons (MLP).

[0117] Based on the emission score corresponding to each statement and the transition score between adjacent statement labels, the label sequence with the highest global total score is searched among all possible label sequences in the statement sequence. The label sequence is a sequence composed of the labels assigned to each statement in the statement sequence. The global total score is the sum of the emission scores and the transition scores under the corresponding label sequence.

[0118] It is understandable that searching for the label sequence with the highest global total score among all possible label sequences in the sentence sequence is achieved through a search algorithm. During the search process, the cumulative optimal score of each label at each position is calculated recursively and the transition path is recorded. The global optimal label sequence is obtained by backtracking in reverse, which is the global optimal solution of the label sequence. The search algorithms that can be used include, but are not limited to, the Viterbi dynamic programming algorithm or beam search.

[0119] The method provided in this embodiment introduces a transition score constraint between adjacent sentence tags, transforming citation recognition into a global sequence labeling task. By combining sentence-level semantic confidence with context tag transition rules, a global optimal search is performed, effectively ensuring the continuity and boundary accuracy of citation fragments and significantly improving the overall performance of citation recognition.

[0120] In one possible implementation, the sample dataset is generated as follows: Obtain the sample document and the corresponding citation information. The citation information includes the semantic function category of each citation in the sample document. For each sample document, identify the citation markers in the sample document. For each citation marker, expand outwards from the citation marker to both ends to obtain the complete sentence, and extract the citation corresponding to the citation marker. For each sample document, the sample document is used as a training sample, and the label corresponding to each sentence in the sample document and the semantic function category corresponding to each sample are used as training labels to construct a sample dataset.

[0121] Specifically, Figure 4 This is a schematic diagram of a citation semantic function classification process framework provided in an embodiment of the present disclosure, such as... Figure 4 As shown, in order to realize the mapping from full text of PDF to marked citation sentences, the disclosed embodiment designed and implemented a multi-stage data processing flow, as follows.

[0122] We obtain the original full-text PDFs of several academic papers to form a basic literature database, but we cannot obtain fine-grained citation intent annotations based solely on the original full-texts.

[0123] Further, annotation data corresponding to each document is obtained from a citation index database, including core fields such as document unique identifier (uid), publication year (pub_year), citation unique identifier (reference_uid), citation location (physical_location), and citation function. The function field represents the semantic function category that the model needs to learn and predict for training and validation in this embodiment. Based on the relevant information of the document, the original PDF full text of the document is aligned with the obtained annotation data to obtain sample documents and their corresponding citation information. The citation information includes the semantic function category of each citation in the sample document. For example, based on information such as DOI, title, author, and publication year, this paper accurately associates the full text of the document with its annotation data through precise metadata matching.

[0124] Citations in a document are located using citation tags, and the process extends outwards from the citation tags to obtain complete sentences, then extracts the citations corresponding to the citation tags. This embodiment employs an algorithm based on heuristic rules and regular expressions to automatically extract contextual sentences, i.e., citations, from the main text of each document.

[0125] This algorithm uses Optical Character Recognition (OCR) to parse and extract the main text, identifies and anchors citation markers (such as "[number]") within the text, and completes citation marker recognition. Centered on this marker, it extends outwards to the complete sentence boundaries to define the context window. Sentence integrity is then verified through syntactic parsing to ensure that the extracted text is a grammatically complete sentence or clause that explicitly mentions the cited literature.

[0126] The extracted results are cleaned and organized to remove records that failed to locate or had incomplete sentences, resulting in a high-quality dataset that can be used for model training.

[0127] In practical applications, a literature data pool covering the target technical field can be constructed through the above process according to the technical requirements of the actual application, and the annotation information corresponding to each literature can be obtained. A sample dataset can then be constructed based on the literature and annotation information.

[0128] For each sample document, the sample document is used as a training sample, and the labels corresponding to each sentence in the sample document and the semantic function category corresponding to each sample are used as training labels to construct a sample dataset. The citation analysis model is trained based on the sample dataset.

[0129] For any document to be analyzed, the task of determining the semantic function category of each citation in the document based on the unlabeled original PDF full text is divided into subtask 1 and subtask 2. The PDF full text is input into the citation analysis model, and the citation analysis model automatically completes subtask 1 and subtask 2 to obtain the semantic function category of each citation.

[0130] The method provided in this disclosure accurately associates the full text of a document with citation annotation records. Based on heuristic rules and regular expressions, it automatically identifies citation markers and extracts complete citation sentences, obtaining high-quality annotated data after data cleaning. A large-scale, multi-disciplinary sample dataset is constructed, providing sufficient and reliable training data for the citation semantic function classification model. This effectively supports the model's learning ability to recognize citation sentences and classify semantic functions, improving the model's generalization performance and classification accuracy in real-world document scenarios.

[0131] Despite significant progress made by deep learning methods in various sub-tasks of citation analysis, the following key gaps remain in current research: First, the focus is on a single task and there is a lack of unified modeling: Most works focus on a single task (only sentiment classification or only function classification), and there is a lack of research on unified modeling of quotation recognition and quotation semantic function classification. This fragmented approach not only increases the complexity of the system, but also easily accumulates errors in pipeline processing, affecting the overall performance.

[0132] Second, the assumption of structured input neglects end-to-end processing: Existing methods typically assume that the input is structured text (such as extracted citations or annotated XML files), while in real-world applications, researchers deal with raw PDF documents. How to develop a complete automated process from the raw PDF full text, through citation location identification, context segmentation, and citation semantic function classification, still lacks systematic research. This "data preprocessing-feature engineering-model training" pipeline not only increases system complexity but also limits the scalability and practicality of the method.

[0133] Third, the potential of end-to-end learning has not been fully explored: existing pipelined methods are prone to accumulating errors across subtasks. How to construct a unified deep learning framework to achieve end-to-end automated annotation from original documents to fine-grained citation tags is a significant challenge currently facing the field of citation analysis.

[0134] The citation semantic function classification method provided in this disclosure starts from the original full-text PDF document, constructs a multi-stage data processing pipeline, and combines the BERT pre-trained model or its variants to unify citation location identification, citation context extraction and citation semantic function category classification into an end-to-end automated citation analysis process, realizing fine-grained automatic annotation of citation context.

[0135] The constructed model was systematically trained and evaluated based on high-quality citation annotation data. Experimental results show that the model can reproduce expert annotation rules with high accuracy on the internal test set, and also shows a certain generalization ability on other external datasets.

[0136] We selected typical research areas to conduct application examples, and deployed the trained model to automatic citation analysis tasks on extended data sources. This demonstrated the application potential of this method in supporting large-scale, fine-grained academic citation analysis and related scientometric research, and provided a technical path that can be referenced for subsequent work.

[0137] Figure 5 This is a schematic diagram of the structure of a citation semantic function classification device provided in an embodiment of the present disclosure, as shown below. Figure 5 As shown, another aspect of this disclosure provides a citation semantic function classification device 50, comprising: The sentence sequence acquisition module 501 is used to obtain the sentence sequence of the document to be analyzed; The citation statement recognition module 502 is used to input the statement sequence into the citation analysis model, obtain the first semantic vector of each statement through the contextual dependency relationship between each statement in the statement sequence, and identify the statements belonging to the citation based on the first semantic vectors of all statements. The functional category classification module 503 is used to obtain the second semantic vector of each citation by using the citation analysis model to understand the contextual dependencies between words in the citation, and to perform semantic function classification on the citation based on the second semantic vector to obtain the corresponding semantic function category of the citation.

[0138] In one possible implementation, the first semantic vector of each statement is obtained based on the contextual dependencies between each statement in the statement sequence, including: For each statement, determine the paragraph to which the statement belongs in the document to be analyzed, and obtain the corresponding sequence segment from the statement sequence with the statement as the center. The sequence segment includes the statement and adjacent statements that belong to the same paragraph as the statement. The sequence fragment is input into the first encoder to obtain the first semantic vector of the statement. The first semantic vector is used to represent the semantic information of the corresponding statement and the semantic information of the adjacent statements of the corresponding statement.

[0139] In one possible implementation, for each citation, a second semantic vector of the citation is obtained based on the contextual dependencies between the words in the citation, including: For each citation, the citation is segmented to obtain a word sequence, where the starting position of the word sequence is the classification tag CLS; The word sequence is encoded in multiple rounds by the second encoder, and the target vectors of CLS output by each encoding operation in the last N rounds of encoding operations are combined into the first vector sequence. The first vector sequence is pooled to compress it into a second vector sequence, the length of which is less than N. The second vector sequence is linearly interpolated to obtain the third vector sequence, the length of which is N. The first vector sequence and the third vector sequence are added element by element and averaged to obtain the fused vector sequence. The second semantic vector of the citation is obtained by calculating the average of all vectors in the fused vector sequence.

[0140] In one possible implementation, each round of encoding operations includes: For this round of encoding operation, the initial vector of each element in the word sequence is determined. The elements include CLS and words. For the first round of encoding operation, the initial vector of each element is obtained by the encoder in advance. For subsequent rounds of encoding operation, the initial vector of each element is the target vector output by the previous round of encoding operation. For each element in the word sequence, the attention weight between each element and other elements is obtained based on the similarity between the initial vector of the element and the initial vectors of other elements in the word sequence. For each element in the word sequence, the initial vectors of all elements in the word sequence are weighted and aggregated according to the attention weight between the element and other elements to obtain the aggregated vector corresponding to the element. The initial vector of the element is updated with the aggregated vector to obtain the target vector of the element.

[0141] In one possible implementation, the first encoder and the second encoder are the same encoder; The first semantic vector of the statement corresponding to the sequence fragment is obtained in the following way: The encoder obtains the second semantic vector of each statement in the sequence segment; The weight of each statement in the sequence segment is determined based on the positional relationship between the statements in the sequence segment. The second semantic vectors of the corresponding statements are weighted and fused according to the weight of each statement to obtain the first semantic vector of the statement corresponding to the sequence segment.

[0142] In one possible implementation, the citation analysis model is obtained as follows: The initial first model is trained iteratively for multiple rounds until the iteration stopping condition is met. The first model that meets the iteration stopping condition is used as the citation analysis model. The first model includes a first sub-network and a second sub-network. The first sub-network and the second sub-network use a shared encoder. In this process, the first and second subnetworks are trained alternately in two adjacent training iterations. Each round of training iterations includes: From the pre-built sample dataset, obtain the sample set corresponding to this round of training. The sample set includes multiple sample documents. Each sentence in each sample document is labeled with a tag, which includes citations or non-citations. Each citation is labeled with the true semantic function category. Input the sample set into the sub-network trained in this round to obtain the prediction results of the sub-network's output; For all sample documents in the sample set, the total loss of this iteration is obtained based on the difference between the prediction results of the subnetwork output and the corresponding annotations of the sample documents. The parameters of the subnetwork and the encoder of this iteration are adjusted with the total loss to obtain the citation analysis model for the next iteration. When the subnetwork trained in this round is the first subnetwork, the prediction results output by the subnetwork include the label of each sentence in the sample document; When the subnetwork used in this round of training is the second subnetwork, the prediction results output by the subnetwork include the probability that each citation in the sample document belongs to each semantic function category.

[0143] In one possible implementation, when the subnetwork trained in this round is the second subnetwork, the total loss is the sum of the losses corresponding to the citations of all sample documents in the sample set; The loss for each citation is determined as follows: Obtain the confusion probability matrix of the application in this round of training. The confusion probability in the i-th row and j-th column of the confusion probability matrix is ​​used to indicate the probability that a citation in the pre-set validation set that is actually of the i-th semantic function category is predicted as the j-th semantic function category by the second sub-network. For citations, the probability of a citation belonging to a true semantic function category is predicted based on the second sub-network, and the basic loss is obtained. For each non-real category, calculate the product of the probability that the second sub-network predicts the citation belongs to the non-real category and the corresponding confusion probability, and sum the products corresponding to all non-real categories to obtain the comprehensive weight of the citation being identified as a non-real semantic function category. The base loss is updated based on the comprehensive weights to obtain the loss corresponding to the citation.

[0144] In one possible implementation, based on the first semantic vector of all statements, the statements belonging to the quotation are identified, including: Based on the first semantic vector of each statement and the first information of pre-training, a label is assigned to each statement in the statement sequence, and the label of each statement is either a citation or a non-citation. Each statement in the statement sequence that is tagged as a quotation is identified as a quotation. The first piece of information is used to represent the transfer pattern between labels of any adjacent statements, and is obtained by training the citation analysis model through a sequence of labeled statements.

[0145] In one possible implementation, the first information includes the transition scores of all tag groups corresponding to adjacent statement tags, and the transition score of each tag group is used to indicate the probability of the previous tag in the tag group transitioning to the next tag; Based on the first semantic vector of each statement and the pre-trained first information, a label is assigned to each statement in the statement sequence, including: For each statement, based on the first semantic vector of the statement, determine the emission score corresponding to each label. The emission score corresponding to each label is used to indicate the confidence that the statement belongs to the label. Based on the emission score corresponding to each statement and the transition score between adjacent statement labels, the label sequence with the highest global total score is searched among all possible label sequences in the statement sequence. The label sequence is a sequence composed of the labels assigned to each statement in the statement sequence. The global total score is the sum of the emission scores and the transition scores under the corresponding label sequence.

[0146] In one possible implementation, the sample dataset is generated as follows: Obtain the sample document and the corresponding citation information. The citation information includes the semantic function category of each citation in the sample document. For each sample document, identify the citation markers in the sample document. For each citation marker, expand outwards from the citation marker to both ends to obtain the complete sentence, and extract the citation corresponding to the citation marker. For each sample document, the sample document is used as a training sample, and the label corresponding to each sentence in the sample document and the semantic function category corresponding to each sample are used as training labels to construct a sample dataset.

[0147] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each disclosure embodiment correspond to the steps in the method of each disclosure embodiment. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0148] Furthermore, in this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Moreover, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0149] This disclosure provides an electronic device (computer device / equipment / system) including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure and achieve the corresponding technical effects.

[0150] In one alternative embodiment, an electronic device is provided. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 6 As shown, the electronic device 60 includes a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, for example, via a bus 602. Optionally, the electronic device 60 may further include a transceiver 604, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 604 is not limited to one type, and the structure of the electronic device 60 does not constitute a limitation on the embodiments of this disclosure.

[0151] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 601 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0152] Bus 602 may include a pathway for transmitting information between the aforementioned components. Bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 602 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0153] The memory 603 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0154] The memory 603 is used to store computer programs that execute embodiments of the present disclosure, and the execution is controlled by the processor 601. The processor 601 is used to execute the computer programs stored in the memory 603 to implement the steps shown in the foregoing method embodiments.

[0155] The electronic devices in this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, and fixed terminals such as digital TVs and desktop computers.

[0156] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0157] This disclosure also provides a computer program product, including a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments and achieve the corresponding technical effects.

[0158] It should be noted that the computer-readable storage medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0159] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0160] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0161] The terms “first,” “second,” “third,” “fourth,” “1,” “2,” etc. (if present) in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in a sequence other than that shown in the figures or text.

[0162] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0163] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A method for classifying the semantic function of citations, characterized in that, include: Obtain the sentence sequence of the document to be analyzed; The statement sequence is input into the citation analysis model. The citation analysis model obtains the first semantic vector of each statement based on the contextual dependencies between each statement in the statement sequence. Based on the first semantic vectors of all statements, the statements belonging to the citation are identified. For each citation, the second semantic vector of the citation is obtained by using the citation analysis model based on the contextual dependencies between the words in the citation. The citation is then classified into semantic functions based on the second semantic vector to obtain the corresponding semantic function category.

2. The method according to claim 1, characterized in that, The step of obtaining the first semantic vector of each statement based on the contextual dependencies between each statement in the statement sequence includes: For each statement, determine the paragraph to which the statement belongs in the document to be analyzed, and obtain the sequence segment corresponding to the statement from the statement sequence with the statement as the center. The sequence segment includes the statement and adjacent statements that belong to the same paragraph as the statement. The sequence fragment is input into a first encoder to obtain a first semantic vector of the statement. The first semantic vector is used to represent the semantic information of the corresponding statement and the semantic information of the adjacent statements of the corresponding statement.

3. The method according to claim 2, characterized in that, For each citation, obtaining the second semantic vector of the citation based on the contextual dependencies between the words in the citation includes: For each citation, the citation is segmented to obtain a word sequence, wherein the starting position of the word sequence is the classification tag CLS; The word sequence is encoded in multiple rounds by the second encoder, and the target vectors of CLS output by each encoding operation in the last N rounds of encoding operations are combined into a first vector sequence. The first vector sequence is pooled to compress it into a second vector sequence, the length of which is less than N. The second vector sequence is subjected to linear interpolation to obtain a third vector sequence, the length of which is N. The first vector sequence and the third vector sequence are added element by element and averaged to obtain the fused vector sequence; The average value of all vectors in the fused vector sequence is calculated to obtain the second semantic vector of the citation.

4. The method according to claim 3, characterized in that, Each round of encoding operations includes: For this round of encoding operation, the initial vector of each element in the word sequence is determined. The element includes CLS and words. For the first round of encoding operation, the initial vector of each element is obtained by the encoder in advance. For non-first round of encoding operation, the initial vector of each element is the target vector output by the previous round of encoding operation. For each element in the word sequence, the attention weight between each element and other elements is obtained based on the similarity between the initial vector of the element and the initial vectors of other elements in the word sequence. For each element in the word sequence, the initial vectors of all elements in the word sequence are weighted and aggregated according to the attention weight between the element and other elements to obtain the aggregated vector corresponding to the element. The initial vector of the element is updated with the aggregated vector to obtain the target vector of the element.

5. The method according to claim 3 or 4, characterized in that, The first encoder and the second encoder are the same encoder; The first semantic vector of the statement corresponding to the sequence fragment is obtained in the following way: The encoder obtains the second semantic vector of each statement in the sequence segment; The weight of each statement in the sequence segment is determined based on the positional relationship of each statement in the sequence segment. The second semantic vectors of the corresponding statements are weighted and fused according to the weights of each statement to obtain the first semantic vector of the statement corresponding to the sequence segment.

6. The method according to claim 5, characterized in that, The citation analysis model was obtained through the following method: The initial first model is trained iteratively for multiple rounds until the iteration stopping condition is met. The first model that meets the iteration stopping condition is used as the citation analysis model. The first model includes a first sub-network and a second sub-network. The first sub-network and the second sub-network use a shared encoder. In this process, the first and second subnetworks are trained alternately in two adjacent training iterations. Each round of training iterations includes: From the pre-constructed sample dataset, obtain the sample set corresponding to this round of training. The sample set includes multiple sample documents. Each sentence in each sample document is labeled with a tag. The tag includes citations or non-citations. Each citation is labeled with the true semantic function category. The sample set is input into the sub-network trained in this round to obtain the prediction result of the sub-network's output; For all sample documents in the sample set, the total loss of this iteration is obtained based on the difference between the prediction result of the sub-network output and the corresponding annotation of the sample document. The parameters of the sub-network and the encoder of this iteration are adjusted with the total loss to obtain the citation analysis model for the next iteration. When the subnetwork trained in this round is the first subnetwork, the prediction result output by the subnetwork includes the label of each sentence in the sample document; When the subnetwork trained in this round is the second subnetwork, the prediction results output by the subnetwork include the probability that each citation in the sample document belongs to each semantic function category.

7. The method according to claim 6, characterized in that, When the subnetwork trained in this round is the second subnetwork, the total loss is the sum of the losses corresponding to the citations of all sample documents in the sample set; The loss for each citation is determined as follows: Obtain the confusion probability matrix of the application in this round of training. The confusion probability in the i-th row and j-th column of the confusion probability matrix is ​​used to indicate the probability that a citation in the pre-set validation set that is actually of the i-th semantic function category is predicted by the second sub-network as the j-th semantic function category. For the quoted text, the probability that the quoted text belongs to the true semantic function category is predicted based on the second sub-network, and the basic loss is obtained; For each non-real category, the product of the probability that the second sub-network predicts the citation belongs to the non-real category and the corresponding confusion probability is calculated, and the products corresponding to all non-real categories are summed to obtain the comprehensive weight of the citation being identified as a non-real semantic function category. The base loss is updated based on the comprehensive weights to obtain the loss corresponding to the citation.

8. The method according to any one of claims 1-4, characterized in that, The step of identifying statements belonging to a quotation based on the first semantic vector of all statements includes: Based on the first semantic vector of each statement and the first information pre-trained, a label is assigned to each statement in the statement sequence, and the label of each statement is either a citation or a non-citation. Each statement in the statement sequence that is tagged as a quotation is identified as a quotation statement; The first information is used to represent the transfer pattern between labels of any adjacent statements, and is obtained by training the citation analysis model through a sequence of labeled statements.

9. The method according to claim 8, characterized in that, The first information includes the transition scores of all tag groups corresponding to adjacent statement tags, and the transition score of each tag group is used to indicate the probability of the previous tag in the tag group being transferred to the next tag; The process of assigning a label to each statement in the statement sequence based on the first semantic vector of each statement and pre-trained first information includes: For each statement, based on the first semantic vector of the statement, the emission score corresponding to each label is determined, and the emission score corresponding to each label is used to indicate the confidence that the statement belongs to the label; Based on the emission score corresponding to each statement and the transition score between adjacent statement labels, the label sequence with the highest global total score is searched among all possible label sequences of the statement sequence. The label sequence is a sequence composed of labels assigned to each statement in the statement sequence. The global total score is the sum of the emission scores and the transition scores under the corresponding label sequence.

10. The method according to claim 7, characterized in that, The sample dataset was generated in the following way: Obtain sample documents and the citation information corresponding to the sample documents, wherein the citation information includes the semantic function category of each citation in the sample documents; For each sample document, identify the citation markers in the sample document. For each citation marker, expand outwards from the citation marker to obtain a complete sentence, and extract the citation corresponding to the citation marker. For each sample document, the sample document is used as a training sample, and the label corresponding to each sentence in the sample document and the semantic function category corresponding to each sample are used as training labels to construct a sample dataset.

11. A citation semantic function classification device, characterized in that, include: The sentence sequence acquisition module is used to obtain the sentence sequence of the document to be analyzed; The citation statement recognition module is used to input the statement sequence into the citation analysis model, obtain the first semantic vector of each statement through the citation analysis model based on the contextual dependencies between each statement in the statement sequence, and identify the statements belonging to the citation based on the first semantic vectors of all statements. The functional category classification module is used to obtain the second semantic vector of each citation by using the citation analysis model and the contextual dependencies between the words in the citation, and to perform semantic function classification on the citation based on the second semantic vector to obtain the semantic function category corresponding to the citation.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

Images