A citation labeling method

Abstract

The application belongs to the field of text processing, and particularly relates to a citation annotation method, aiming to solve the problem of poor recognition effect of citation missing phenomenon. The method comprises the following steps: segmenting a to-be-annotated text into multiple subtexts and determining the position identifier of any subtext; in the case that a citation annotation symbol is recognized in any subtext, determining that the citation type identifier thereof is a first identifier; in the case that any subtext matches a citation description set comprising multiple citation description structures successfully, determining that the citation type identifier thereof is a second identifier, the citation description structure being obtained by screening from alternative citation description structures, the alternative citation description structure being obtained by performing citation description structure recognition on the subtexts of multiple to-be-annotated texts; and performing citation annotation in the to-be-annotated text based on the citation type identifier and the position identifier of each subtext. The application can annotate all citation information of the text, and improve the efficiency and intelligent level of citation annotation in various application scenarios.

Description

Technical Field

[0001] This application belongs to the field of text processing, and specifically relates to a citation annotation method. Background Technology

[0002] In academic papers and other written works, citing others' research results, data, academic viewpoints, etc., clearly indicates the original source of the content, demonstrates respect for previous academic research, and avoids plagiarism.

[0003] Existing citation annotation methods are mainly based on author-initiated annotation, such as authors adding citation symbols and references to their papers. However, many papers commonly suffer from "using but not citing," i.e., missing citations.

[0004] Existing deep learning-based citation annotation methods lack effective ways to obtain citation context features, resulting in poor automatic annotation performance and potentially slow annotation speed, leading to low efficiency. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, namely the poor performance in identifying missing citations, this application proposes a citation annotation method, which includes:

[0006] Step S10: Based on the layout of the text to be annotated, extract the original annotation information of the text to be annotated, wherein the original annotation information includes citation annotations;

[0007] Step S20: Divide the text to be annotated into multiple sub-texts and determine the position identifier of any sub-text;

[0008] Step S30: If a citation marker is identified in any subtext, determine that the citation type identifier of any subtext is the first identifier;

[0009] Step S40: If any subtext successfully matches the reference description set, determine the reference type identifier of any subtext as the second identifier. The reference description set consists of multiple reference description structures. The reference description structure is obtained by filtering from the candidate reference description structures based on the reference type of the reference text and the number of candidate reference description structures. The reference text is the text used for reference, including the original annotation information. The candidate reference description structures are obtained by identifying the reference description structure of the subtexts of multiple texts to be annotated.

[0010] Step S50: Based on the citation type identifier and position identifier of each sub-text, perform citation annotation in the text to be annotated.

[0011] In some embodiments, the citation description structure is obtained by filtering from the candidate citation description structures based on the citation type of the referenced text and the number of candidate citation description structures, including:

[0012] Retrieve multiple reference texts and determine the reference reference type identifier of any reference subtext within the reference texts;

[0013] Based on the reference type identifier, the reference subtext is divided into the first reference subtext and the second reference subtext;

[0014] Filter target reference subtexts with alternative reference description structures from the first reference subtext, and determine the hit index of alternative reference description structures based on the reference type of the target reference subtext.

[0015] The number of alternative citation description structures included in the second reference subtext is counted, and an exclusion index for alternative citation description structures is determined based on the number.

[0016] Based on the hit index and exclusion index, determine the effective hit index of the candidate reference description structure;

[0017] The candidate reference description structures are filtered based on the effective hit index to obtain the reference description structures.

[0018] In some embodiments, the hit index of candidate reference description structures is determined based on the reference type of the target reference subtext, including:

[0019] Determine the weight coefficient for any reference type;

[0020] Obtain the product of the weight coefficients of each reference type included in any target reference subtext, and use it as the hit confidence value of any target reference subtext;

[0021] The sum of the hit trust values ​​of each target reference subtext is obtained as the hit index.

[0022] In some embodiments, determining the exclusion index for alternative reference description structures based on quantity includes:

[0023] Determine the exclusion trust value for the second reference subtext;

[0024] The sum of the exclusion trust values ​​of each second reference subtext is obtained as the exclusion index.

[0025] In some embodiments, the effective hit index of the candidate reference description structure is determined based on the hit index and the exclusion index, including:

[0026] The sum of the exclusion index and the preset value is used as the target exclusion index;

[0027] The effective hit index is the ratio of the hit index to the target exclusion index.

[0028] In some embodiments, candidate reference description structures are filtered based on an effective hit index to obtain reference description structures, including:

[0029] If the effective hit index is greater than or equal to the effective hit threshold, the candidate reference description structure is determined as the reference description structure.

[0030] In some embodiments, based on a reference reference type identifier, the reference subtext is divided into a first reference subtext and a second reference subtext, including:

[0031] If the reference type identifier is not empty, mark the referenced subtext as the first referenced subtext;

[0032] If the reference type identifier is empty, the referenced subtext is marked as the second referenced subtext.

[0033] In some embodiments, the alternative citation description structure is obtained by identifying the citation description structure of subtexts of multiple texts to be annotated, including:

[0034] Input multiple de-tagged subtexts into a large model, and based on the prompt words identified by the citation description structure, obtain the citation fragment of any de-tagged subtext and output the citation description structure. Here, the de-tagged subtext is the subtext with citation tags removed.

[0035] If the location of the subtext's position marker overlaps with the location of the referenced fragment, the output reference description structure is determined as the alternative reference description structure.

[0036] In some embodiments, determining the location identifier of any subtext includes:

[0037] Determine the first and last characters of any subtext;

[0038] The coordinates formed by the first and last characters are used as position identifiers for any subtext.

[0039] In some embodiments, citation annotation is performed in the text to be annotated based on the citation type identifier and position identifier of each subtext, including:

[0040] Locate the target position of the subtext based on the location identifier;

[0041] The citation is annotated in the corresponding form at the target location based on the citation type identifier.

[0042] The beneficial effects of this application are:

[0043] (1) Based on the layout of the text to be annotated, the original annotation information is extracted, and the author's self-annotated citation information in the text to be annotated can be obtained; the text to be annotated is divided into multiple sub-texts, and the entire text can be divided into multiple small sub-texts as the smallest unit for text processing, and the position identifier of any sub-text can be determined, so as to determine the position information of each sub-text for text positioning; when a citation mark is identified in any sub-text, the citation type identifier of any sub-text is determined as the first identifier, and the author's self-annotated citation content and citation type can be determined according to the original annotation information; any sub-text is matched with the citation description set, wherein the citation description set consists of multiple citation description structures, and the citation description structure is selected from the candidate citation description structures based on the citation type of the reference text and the number of candidate citation description structures. The obtained reference text is text used for reference, including the original annotation information. The alternative citation description structure is obtained by identifying the citation description structure of multiple subtexts of the text to be annotated. By matching any subtext with the citation description structure, it is determined whether there is a citation type of the citation description structure. If the match is successful, it means that there is a citation of the citation description structure in any subtext, and the corresponding citation type is identified as the second identifier. This can identify the phenomenon of missing citations and make up for the shortcomings of self-annotation. Based on the citation type identifier and position identifier of each subtext, citation annotation is performed in the text to be annotated. It can identify and annotate both the citation content self-annotated by the author and the citation content not annotated in the text to be annotated. It can clearly obtain and annotate all citation information of the text, improving the efficiency and intelligence of citation annotation in various application scenarios.

[0044] (2) This application can detect multiple types of citations at the same time. Citation identification does not rely solely on author self-annotation. It can effectively identify the situation of "using but not citing". The content of citation annotation is comprehensive and has a wide range of applications. Attached Figure Description

[0045] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0046] Figure 1 This is a flowchart illustrating a citation annotation method provided in one embodiment of this application;

[0047] Figure 2 This is a flowchart of one step of a citation annotation method provided in one embodiment of this application;

[0048] Figure 3 This is a system block diagram of a citation annotation system provided in one embodiment of this application. Detailed Implementation

[0049] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0050] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0051] A citation annotation method according to a first embodiment of this application includes steps S10-S50, such as... Figure 1 As shown, the method includes:

[0052] Step S10: Based on the layout of the text to be annotated, extract the original annotation information of the text to be annotated, wherein the original annotation information includes citation annotations.

[0053] Optional, layout refers to the page format, specifically the size of the text area and surrounding blank space, the font, font size, layout style of the text, the number of words, their position, as well as the arrangement of items such as the table of contents and titles, footnotes, tables, figure titles, figure captions, punctuation marks, running heads, page numbers, and page decorations.

[0054] In the embodiments of this application, the format of the text to be annotated includes at least the main text, footnotes, endnotes, and references.

[0055] Furthermore, based on the layout of the text to be annotated, the original annotation information of the self-annotated text is extracted, wherein the original annotation information includes at least the reference annotation.

[0056] In the embodiments of this application, citation markers are symbols or formats used to indicate the source of a citation, such as bracketed citations (author, year), citation numbers, footnotes, endnotes, abbreviation systems (ibid.), etc.

[0057] In other embodiments, the original annotation information may also include identifiers, such as double quotes, for directly referencing the original text of the citation.

[0058] Step S20: Divide the text to be annotated into multiple sub-texts and determine the position identifier and reference type identifier of any sub-text.

[0059] Optionally, the delimiter for the text to be annotated can be determined based on the language. For example, full-width characters “.”, “!”, and “?” can be used to delimit Chinese text.

[0060] As an example, consider the following segment:

[0061] On that evening, bathed in the orange glow of the setting sun, the streets of the small town were filled with the faint fragrance of flowers and the warm aroma of freshly baked bread. A group of children chased and played, their laughter clear and melodious, like the most beautiful music of a summer morning! Under an old locust tree by the roadside, an old man leisurely fanned himself, his eyes revealing a deep longing for bygone days.

[0062] The text is split using a delimiter, resulting in three sub-texts:

[0063] ①On that evening, when the setting sun painted the town orange-red, the streets of the small town were filled with the faint fragrance of flowers and the warm aroma of freshly baked bread.

[0064] ②A group of children chased and played, their laughter clear and melodious, like the most beautiful music on a summer morning!

[0065] ③Under an old locust tree by the roadside, an old man leisurely fanned himself, his eyes revealing a deep longing for the past.

[0066] Optionally, the location identifier of any subtext can be obtained by locating the text.

[0067] As one possible implementation, the first character position B and the last character position E of any subtext are determined; the first character position and the last character position are combined to form coordinates (B, E), which serve as the position identifier of any subtext.

[0068] For the above three sub-texts, sub-text ① contains 41 characters, sub-text ② contains 32 characters, and sub-text ③ contains 37 characters. Then the position identifiers (B,E) of the above three sub-texts are (0,40), (41,72) and (73,109) respectively.

[0069] It should be noted that after step S20 is executed, steps S30 and S40 are executed in parallel.

[0070] Step S30: If a citation marker is identified in any subtext, determine the citation type identifier of any subtext as the first identifier.

[0071] Optionally, if a citation marker is identified in any subtext, the corresponding citation type can be determined and the corresponding citation type identifier can be obtained.

[0072] For example, if a footnote or endnote citation marker is detected in any subtext, it indicates that the subtext contains a citation type with footnote and endnote number information; if a square bracket or parenthesis citation marker is detected in any subtext, it indicates that the subtext contains a reference citation type.

[0073] As one possible implementation, the location of footnote / endnote numbers is searched in any subtext, and if the presence of footnote / endnote numbers is detected, the reference type is determined to be a footnote / endnote reference.

[0074] As an example, the text content included in any subtext is as follows:

[0075] On that evening, bathed in the orange glow of the setting sun, the streets of the small town were filled with the faint fragrance of flowers and the warm aroma of freshly baked bread. A group of children chased and played, their laughter clear and melodious, like the most beautiful music of a summer morning! Under an old locust tree by the roadside, an old man leisurely fanned himself, his eyes revealing a deep longing for bygone days. 1

[0076] 1. Author 1: Example Title 1, Example Publisher, 2023.

[0077] Based on the above, the third sentence is a footnote, and it contains a footnote symbol at the end, which confirms that the citation type in the above text is a footnote or endnote.

[0078] As one possible implementation, search any subtext, locate all citation marks using square brackets, obtain the citation mark number, and search in the "References" of the text to be annotated. If found, determine the citation type as a reference citation.

[0079] As another possible implementation, search any subtext, locate all citation marks using parentheses, obtain the author name + year within the parentheses, and search in the "References" of the text to be annotated. If there is a reference that matches both the author name and year, determine the citation type as a reference citation.

[0080] As another possible implementation, the full text of each reference in the text to be annotated is obtained, and any subtext is compared with each reference for similarity. If there is overlapping text, the citation type is determined to be a reference citation, and the overlapping position with each reference is the citation position.

[0081] As an example, the text content included in any subtext is as follows:

[0082] {On that evening, when the setting sun painted the streets orange-red, the town was filled with the faint fragrance of flowers and the warm aroma of freshly baked bread[2]. A group of children chased and played, their laughter clear and melodious, like the most beautiful music of a summer morning (author 2.2023). Under the old locust tree by the roadside, an old man leisurely fanned himself, his eyes revealing a deep longing for the past.}

[0083] References

[0084] [1] Author 1. Example Name 1 [J]. Example Publisher, 2023, (12): 129-130.

[0085] [2] Author 2. Example Name 2 [J]. Example Publisher, 2023, (12): 129-130.

[0086] [3] Author 3. Example Title 3 [J]. Example Publisher, 2023, (12): 129-130.

[0087] Understandably, both the first and second sentences contain pre-defined citations, namely square bracket citations and parenthesis citations.

[0088] Furthermore, the full text of the third reference is extracted and compared with the original text. It is found that the third sentence is highly similar to a certain passage in the original text, so it is determined to be a reference citation.

[0089] In other embodiments, a knowledge base can be established, and any subtext can be retrieved within the knowledge base. If similar data exists, the reference type is determined to be a database reference.

[0090] The knowledge base includes common-sense texts such as axioms, concepts, policies, legal provisions, ancient poems, and famous works.

[0091] As one possible implementation, any subtext is retrieved from the knowledge base. If any subtext has an overlap in the knowledge base, the reference type of any subtext is determined to be a knowledge base reference.

[0092] In other embodiments, reference type determination can also be performed by recognizing identifier symbols.

[0093] As one possible implementation, traverse any subtext, and if double quotes are present, determine that the reference type is a punctuated reference.

[0094] As an example, the text content included in any subtext is as follows:

[0095] An economist pointed out in his research: "The rapid development and widespread application of automation technology, while improving production efficiency to some extent, may also lead to the disappearance of some traditional occupations, thereby triggering changes in the employment structure." This statement profoundly reveals the complex relationship between technological progress and the job market.

[0096] By traversing the above text and identifying double quotation marks, the text citation type is determined to be a punctuated citation. Optionally, the citation type TYPE is distinguished by a citation type identifier.

[0097] In the embodiments of this application, the first identifier includes the citation type identifier for footnotes and endnotes, and the citation type identifier for references.

[0098] As one possible implementation, a reference type identifier table is queried to determine the reference type identifier of any subtext, wherein each reference type in the reference type identifier table corresponds to a uniquely determined reference type identifier.

[0099] As an example, the citation type identifier for footnote and endnote citations is 'a', and the citation type identifier for bibliographic citations is 'b'.

[0100] Furthermore, in this embodiment, a quadruple can be set for each subtext to describe the citation annotation of each subtext. The quadruple is represented as (ID, (B, E), TYPE, MESSAGE), where ID is the unique identifier of the subtext in the text to be annotated, (B, E) is the position of the subtext in the text to be annotated, TYPE is the identifier of the citation type present in the subtext, and MESSAGE is supporting information for the citation type identifier TYPE.

[0101] As an example, search for the footnote / endnote number position in the text to be annotated, locate the subtext containing the footnote / endnote, add the reference type identifier 'a' to the TYPE of the corresponding quadruple of the subtext, and add the footnote / endnote information to the MESSAGE of the corresponding quadruple of the subtext.

[0102] For example, in the footnote citation example above, the third sentence is a footnote, denoted by ID 3, position identifier (73,109), and citation type identifier a. The corresponding quadruple is (3,(73,109),[a],[Author 1: Example Name 1, Example Publisher, 2023.]).

[0103] As another example, search the text to be annotated, locate all citation marks using square brackets, obtain the citation mark number, search in the "References" section of the text to be annotated, and if found, add the citation type identifier b to the TYPE of the quadruple corresponding to the subtext, and add the corresponding reference entry to the MESSAGE of the quadruple corresponding to the subtext.

[0104] As another example, search the text to be annotated, locate all citation markers using parentheses, obtain the author name + year within the parentheses, and search in the "References" section of the text to be annotated. If a reference exists that matches both the author name and year, add the citation type identifier b to the TYPE of the quadruple corresponding to the subtext, and add the corresponding reference entry to the MESSAGE of the quadruple corresponding to the subtext.

[0105] As another example, obtain the full text of each reference in the text to be annotated, compare the similarity between the subtext and each reference, if there is overlapping text, determine the overlapping position in the subtext and each reference as the citation position, add the citation type identifier b to the TYPE of the quadruple corresponding to the subtext, and add the corresponding reference entry to the MESSAGE of the quadruple corresponding to the subtext.

[0106] For example, in the above reference citation example, the position identifiers (B,E) of the three subtexts are (0,40), (41,72) and (73,109) respectively, with corresponding IDs of 1, 2 and 3 respectively. The citation type identifier is b, and the corresponding quadruples are (1,(0,40),[b],[Author 1. Example Name 1[J]. Example Publisher, 2023,(12):129-130.]), (2,(41,72),[b],[Author 2. Example Name 2[J]. Example Publisher, 2023,(12):129-130.]), (3,(73,109),[b],[Author 3. Example Name 3[J]. Example Publisher, 2023,(12):129-130.]).

[0107] In other embodiments, the reference type identifier of the database reference and the reference type identifier of the identifier reference may be designated as the first identifier.

[0108] As an example, the reference type identifier for a knowledge base reference is c, and the reference type identifier for a reference to an identifier is d.

[0109] As one possible implementation, a knowledge base is established. Any subtext is searched within the knowledge base. If similar data exists, the reference type identifier c is added to the TYPE of the quadruple corresponding to the subtext, and the matched knowledge base data information is added to the MESSAGE of the quadruple corresponding to the subtext.

[0110] As another possible implementation, iterate through the text to be annotated, obtain the position range of each pair of double quotes, compare the text content of each pair of double quotes position range with any subtext, if there is an overlap, add the reference type identifier d to the TYPE of the quadruple corresponding to the subtext, calculate the position information of the overlap position relative to each subtext, and add the relative position range to the MESSAGE of the subtext.

[0111] For example, in the above example of annotation and citation, the fragment is divided into two sub-texts with segmentation positions (B,E) of [0,75] and [76,100], respectively. The double quotation mark position range is [(14,75)]. By cross-checking the two sub-texts with the double quotation mark position, it is found that the first sub-text has an intersection with the double quotation mark position range. Therefore, the sub-text has an annotation and citation, and the quadruple is updated to (1,(0,75),[d],[(14,75)]).

[0112] Step S40: If any subtext successfully matches the reference description set, determine the reference type identifier of any subtext as the second identifier. The reference description set consists of multiple reference description structures. The reference description structure is obtained by filtering from the candidate reference description structures based on the reference type of the reference text and the number of candidate reference description structures. The reference text is the text used for reference, including the original annotation information. The candidate reference description structures are obtained by identifying the reference description structure of the subtexts of multiple texts to be annotated.

[0113] Optionally, any subtext can be matched with the set of reference descriptions. If a match is found, it can be determined that the reference type of the subtext is a reference description structure reference, and the reference type identifier of any subtext can be determined as the second identifier.

[0114] In this embodiment of the application, the second identifier includes a reference type identifier for the reference description structure, and the reference type identifier for the text description reference type is e.

[0115] The alternative reference description structure is obtained in the following way:

[0116] Multiple de-tagged subtexts are input into a large model. Based on the prompt words identified by the citation description structure, the citation fragment and output citation description structure of any de-tagged subtext are obtained. The de-tagged subtext is the subtext with the citation markers removed. When the position of the subtext's position marker overlaps with the position of the citation fragment, the output citation description structure is determined as the candidate citation description structure.

[0117] As one possible implementation, multiple texts to be labeled are obtained, and multiple sub-texts and corresponding quadruples are obtained through label recognition and text segmentation. The reference labels of the sub-texts are removed to obtain unlabeled text, and the position identifier is re-obtained to establish a mapping quadruple (ID, (B1, E1), TYPE, MESSAGE).

[0118] As an example, a certain subtext is:

[0119] On the one hand, digital technology has broken the limitations of traditional education in terms of time and space. For example, online education platforms enable students in remote areas to access high-quality educational resources, which to some extent narrows the educational gap between urban and rural areas and between regions. As UNESCO pointed out in a report, "Online education provides new ways of learning for those who cannot access traditional education opportunities"[2]. Through Internet technology, students can participate in course learning anytime and anywhere, and interact with teachers and classmates around the world, which greatly broadens their learning horizons and channels.

[0120] The quadruple corresponding to this subtext can be represented by a quadruple table, as shown in Table 1.

[0121] Table 1

[0122]

[0123] The unmarked subtext after removing the quotation marks is:

[0124] On the one hand, digital technology has broken down the time and space limitations of traditional education. For example, online education platforms enable students in remote areas to access high-quality educational resources, which to some extent narrows the educational gap between urban and rural areas and between different regions. As a UNESCO report pointed out, online education provides new learning pathways for those who cannot access traditional education. Through internet technology, students can participate in course learning anytime, anywhere, and interact with teachers and classmates around the world, greatly broadening their learning horizons and channels.

[0125] Because the explicit quotation marks, double quotes and square brackets, are removed, the character positions change, resulting in a new quadruple, which can be represented by a mapping quadruple table, as shown in Table 2.

[0126] Table 2

[0127]

[0128] Furthermore, by applying prompt words and using large model text generation technology, the quoted fragments within the untagged subtext are obtained.

[0129] As an example, the prompt words used are:

[0130] Determine if the following text contains a quote. If so, output the full quote and provide the characteristics that indicate the original text is a quote. On the one hand, digital technology has broken down the time and space limitations of traditional education. For example, online education platforms enable students in remote areas to access high-quality educational resources, which to some extent narrows the educational gap between urban and rural areas and between different regions. As a UNESCO report pointed out, online education provides new learning pathways for those who cannot access traditional education. Through internet technology, students can participate in course learning anytime, anywhere, and interact with teachers and classmates worldwide, greatly broadening their learning horizons and channels.

[0131] Among them, ### is used to separate instructions and content, and enhance the understanding of prompt words by the large model. In the above prompt words, ### separates the instructions from the text that needs to be judged by the large model.

[0132] The large model returned the following results:

[0133] This passage contains a quote; the full quote is as follows:

[0134] "As UNESCO noted in a report, online education provides new learning pathways for people who do not have access to traditional education."

[0135] The characteristic of identifying a quote in the original text is: "...in which it is stated...".

[0136] It should be noted that the de-tagged subtext does not contain obvious citation markers such as double quotes and square brackets. The citation recognition result is obtained by a large model through pre-training and fine-tuning on massive amounts of data. This method is slow and not suitable for high-concurrency, real-time, and massive call processing. Moreover, it is extremely costly and cannot be applied in practice. Therefore, it is not possible to directly use a large model for citation annotation.

[0137] On the other hand, if the citation markers of the subtext, such as double quotes and square brackets [2], are not removed, the large model will give these two explicit, conventional citation features, while ignoring the hidden, unknown descriptive features. Therefore, in order to ensure the comprehensiveness of citation recognition, the input of the large model is the de-tagged subtext.

[0138] Furthermore, the position of the reference fragment generated by the large model in the de-tagged subtext is determined and compared with each new quadruple. If the position of the location marker in the new quadruple overlaps with the position of the reference fragment, the output reference description structure is determined as the candidate reference description structure.

[0139] Taking the above large model recognition results as an example, the quoted segment in the untagged subtext is: "As the United Nations... new approach." The position in the tagged subtext is (84, 132), and the position of the location identifier in the new quadruple is (84, 132). Since the two positions overlap, the citation feature "...indicates in..." is taken as the candidate citation description structure.

[0140] Understandably, this method uncovers implicit reference description features. Furthermore, since the sentence is indeed a reference, it enhances the probability that the structure is a reliable reference representation. However, this is an isolated case, and it cannot be definitively identified as a reference description structure based solely on a single judgment by the larger model. Therefore, it is considered a candidate reference description structure and will be further filtered and verified using subsequent methods.

[0141] In this embodiment of the application, multiple texts including original annotation information are obtained as reference texts. Based on the reference type of the reference texts and the number of candidate reference description structures, the candidate reference description structures are filtered to obtain reference structure descriptions and form a reference description set.

[0142] Please see Figure 2 The specific steps to obtain the reference structure description include:

[0143] Step S401: Obtain multiple reference texts and determine the reference reference type identifier of any reference subtext in the reference texts.

[0144] Optionally, a massive collection of papers can be obtained. Since each paper contains original annotation information, any paper can be used as a reference text. By performing text segmentation on the reference text, reference subtexts can be generated, and a quadruple can be constructed for each reference subtext to determine the reference citation type identifier of any reference subtext.

[0145] Step S402: Based on the reference reference type identifier, the reference subtext is divided into a first reference subtext and a second reference subtext.

[0146] Optionally, if the reference type identifier is not empty, the reference subtext is marked as the first reference subtext; if the reference type identifier is empty, the reference subtext is marked as the second reference subtext.

[0147] It is understandable that the first reference subtext contains obvious citation markers, i.e., the reference type is one or more of a, b, c, and d; the second reference subtext does not contain obvious citation markers, and is initially identified as not containing any citations.

[0148] Step S403: Filter target reference subtexts with candidate reference description structures from the first reference subtext, and determine the hit index of candidate reference description structures based on the reference type of the target reference subtext.

[0149] Optionally, determining whether there is a reference description structure in the first reference subtext can be done in various ways, such as regular expression matching: converting "...in...in" into the regular expression ".*?in.*?in".

[0150] It should be noted that by filtering target reference subtexts in the first reference subtext through regular expression matching, the recognition of large models (with a single recognition time of more than seconds) is transformed into simple regular expressions (less than milliseconds).

[0151] Furthermore, determine the weight coefficient corresponding to any reference type; obtain the product of the weight coefficients of each reference type included in any target reference subtext as the hit confidence value of any target reference subtext; obtain the sum of the hit confidence values ​​of each target reference subtext as the hit index.

[0152] Optionally, each reference type can be assigned a corresponding weight coefficient. The weight coefficient is greater than 0 and is set based on prior knowledge. For example, based on experience, when using double quotes, there is a greater likelihood that a fixed sentence structure describing the reference will appear before it, so a high weight is assigned to identify this reference type.

[0153] In this embodiment of the application, the weight coefficient corresponding to footnote and endnote citations (i.e., citation type identifier a) is set to 2, the weight coefficient corresponding to reference citations (i.e., citation type identifier b) is set to 2, the weight coefficient corresponding to knowledge base citations (i.e., citation type identifier c) is set to 4, and the weight coefficient corresponding to identifier citations (i.e., citation type identifier d) is set to 6.

[0154] It should be noted that the above-mentioned weighting coefficient setting is only an example. In other embodiments, it can be set to other values, or it can be optimized to a value that can be dynamically adjusted. This application does not limit this.

[0155] Optionally, the hit trust value v can be determined using the following formula:

[0156]

[0157] in, This represents the weight coefficient for the reference type identifier t.

[0158] Understandably, the hit confidence value for the first reference subtext of the non-alternative reference description structure is 0.

[0159] As an example, a target reference subtext includes two reference types, b and d, with corresponding hit trust values. .

[0160] Understandably, when multiple citation types coexist, the credibility of the existing citation description structure within the sentence is increased, and the reward for its credibility is increased by calculating the product.

[0161] Furthermore, the sum of the hit trust values ​​of each target reference subtext is calculated as the hit index of the alternative reference description structure.

[0162] Understandably, the first reference subtext is the subtext that is confirmed to contain a reference. The more candidate reference description structures are hit in the first reference subtext, and the higher the hit index, the higher the credibility of the candidate reference description structure and the more likely it is to be a reference description structure.

[0163] Step S404: Count the number of candidate reference description structures included in the second reference subtext, and determine the exclusion index of candidate reference description structures based on the number.

[0164] Optionally, determine the exclusion trust value of the second reference subtext; obtain the sum of the exclusion trust values ​​of each second reference subtext as the exclusion index.

[0165] In this embodiment of the application, the exclusion trust value of any reference subtext is set to 1, and the sum of the exclusion trust values ​​of each second reference subtext is obtained, that is, the number of second reference subtexts with alternative reference description structures, which is used as the exclusion index.

[0166] In other embodiments, the exclusion trust value may be set to other values, which are not limited here.

[0167] It is understandable that the second reference subtext does not contain obvious citation markers and is therefore a subtext without citations. If alternative citation description structures are identified in the second reference subtext, the credibility of the alternative citation description structures will be reduced. The more alternative citation description structures are identified in the second reference subtext, the greater the reduction in credibility. Therefore, the exclusion index is determined by counting the number of alternative citation description structures included in the second reference subtext.

[0168] Step S405: Based on the hit index and exclusion index, determine the effective hit index of the candidate reference description structure.

[0169] Optionally, the sum of the exclusion index and the preset value can be obtained as the target exclusion index, and the ratio of the hit index to the target exclusion index can be used as the effective hit index.

[0170] As an example, the effective hit index R can be determined by the following formula:

[0171]

[0172] in, Indicates the hit index. This represents the exclusion index, where 1 is a preset value set in this application embodiment. This indicates the target exclusion index.

[0173] Understandably, by adding the result to a preset value to obtain the target exclusion index, the risk of the denominator being 0 can be avoided.

[0174] The credibility of the alternative citation description index is quantified by calculating the ratio of the hit index to the target exclusion index. The more alternative citation description structures are identified in the first reference subtext containing citation content, the higher the credibility of the alternative citation description structures in identifying citation content. Conversely, the fewer alternative citation description structures are identified in the second reference subtext containing no citation content, the higher the credibility of the alternative citation description structures in identifying citation content.

[0175] Step S406: Filter the candidate reference description structures based on the effective hit index to obtain the reference description structures.

[0176] Optionally, if the effective hit index is greater than or equal to the effective hit threshold, the candidate reference description structure is determined as the reference description structure.

[0177] As an example, the effective hit threshold is set to 5. In other embodiments, the effective hit threshold can be set according to the usage, which is not limited here.

[0178] Understandably, after filtering by the effective hit threshold, each reference description structure has passed statistical verification based on large models and massive amounts of data, greatly improving the credibility of the description structure.

[0179] Optionally, after determining the reference description structure, a reference description set including all reference description structures is constructed.

[0180] Understandably, by constructing a set of citation descriptions and matching any subtext with the citation description structure within that set, citation recognition can be performed on subtexts that are not explicitly labeled, making the application scenarios for citation recognition more comprehensive.

[0181] Step S50: Based on the citation type identifier and position identifier of each sub-text, perform citation annotation in the text to be annotated.

[0182] Optionally, by determining the citation type identifier of each subtext, the citation type of each subtext can be symbolized. After the position identifier of each subtext is determined, the position of each subtext in the text to be annotated can be confirmed. Furthermore, based on the citation type and position information of the subtext, citation annotation can be performed on each subtext.

[0183] As one possible implementation, the target position of the subtext is located based on the location identifier; and the corresponding form of citation annotation is performed at the target position based on the citation type identifier.

[0184] In the embodiments of this application, the citation annotation forms for different citation types are different. For example, the subtext identified by any citation type is highlighted at the target position with a unique corresponding color.

[0185] It should be noted that in other embodiments, the citation annotation format for different citation types can be the same. For example, the identified citations can be displayed in a uniform annotation format at the target location.

[0186] Although the steps in the above embodiments are described in the above order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not need to be executed in such order. They can be executed simultaneously (in parallel) or in reverse order. These simple changes are all within the protection scope of this application.

[0187] This application's embodiments extract original annotation information based on the layout of the text to be annotated, enabling the acquisition of author-annotated citation information within the text. The text to be annotated is segmented into multiple sub-texts, dividing the entire text into smaller sub-texts as the smallest unit for text processing, and determining the positional identifier of any sub-text to locate its position. When a citation marker is identified in any sub-text, its citation type identifier is determined as the first identifier, enabling the determination of the author-annotated citation content and type based on the original annotation information. Any sub-text is matched against a set of citation descriptions, where the set consists of multiple citation description structures. These structures are selected based on the citation type of the reference text and the number of alternative citation description structures. The selected reference texts are texts containing the original annotation information for reference. The candidate citation description structures are obtained by identifying the citation description structures of subtexts from multiple texts to be annotated. By matching any subtext with the citation description structure, it is determined whether a citation type exists. If a match is successful, it indicates that a citation description structure exists within any subtext, and the corresponding citation type is identified as the second identifier. This method can identify missing citations and compensate for the shortcomings of self-annotation. Based on the citation type and position identifiers of each subtext, citation annotation is performed in the text to be annotated. This method can identify and annotate both author-annotated and unannotated citations in the text, clearly obtaining and annotating all citation information in the text, thus improving the efficiency and intelligence of citation annotation in various application scenarios.

[0188] The citation annotation system of the second embodiment of this application, such as... Figure 3 As shown, it includes an extraction module 100, a segmentation module 200, a matching module 300, a generation module 400, and a citation annotation module 500.

[0189] The extraction module 100 is used to extract the original annotation information of the text to be annotated based on the layout of the text to be annotated, wherein the original annotation information includes citation annotations;

[0190] The segmentation module 200 is used to segment the text to be annotated into multiple sub-texts and determine the position identifier of any sub-text;

[0191] The first identifier determination module 300 is used to determine the reference type identifier of any subtext as the first identifier when a reference marker is identified in any subtext;

[0192] The second identifier determination module 400 is used to determine the reference type identifier of any subtext as the second identifier when any subtext successfully matches the reference description set. The reference description set consists of multiple reference description structures. The reference description structure is obtained by filtering from the candidate reference description structures based on the reference type of the reference text and the number of candidate reference description structures. The reference text is the text used for reference, which includes the original annotation information. The candidate reference description structures are obtained by identifying the reference description structure of the subtexts of multiple texts to be annotated.

[0193] The citation annotation module 500 is used to perform citation annotation in the text to be annotated based on the citation type identifier and position identifier of each subtext.

[0194] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0195] It should be noted that the citation annotation system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of this application can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of this application are only for distinguishing the various modules or steps and are not considered as an improper limitation of this application.

[0196] An electronic device according to a third embodiment of this application includes:

[0197] At least one processor; and

[0198] A memory communicatively connected to at least one of the processors; wherein,

[0199] The memory stores instructions that can be executed by the processor to implement the citation annotation method described above.

[0200] A computer-readable storage medium according to a fourth embodiment of this application stores computer instructions that are executed by the computer to implement the above-described citation annotation method.

[0201] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the electronic device and computer-readable storage medium described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0202] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0203] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0204] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0205] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0206] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0207] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A citation tagging method, characterized by, The method comprises the following steps: Step S10, extracting original annotation information of the text to be annotated based on the format of the text to be annotated, wherein the original annotation information comprises a reference annotation symbol; Step S20, dividing the text to be annotated into a plurality of subtexts and determining the position identifier of any subtext; Step S30, in the case that the reference annotation symbol is identified in the any subtext, determining that the reference type identifier of the any subtext is a first identifier; Step S40, in the case that the any subtext matches the reference description set successfully, determining that the reference type identifier of the any subtext is a second identifier, wherein the reference description set is composed of a plurality of reference description structures, the reference description structure is obtained by screening in the alternative reference description structure based on the reference type of the reference text and the number of alternative reference description structures, the reference text is a text including original annotation information for reference, and the alternative reference description structure is obtained by performing reference description structure identification on the subtexts of a plurality of texts to be annotated; Step S50, performing citation annotation in the text to be annotated based on the reference type identifier and the position identifier of each subtext; The reference description structure is obtained by screening in the alternative reference description structure based on the reference type of the reference text and the number of alternative reference description structures, and comprises the following steps: Obtaining a plurality of reference texts and determining the reference reference type identifier of any reference subtext in the reference texts; Dividing the reference subtext into a first reference subtext and a second reference subtext based on the reference reference type identifier; Screening a target reference subtext in which the alternative reference description structure exists in the first reference subtext, and determining a hit index of the alternative reference description structure based on the reference type of the target reference subtext; Counting the number of alternative reference description structures included in the second reference subtext, and determining an exclusion index of the alternative reference description structure based on the number; Determining an effective hit index of the alternative reference description structure based on the hit index and the exclusion index; Screening the alternative reference description structure based on the effective hit index to obtain the reference description structure.

2. The method of claim 1, wherein, The determination of the hit index of the alternative reference description structure based on the reference type of the target reference subtext comprises the following steps: Determining a weight coefficient corresponding to any reference type; Obtaining the product of the weight coefficients of each reference type included in any target reference subtext as the hit trust value of the any target reference subtext; Obtaining the sum of the hit trust values of each target reference subtext as the hit index.

3. The method of claim 1, wherein, The determination of the exclusion index of the alternative reference description structure based on the number comprises the following steps: Determining an exclusion trust value of the second reference subtext; Obtaining the sum of the exclusion trust values of each second reference subtext as the exclusion index.

4. The method of claim 1, wherein, The determination of the effective hit index of the alternative reference description structure based on the hit index and the exclusion index comprises the following steps: Obtaining the sum of the exclusion index and a preset value as a target exclusion index; Take the ratio of the hit index and the target exclusion index as the effective hit index.

5. The method of claim 1, wherein, The screening of the candidate citation description structure based on the effective hit index includes: In the case that the effective hit index is greater than or equal to an effective hit threshold, determining that the candidate citation description structure is the citation description structure.

6. The method of claim 1, wherein, The division of the reference subtext into a first reference subtext and a second reference subtext based on the reference citation type identifier includes: In the case that the reference citation type identifier is not empty, marking the reference subtext as a first reference subtext; In the case that the reference citation type identifier is empty, marking the reference subtext as a second reference subtext.

7. The method of claim 1, wherein, The candidate citation description structure is obtained by citation description structure identification on subtexts of a plurality of texts to be annotated, and includes: Inputting a plurality of de-labeled subtexts into a large model, and based on a prompt word of citation description structure identification, obtaining a citation fragment and an output citation description structure of any de-labeled subtext, wherein the de-labeled subtext is a subtext from which the citation mark is removed; In the case that the position located by the position identifier of the subtext overlaps with the position of the citation fragment, determining that the output citation description structure is a candidate citation description structure.

8. The method of claim 1, wherein, The determination of the position identifier of any subtext includes: Determining the first character position and the tail character position of the any subtext; Taking the first character position and the tail character position as coordinates as the position identifier of the any subtext.

9. The method of claim 1, wherein, The citation annotation in the text to be annotated based on the citation type identifier and the position identifier of each subtext includes: Positioning to the target position of the subtext based on the position identifier; Performing citation annotation of a corresponding form at the target position based on the citation type identifier.

Images