Document annotation method and device, equipment, medium and product

Abstract

The invention discloses a document annotation method, device and equipment, a medium and a product. The method comprises the steps of obtaining a target document and segmenting the target document to obtain at least one document fragment matched with the target document; inputting the document fragments into a pre-trained document annotation model to obtain fragment annotation results respectively matched with the document fragments; and based on a preset post-processing rule and the annotation result of each fragment, performing annotation post-processing operation on each document fragment to obtain a document annotation result matched with the target document. By means of the technical scheme, annotation generation for the target document can be achieved, the document annotation work efficiency and annotation flexibility are improved, and meanwhile the annotation result accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a document annotation method, device, equipment, medium and product. Background Art

[0002] With the rapid development of artificial intelligence (AI), text classification technology has been widely used in fields such as financial analysis and news classification. In vertical fields, the need for content annotation of professional documents is becoming increasingly prominent. The core of this is to annotate each sentence in the document using text classification technology. However, this type of annotation task differs significantly from general text classification: on the one hand, it requires higher professionalism and requires precise annotation in combination with domain knowledge; on the other hand, the document context has strong semantic associations, and it is difficult to determine the category based on a single sentence. This is especially true in multi-label and multi-level label classification scenarios, where the complexity of the task increases exponentially.

[0003] Currently, complex text annotation mainly relies on manual annotation. This traditional method has significant drawbacks: annotating a single document requires a lot of manpower, is time-consuming, and inefficient. In addition, manual operations are easily affected by subjective factors, resulting in fluctuations in the accuracy of the annotation results. It is difficult to strike a balance between efficiency and precision in large-scale complex text processing. If the existing model is used, there are still two problems: First, the granularity of text segmentation is fixed and cannot adapt to the dynamic segmentation requirements of complex documents. Taking punctuation segmentation as an example, splitting by "."?!" is likely to cause semantically related sentences to be separated, while fine-grained splitting by ",,:" will produce a large number of fragmented short sentences. Both methods cannot achieve reasonable semantic unit division. Second, some sentences rely on contextual semantics to be accurately classified. When the existing model processes a single sentence independently, it cannot accurately identify the category of such sentences due to the lack of contextual semantic associations.

[0004] In summary, the inefficiency of traditional manual annotation and the limitations of existing technical models have led to the problems of poor efficiency and flexibility in existing document annotation work, as well as low accuracy of annotation results. Summary of the Invention

[0005] The present invention provides a document annotation method, device, equipment, medium and product, which can solve the problems of poor efficiency and flexibility in existing document annotation work and low accuracy of annotation results.

[0006] In a first aspect, an embodiment of the present invention provides a document annotation method, the method comprising:

[0007] Acquire a target document and segment the target document to obtain at least one document segment matching the target document;

[0008] Input each document fragment into a pre-trained document annotation model to obtain a fragment annotation result that matches each document fragment, wherein the fragment annotation result includes at least one of a first-class label, a second-class label, a third-class label, and a conventional label;

[0009] Based on the preset post-processing rules and the annotation results of each fragment, an annotation post-processing operation is performed on each document fragment to obtain a document annotation result that matches the target document.

[0010] In a second aspect, an embodiment of the present invention provides a document annotation device, the device comprising:

[0011] A document segmentation module, configured to acquire a target document and segment the target document to obtain at least one document segment matching the target document;

[0012] a model processing module, configured to input each document segment into a pre-trained document annotation model to obtain segment annotation results that match each document segment, wherein the segment annotation results include at least one of a first-class label, a second-class label, a third-class label, and a conventional label;

[0013] The post-processing module is used to perform annotation post-processing operations on each document segment based on preset post-processing rules and the annotation results of each segment, so as to obtain a document annotation result that matches the target document.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the document annotation method described in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a document annotation method described in any embodiment of the present invention when executed.

[0019] In a fifth aspect, an embodiment of the present invention provides a computer program product, which is used to enable a processor to implement a document annotation method described in any embodiment of the present invention when the computer program is executed.

[0020] The technical solution of the embodiment of the present invention obtains a target document and divides the target document to obtain at least one document fragment matching the target document, then inputs each document fragment into a pre-trained document annotation model to obtain a fragment annotation result matching each document fragment, and finally performs annotation post-processing operations on each document fragment based on preset post-processing rules and each fragment annotation result to obtain a document annotation result matching the target document, which solves the problems of poor efficiency and annotation flexibility in existing document annotation work and low accuracy of annotation results, realizes annotation generation for target documents, improves the efficiency and annotation flexibility of document annotation work, and at the same time improves the accuracy of annotation results.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flowchart of a document annotation method provided according to embodiment 1 of the present invention;

[0024] Figure 2 is a flowchart of a document annotation method provided according to the second embodiment of the present invention;

[0025] Figure 3 2 is a schematic structural diagram of a document annotation device provided according to a third embodiment of the present invention;

[0026] Figure 4 The present invention is a schematic structural diagram of an electronic device for implementing a document annotation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variations of the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a document annotation method provided in the first embodiment of the present invention. This embodiment is applicable to the situation of annotating documents. The method can be executed by a document annotation device. The document annotation device can be implemented in the form of hardware and / or software. The document annotation device can be configured in a terminal acquisition server with a document annotation function.

[0031] like Figure 1 As shown, the method includes:

[0032] S110: Acquire a target document and segment the target document to obtain at least one document segment matching the target document.

[0033] Among them, the target document refers to a complex text to be annotated, such as a document containing financial reports, business specifications, contract terms, etc. Such documents often have close contextual semantic associations and require multi-label or multi-level label annotation. Furthermore, the text segmentation refers to breaking down the complete document into text paragraphs of different granularities according to specific rules for subsequent annotation and analysis. Specifically, the segmentation rules may include coarse-grained segmentation, that is, initially splitting the text into paragraphs based on sentences according to punctuation marks such as periods, exclamation marks, and question marks, and then segmenting them into independent sentences. Furthermore, the segmentation rules may include fine-grained segmentation, that is, further splitting long sentences into smaller segments according to semantic requirements. It should be noted that the specific rules for the text segmentation can be defined by the user according to the user's actual usage scenario, and this embodiment does not impose any restrictions on this. Finally, all segmented sentences are deduplicated to avoid duplicate segments affecting the efficiency of subsequent annotation. Through the above segmentation process, multiple document segments that match the semantics of the original text can be obtained from the target document, providing a basis for subsequent data annotation and model processing.

[0034] S120: Input each document segment into a pre-trained document annotation model to obtain segment annotation results that match each document segment.

[0035] The fragment annotation result includes at least one of: a first-class label, a second-class label, a third-class label and a regular label.

[0036] Among them, the conventional tags refer to basic tags that directly correspond to business categories, such as "risk warning", "business process", "compliance requirements" in financial documents, etc., which are used to annotate the core semantic categories of the fragments; further, the first-class tags, second-class tags, and third-class tags are auxiliary annotation categories.

[0037] Specifically, the first type of tags are category split tags, used to mark sentences that need to be split according to different semantic categories and combined with other segments for annotation. For example, in a financial business specification, "Customers must provide the original and a copy of their ID card and proof of income, with the copy requiring the official seal." This document fragment contains more than one semantic category and should be annotated as "category split." Furthermore, the second type of tags are content split tags, used to mark long sentences that require further fine-grained splitting. For example, in a financial report, the sentence "This product supports automatic conversion of current deposits to fixed deposits, automatic conversion upon maturity of fixed deposits, and flexible interest adjustment for early withdrawals" can be split into three sections: "automatic conversion of current deposits to fixed deposits," "automatic conversion upon maturity of fixed deposits," and "flexible interest adjustment for early withdrawals." The original sentence should be annotated as "content split." Furthermore, the third type of tags are unknown merge tags, used to mark semantically incomplete segments that need to be merged with the context to determine the category. For example, in a financial contract, "If a breach of contract occurs, the relevant liability shall be assumed in accordance with the terms of the agreement." If the preceding text does not clearly define the breaching entity or the type of liability, the sentence should be annotated as "unknown merge."

[0038] It should be noted that in this embodiment, the fragment annotation result for a document fragment can only contain one or none of the first-category tags, second-category tags, or third-category tags. At the same time, if the fragment annotation result contains a second-category tag, the fragment annotation result only contains the second-category tag and does not contain any regular tags. For example, based on the above steps, if there is a document fragment that is "This product supports the automatic conversion of current deposits to fixed deposits, automatic conversion of fixed deposits upon maturity, and flexible adjustment of interest calculation for early withdrawals", then the fragment annotation result matching the document fragment can be "content separation", that is, the fragment annotation result consists of a second-category tag. At the same time, if there is a document fragment that is "The borrower must repay the principal and interest on time. If overdue, corresponding fees will be charged", then the fragment annotation result matching the document fragment can be "category separation, repayment obligation-repayment on time, breach of contract liability-overdue fees", consisting of a first-category tag and two regular tags.

[0039] Optionally, the training process of the document annotation model includes: obtaining a sample data set, the sample data including at least one sample segment and sample annotation results respectively matching each sample segment; using the sample data set to train a preset large language model to obtain the document annotation model, wherein the sample annotation results are used as annotation data in the sample data.

[0040] The sample data may be complex text including financial reports, business agreements, risk assessment documents, etc.; further, the sample annotation results include sample annotation results that match the sample data, and the sample annotation results include at least one of a first-class label, a second-class label, a third-class label, and a regular label.

[0041] Furthermore, the large language model can be a generalized autoregressive pre-trained model or a bidirectional encoder representation model. The specific process is as follows: sample segments are input into the model, semantic features are extracted through the model, and the model output layer generates a predicted probability distribution for conventional labels and auxiliary labels based on the feature vector. During training, the annotated sample annotation results are used as the supervision signal. The cross-entropy loss function is used to calculate the difference between the predicted value and the true value, and the backpropagation algorithm is used to update the model parameters until the loss converges.

[0042] S130 : Based on preset post-processing rules and the annotation results of each fragment, perform annotation post-processing operations on each document fragment to obtain a document annotation result that matches the target document.

[0043] The technical solution of the embodiment of the present invention obtains a target document and divides the target document to obtain at least one document fragment matching the target document, then inputs each document fragment into a pre-trained document annotation model to obtain a fragment annotation result matching each document fragment, and finally performs annotation post-processing operations on each document fragment based on preset post-processing rules and each fragment annotation result to obtain a document annotation result matching the target document, thereby realizing annotation generation for the target document, improving the efficiency and flexibility of document annotation work, and at the same time improving the accuracy of the annotation results.

[0044] Example 2

[0045] Figure 2 A flowchart of a document annotation method provided in the second embodiment of the present invention is provided. This embodiment is refined based on the above embodiment. In this embodiment, the method of performing post-processing operations on each document fragment based on preset post-processing rules and the annotation results of each fragment to obtain a document annotation result that matches the target document is specifically refined.

[0046] like Figure 2 As shown, the method includes:

[0047] S210: Acquire a target document and segment the target document to obtain at least one document segment matching the target document.

[0048] S220: Input each document segment into a pre-trained document annotation model to obtain segment annotation results that match each document segment.

[0049] The fragment annotation result includes at least one of: a first-class label, a second-class label, a third-class label, and a regular label.

[0050] S230 , in each document segment, the document segments whose segment annotation results do not include the first-category label, the second-category label, and the third-category label are filtered and extracted as regular document segments.

[0051] S240 , performing a similar merging operation on each regular document segment using a word frequency-inverse document frequency text similarity algorithm to obtain a regular annotation result that matches the target document.

[0052] Among them, the term frequency-inverse document frequency algorithm (TF-IDF) is an algorithm used to measure the importance of words in a document. It determines the word's ability to represent the document topic by calculating the word's frequency of occurrence (TF) in a single document fragment and the inverse document frequency (IDF) of the word in the entire target document, thereby determining the semantic similarity between document fragments.

[0053] For example, there are document fragments such as "body temperature 39.4 degrees, high fever does not subside" (annotation "status - sick") and "persistent abdominal pain, awake state" (annotation "status - sick"), although the specific symptom descriptions are different, but both contain high-frequency words related to "status" and belong to the same "sick" category. The algorithm will merge them into a comprehensive annotation of "status - sick"; in the financial scenario, "customers need to provide a copy of the business license" (annotation "business process - information submission") and "financial statements for the past three years need to be submitted" (annotation "business process - information submission") share the document annotation of business process - information submission, so even if the specific information types are different, they will be merged into a unified annotation of "business process - information submission". This process can eliminate duplicate annotation results and integrate scattered semantic units through similar induction at the semantic level to form a regular annotation result covering the core content of the target document.

[0054] S250 , in each document segment, filter the segment annotation results to extract each document segment containing one-category labels, two-category labels, or three-category labels as an unconventional segment.

[0055] S260 , based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule to obtain an unconventional annotation result matching each unconventional fragment.

[0056] Specifically, based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, including: if the fragment annotation result of the unconventional fragment includes a type of label, then based on each conventional label in the fragment annotation result, the unconventional fragment is annotated multiple times to obtain an unconventional annotation result matching the unconventional fragment.

[0057] For example, if the fragment "The borrower is required to repay the principal and interest on time, and if overdue, corresponding fees will be charged" in the financial contract is annotated as "Category separation, repayment obligation-repayment on time, liability for breach of contract-overdue fees", then the unconventional fragment is annotated multiple times according to the rules, and the final unconventional annotation results are "The borrower is required to repay the principal and interest on time, and if overdue, corresponding fees will be charged, repayment obligation-repayment on time" and "The borrower is required to repay the principal and interest on time, and if overdue, corresponding fees will be charged, liability for breach of contract-overdue fees".

[0058] Optionally, based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, and it also includes: if the fragment annotation results of the unconventional fragment include two types of labels, then the unconventional fragment is split to obtain each split fragment matching the unconventional fragment; each split fragment is input into a pre-trained label annotation model to obtain at least one target label matching each split fragment; each split fragment and the target label matching each split fragment are aggregated, and the aggregation result is used as the unconventional annotation result matching the unconventional fragment.

[0059] For example, in a financial report, the segment annotated as "Content Split" reads, "This product supports automatic conversion of demand deposits to fixed deposits, automatic conversion upon maturity of fixed deposits, and flexible interest adjustments for early withdrawals." The segment is then split, yielding three matching segments: "This product supports automatic conversion of demand deposits to fixed deposits," "Automatic conversion upon maturity of fixed deposits," and "Flexible interest adjustments for early withdrawals." Each segment is then fed into a pre-trained label annotation model to obtain at least one target label matching each segment: "This product supports automatic conversion of demand deposits to fixed deposits, product - automatic conversion to fixed deposits," "Automatic conversion upon maturity of fixed deposits, product - automatic conversion," and "Flexible interest adjustments for early withdrawals, product - interest adjustment." The label annotation model is an artificial intelligence model trained on a large amount of annotated text data, used to identify text semantics and match corresponding label categories. Finally, each segment and its matching target label are aggregated, and the aggregated result is used as the matching unconventional annotation result for the segment.

[0060] Optionally, based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, and it also includes: if the fragment annotation results of the unconventional fragment include three types of labels, then searching in each document fragment whether there are sentences that meet the preset query criteria as the three types of sentences matching the unconventional fragment; if so, the conventional labels corresponding to the three types of sentences are used as the unconventional annotation results matching the unconventional fragment; if not, the three types of labels in the fragment annotation results corresponding to the unconventional fragment are deleted, and the updated fragment annotation results are used as the unconventional annotation results matching the unconventional fragment.

[0061] For example, the document fragment of a financial product is "This product supports regular transfer and automatic renewal functions, see the attachment for specific rules" and the annotation is "Unknown merger, product function-transfer, product function-renewal". The system will search other document fragments of the financial product to see whether there are three types of statements directly related to "transfer" and "renewal" (such as "Fixed deposits are automatically transferred to deposits of the same term after maturity, product function-transfer"). If matching three types of statements are found, the regular label "product function-transfer" is extracted as the annotation result of the unconventional fragment; if not found, the three types of labels are deleted, and only "product function-transfer, product function-renewal" are retained as simplified annotations to ensure the integrity of the annotation logic.

[0062] S270: Generate a document annotation result that matches the target document according to the conventional annotation result and the unconventional annotation result.

[0063] Furthermore, a document annotation result matching the target document is generated based on the conventional annotation result and the unconventional annotation result, including: based on the fragment annotation results of each conventional document fragment and the unconventional annotation results of each unconventional fragment, each conventional document fragment and each unconventional fragment are subjected to a similar merging operation through a word frequency-inverse document frequency text similarity algorithm to obtain a document annotation result matching the target document.

[0064] In this embodiment, when the document annotation results are finally generated, the regular annotation results and the unconventional annotation results need to be secondary aggregated at the semantic level. Using the term frequency-inverse document frequency (TF-IDF) algorithm, the annotation results and text content of all regular document segments and processed unconventional segments are similarly calculated. For example, in a medical scenario, the regular segment "Body temperature returns to normal" (labeled "status-healthy") and the unconventional processed segment "Abdominal pain symptoms relieved" (labeled "status-healthy") share high-frequency words such as "status" and "recovery", so the algorithm will merge them into a comprehensive annotation of "status-healthy". In a financial scenario, the unconventional segment "Copy requires official seal" (annotated "Document submission-stamping requirements" after three-category labeling) and the regular segment "Document copy requires signature" (labeled "Document submission-signature requirements") belong to the same "Document submission" theme and will be aggregated into a unified annotation of "Document submission-formal requirements", thus forming a complete annotation result covering the full semantic dimension of the target document.

[0065] The technical solution of the embodiment of the present invention obtains a target document and divides the target document to obtain at least one document fragment matching the target document, then inputs each document fragment into a pre-trained document annotation model to obtain a fragment annotation result matching each document fragment, and in each document fragment, screens out each document fragment whose fragment annotation result does not contain a first-class label, a second-class label, and a third-class label as a regular document fragment and extracts it, then based on the fragment annotation result of each regular document fragment, performs a similar merging operation on each regular document fragment through a word frequency-inverse document frequency text similarity algorithm to obtain a regular annotation result matching the target document , then, in each document fragment, the document fragments whose fragment annotation results contain one-category labels, two-category labels or three-category labels are filtered and extracted as unconventional fragments, and the fragment processing rules matching each unconventional fragment are obtained based on the fragment annotation results of each unconventional fragment, and each unconventional fragment is processed separately based on each fragment processing rule to obtain unconventional annotation results matching each unconventional fragment, and finally, a document annotation result matching the target document is generated according to the conventional annotation results and the unconventional annotation results, thereby realizing the annotation generation for the target document, improving the efficiency and flexibility of the document annotation work, and at the same time improving the accuracy of the annotation results.

[0066] Example 3

[0067] Figure 3 This is a schematic diagram of the structure of a document annotation device provided by the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0068] A document segmentation module 310 is configured to obtain a target document and segment the target document to obtain at least one document segment matching the target document;

[0069] The model processing module 320 is configured to input each document segment into a pre-trained document annotation model to obtain segment annotation results that match each document segment, wherein the segment annotation results include at least one of a first-class label, a second-class label, a third-class label, and a conventional label.

[0070] The post-processing module 330 is configured to perform annotation post-processing operations on each document segment based on preset post-processing rules and the annotation results of each segment, to obtain a document annotation result that matches the target document.

[0071] The technical solution of the embodiment of the present invention obtains a target document and divides the target document to obtain at least one document fragment matching the target document, then inputs each document fragment into a pre-trained document annotation model to obtain a fragment annotation result matching each document fragment, and finally performs annotation post-processing operations on each document fragment based on preset post-processing rules and each fragment annotation result to obtain a document annotation result matching the target document, thereby realizing annotation generation for the target document, improving the efficiency and flexibility of document annotation work, and at the same time improving the accuracy of the annotation results.

[0072] Based on the above embodiment, the post-processing module 330 includes:

[0073] A screening unit is used to screen out, from among the document segments, the document segments whose segment annotation results do not contain the first-category label, the second-category label, and the third-category label and extract them as regular document segments;

[0074] A first merging unit is configured to perform a similar merging operation on each regular document fragment based on the fragment annotation results of each regular document fragment by using a word frequency-inverse document frequency text similarity algorithm to obtain a regular annotation result that matches the target document;

[0075] An unconventional segment extraction unit is used to select, from among the document segments, document segments whose segment annotation results contain one-category labels, two-category labels, or three-category labels and extract them as unconventional segments;

[0076] An unconventional annotation unit, configured to obtain a segment processing rule matching each unconventional segment based on the segment annotation result of each unconventional segment, and process each unconventional segment based on each segment processing rule to obtain an unconventional annotation result matching each unconventional segment;

[0077] The document annotation result generating unit is configured to generate a document annotation result matching the target document according to the conventional annotation result and the unconventional annotation result.

[0078] Based on the above embodiment, the unconventional annotation unit is specifically used to: if the fragment annotation result of the unconventional fragment includes a type of label, then the unconventional fragment is annotated multiple times based on the various conventional labels in the fragment annotation result to obtain an unconventional annotation result that matches the unconventional fragment.

[0079] Based on the above embodiment, the unconventional annotation unit further includes:

[0080] a second-category processing unit configured to, if the segment annotation result of the unconventional segment includes the second-category label, split the unconventional segment to obtain split segments matching the unconventional segment;

[0081] A model annotation unit, configured to input each split segment into a pre-trained label annotation model to obtain at least one target label that matches each split segment;

[0082] The aggregation unit is used to aggregate the split segments and the target tags respectively matched with the split segments, and use the aggregation result as the unconventional annotation result matched with the unconventional segment.

[0083] Based on the above embodiment, the unconventional annotation unit further includes:

[0084] a three-category processing unit configured to search, in each document segment, whether there is a sentence that meets a preset query criterion as a three-category sentence matching the unconventional segment if the segment annotation result of the unconventional segment includes three-category tags;

[0085] an annotation definition unit, configured to, if present, use the conventional tags corresponding to the three types of sentences as unconventional annotation results that match the unconventional fragments;

[0086] The annotation deletion unit is used to delete the three types of labels in the segment annotation result corresponding to the unconventional segment if they do not exist, and use the updated segment annotation result as the unconventional annotation result matching the unconventional segment.

[0087] On the basis of the above embodiment, the document annotation result generation unit is specifically used to: based on the fragment annotation results of each regular document fragment and the unconventional annotation results of each unconventional fragment, perform similar merging operations on each regular document fragment and each unconventional fragment through the word frequency-inverse document frequency text similarity algorithm to obtain a document annotation result that matches the target document.

[0088] Based on the above embodiment, the document annotation device includes:

[0089] The model training module is used to obtain a sample data set, wherein the sample data includes at least one sample segment and sample annotation results respectively matched with each sample segment; use the sample data set to train a preset large language model to obtain the document annotation model, wherein the sample annotation results are used as annotation data in the sample data.

[0090] A document annotation device provided by an embodiment of the present invention can execute a document annotation method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method.

[0091] Example 4

[0092] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0093] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12 and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0094] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0095] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a document annotation method.

[0096] Accordingly, the method includes:

[0097] Acquire a target document and segment the target document to obtain at least one document segment matching the target document;

[0098] Input each document fragment into a pre-trained document annotation model to obtain a fragment annotation result that matches each document fragment, wherein the fragment annotation result includes at least one of a first-class label, a second-class label, a third-class label, and a conventional label;

[0099] Based on the preset post-processing rules and the annotation results of each fragment, an annotation post-processing operation is performed on each document fragment to obtain a document annotation result that matches the target document.

[0100] In some embodiments, a method for annotating a document may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for annotating a document described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a method for annotating a document in any other appropriate manner (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0105] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0106] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0107] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

Claims

1. A document annotation method, characterized in that: include: Acquire a target document and segment the target document to obtain at least one document segment matching the target document; Input each document fragment into a pre-trained document annotation model to obtain a fragment annotation result that matches each document fragment, wherein the fragment annotation result includes at least one of a first-class label, a second-class label, a third-class label, and a conventional label; Based on the preset post-processing rules and the annotation results of each fragment, an annotation post-processing operation is performed on each document fragment to obtain a document annotation result that matches the target document.

2. The method according to claim 1, characterized in that Based on the preset post-processing rules and the annotation results of each fragment, each document fragment is annotated and processed to obtain a document annotation result that matches the target document, including: In each document fragment, the document fragments whose fragment annotation results do not contain the first-category label, the second-category label, and the third-category label are extracted as regular document fragments; Based on the fragment annotation results of each regular document fragment, performing a similar merging operation on each regular document fragment using a word frequency-inverse document frequency text similarity algorithm to obtain a regular annotation result that matches the target document; In each document segment, the document segments whose segment annotation results contain one-category labels, two-category labels, or three-category labels are extracted as unconventional segments; Based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule to obtain an unconventional annotation result matching each unconventional fragment; A document annotation result matching the target document is generated according to the conventional annotation result and the unconventional annotation result.

3. The method according to claim 2, characterized in that Based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, including: If the segment annotation result of the unconventional segment includes a type of label, the unconventional segment is annotated multiple times based on each conventional label in the segment annotation result to obtain an unconventional annotation result matching the unconventional segment.

4. The method according to claim 2, characterized in that Based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, further comprising: If the segment annotation result of the unconventional segment includes two types of labels, performing a split operation on the unconventional segment to obtain split segments matching the unconventional segment; Input each split segment into a pre-trained label annotation model to obtain at least one target label that matches each split segment; The split segments and the target tags respectively matched with the split segments are aggregated, and the aggregation result is used as the unconventional annotation result matched with the unconventional segment.

5. The method according to claim 2, characterized in that Based on the fragment annotation results of each unconventional fragment, a fragment processing rule matching each unconventional fragment is obtained, and each unconventional fragment is processed separately based on each fragment processing rule, further comprising: If the segment annotation result of the unconventional segment includes three types of tags, searching in each document segment whether there are sentences that meet the preset query criteria as the three types of sentences matching the unconventional segment; If so, the conventional tags corresponding to the three types of sentences are used as the unconventional annotation results that match the unconventional fragments; If not, the three types of labels in the segment annotation result corresponding to the unconventional segment are deleted, and the updated segment annotation result is used as the unconventional annotation result matching the unconventional segment.

6. The method according to claim 2, characterized in that Generating a document annotation result matching the target document according to the conventional annotation result and the unconventional annotation result includes: Based on the fragment annotation results of each regular document fragment and the unconventional annotation results of each unconventional fragment, each regular document fragment and each unconventional fragment are subjected to similar merging operations through the word frequency-inverse document frequency text similarity algorithm to obtain a document annotation result that matches the target document.

7. The method according to claim 1, characterized in that The training process of the document annotation model includes: Acquire a sample data set, wherein the sample data includes at least one sample segment and sample annotation results respectively matched with each sample segment; The sample data set is used to train a preset large language model to obtain the document annotation model, wherein the sample annotation result is used as annotation data in the sample data.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute a document annotation method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a document annotation method according to any one of claims 1 to 7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements a document annotation method according to any one of claims 1 to 7.

Images