Key information extraction methods, devices, equipment and storage media

Abstract

This application discloses a method, apparatus, device, and storage medium for extracting key information, relating to the field of information extraction technology. The method includes: performing text detection on a dataset of images to be extracted, obtaining text detection results; performing text recognition on the dataset of images to be extracted based on the text detection results, obtaining text recognition results; performing pattern alignment between the text recognition results and the manually labeled results corresponding to the dataset of images to be extracted, obtaining pattern alignment metadata; and obtaining key information extraction results corresponding to the dataset of images to be extracted based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model. By applying the above technical solution, the technical problem of low accuracy in key information extraction during OCR key information extraction is solved, due to the low accuracy of information annotation through manual annotation and / or detection model annotation methods.

Description

Technical Field

[0001] This application relates to the field of information extraction technology, and in particular to methods, apparatus, equipment and storage media for extracting key information. Background Technology

[0002] OCR (Optical Character Recognition) refers to the process of detecting and recognizing the shape, color, and other information of printed or handwritten characters, and then translating this information into computer text using character recognition methods. OCR has numerous industrial applications, evolving from simple detection and recognition of text information in the past to the current need to extract and integrate key information from numerous recognized characters—a process known as Key Information Extraction (KIE) in OCR. Examples include extracting key information such as names and addresses from ID cards and converting them into computer text.

[0003] In practical applications, the OCR key information extraction process typically includes three stages: annotation, modeling, and prediction. In the annotation stage, users can select the key information to be extracted and provide the relationships between the information. This annotation information is then input into the modeling process. The modeling stage trains two models: one for classifying the recognized text information, and the other for matching relationships between text information with category labels. Finally, the prediction process yields the information extraction result. Currently, information annotation can usually be done manually or using detection models. However, both manual and detection model annotation methods have certain drawbacks, leading to low accuracy and ultimately affecting the accuracy of key information extraction. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and storage medium for extracting key information, aiming to solve the technical problem in the prior art where the accuracy of key information extraction is low due to the low precision of information annotation through manual annotation and / or detection model annotation.

[0005] To achieve the above objectives, this application proposes a method for extracting key information, the method comprising:

[0006] Perform text detection on the image dataset to be extracted and obtain the text detection results;

[0007] Based on the text detection results, text recognition is performed on the image dataset to be extracted to obtain text recognition results;

[0008] The text recognition results and the manually labeled results corresponding to the image dataset to be extracted are pattern aligned to obtain pattern alignment metadata.

[0009] Based on the pattern-aligned metadata, the preset text information classification model, and the preset text information relationship matching model, the key information extraction results corresponding to the image dataset to be extracted are obtained.

[0010] In one embodiment, the step of performing pattern alignment between the text recognition result and the manually labeled result corresponding to the image dataset to be extracted, and obtaining pattern alignment metadata, includes:

[0011] The first metadata is determined based on the text recognition result and the manual annotation result corresponding to the image dataset to be extracted. The first metadata is the text content in the text recognition result and the manual annotation result where the text content is the same and the annotation box overlap is greater than a preset overlap threshold.

[0012] Determine the second metadata in the text recognition result, wherein the second metadata is the text content that exists in each image sample of the text recognition result and appears a preset number of times;

[0013] Third-party metadata is obtained based on the differences between the text recognition results and the manually labeled results;

[0014] The first metadata, the second metadata, and the third metadata are merged to obtain pattern-aligned metadata.

[0015] In one embodiment, the step of determining the first metadata based on the text recognition result and the manually labeled result corresponding to the image dataset to be extracted includes:

[0016] Obtain the first bounding box from the text recognition result and the second bounding box from the manual annotation result corresponding to the image dataset to be extracted;

[0017] Identify a first target annotation box and a second target annotation box that contain the same text content as the first annotation box and the second annotation box;

[0018] Determine the area of ​​overlap and the area of ​​union of the regions between the first target bounding box and the second target bounding box;

[0019] The overlap ratio between the first target bounding box and the second target bounding box is determined based on the overlapping area of ​​the regions and the union area of ​​the regions.

[0020] If the overlap of the annotation boxes is greater than a preset overlap threshold, then the text content in the first target annotation box and the second target annotation box is recorded as first metadata.

[0021] In one embodiment, the step of obtaining third-party metadata based on the difference between the text recognition result and the manually labeled result includes:

[0022] The method for recording differences is determined based on the differences between the text recognition results and the manual annotation results.

[0023] The difference content is recorded using the difference recording method to obtain difference metadata records;

[0024] The differential metadata records are deduplicated using a preset deduplication rule to obtain third metadata.

[0025] In one embodiment, the step of determining the difference recording method based on the difference between the text recognition result and the manual annotation result includes:

[0026] Based on the text recognition results, the detection boxes in the image dataset to be extracted are determined;

[0027] Based on the detection box and the annotation box corresponding to the manual annotation result, determine the difference type corresponding to the difference content between the text recognition result and the manual annotation result;

[0028] The method of recording differences is determined based on the type of difference.

[0029] In one embodiment, the step of obtaining the key information extraction results corresponding to the image dataset to be extracted based on the pattern alignment metadata, the preset text information classification model, and the preset text information relationship matching model includes:

[0030] Based on the pattern alignment metadata, the differences in the text recognition results are split to obtain splitting correction records;

[0031] The splitting and correcting records are input into a preset text information classification model to obtain information classification prediction results;

[0032] The problem type text of the information classification prediction result is corrected by aligning the metadata with the pattern, and a problem correction record is obtained.

[0033] The problem correction record is input into a preset text information relationship matching model to obtain the information relationship matching prediction result;

[0034] The answer type text of the information relationship matching prediction result is corrected by using the pattern alignment metadata to obtain the key information extraction result corresponding to the image dataset to be extracted.

[0035] In one embodiment, before the step of performing text detection on the image dataset to be extracted and obtaining the text detection results, the method further includes:

[0036] Determine the coordinates, categories, and link information of the bounding boxes in the image dataset to be extracted;

[0037] Several annotation point records are generated based on the image identifiers, the coordinate positions of the annotation boxes, the categories of the annotation boxes, and the link information of the annotation boxes in the image dataset to be extracted;

[0038] The image dataset to be extracted is annotated based on each annotation point record to obtain the manual annotation results corresponding to the image dataset to be extracted.

[0039] Furthermore, to achieve the above objectives, this application also proposes a key information extraction device, the device comprising:

[0040] The text detection module is used to perform text detection on the image dataset to be extracted and obtain text detection results;

[0041] The text recognition module is used to perform text recognition on the image dataset to be extracted based on the text detection results, and obtain text recognition results;

[0042] The pattern alignment module is used to perform pattern alignment between the text recognition results and the manually labeled results corresponding to the image dataset to be extracted, and to obtain pattern alignment metadata.

[0043] The information extraction module is used to obtain the key information extraction results corresponding to the image dataset to be extracted based on the pattern-aligned metadata, the preset text information classification model, and the preset text information relationship matching model.

[0044] In addition, to achieve the above objectives, this application also proposes a key information extraction device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the key information extraction method described above.

[0045] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the key information extraction method described above.

[0046] This application provides a method for extracting key information. The method involves: performing text detection on a dataset of images to be extracted to obtain text detection results; performing text recognition on the dataset based on the text detection results to obtain text recognition results; aligning the text recognition results with the manually labeled results corresponding to the dataset to be extracted to obtain pattern alignment metadata; and obtaining key information extraction results corresponding to the dataset based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model. Compared to existing technologies where information annotation through manual annotation and / or detection model annotation is not very accurate, resulting in low accuracy of key information extraction, this invention can align the text recognition results and manually labeled results of the dataset to be extracted to obtain pattern alignment metadata, and obtain key information extraction results based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model. This allows for the combination of manually labeled results and model recognition results, and key information extraction through the model, thereby solving the technical problem in existing technologies where the accuracy of key information extraction is low due to the low precision of information annotation through manual annotation and / or detection model annotation. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating an embodiment of the key information extraction method of this application.

[0050] Figure 2 This is a schematic diagram of the overall process of OCR key information extraction in the key information extraction method of this application;

[0051] Figure 3 This is a flowchart illustrating the process of adding post-processing OCR key information extraction to the key information extraction method of this application;

[0052] Figure 4 This is an example image of character detection in the key information extraction method of this application;

[0053] Figure 5This diagram illustrates the stages of OCR key information extraction in the key information extraction method of this application.

[0054] Figure 6 This is an example image with manual annotation in the key information extraction method of this application;

[0055] Figure 7 This is an example image used for model annotation of an image in the key information extraction method of this application;

[0056] Figure 8 This is a flowchart illustrating the overall annotation and training process of the key information extraction method in this application;

[0057] Figure 9 This is a flowchart illustrating Embodiment 2 of the key information extraction method of this application;

[0058] Figure 10 This is a diagram illustrating the differences in the key information extraction method used in this application;

[0059] Figure 11 This is a flowchart illustrating Embodiment 3 of the key information extraction method of this application;

[0060] Figure 12 This is a flowchart illustrating the overall prediction process of the key information extraction method in this application.

[0061] Figure 13 This is a schematic diagram of the module structure of the key information extraction device in an embodiment of this application;

[0062] Figure 14 This is a schematic diagram of the device structure of the hardware operating environment involved in the key information extraction method in the embodiments of this application.

[0063] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0064] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0065] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0066] The main solution of this application embodiment is as follows: perform text detection on the image dataset to be extracted to obtain text detection results; perform text recognition on the image dataset to be extracted based on the text detection results to obtain text recognition results; perform pattern alignment between the text recognition results and the manually labeled results corresponding to the image dataset to be extracted to obtain pattern alignment metadata; and obtain the key information extraction results corresponding to the image dataset to be extracted based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model.

[0067] In existing technologies, key information extraction in OCR typically involves manual annotation or detection model annotation. However, if manual annotation is used, the information obtained during text detection during prediction may not be combined in a way that humans understand, which can lead to differences between the text combination used in modeling and the text combination used in prediction, thus affecting the model's prediction performance. If detection model annotation is used, the accuracy of the annotation is low because the detection model cannot recognize human intent.

[0068] This application provides a solution that enables pattern alignment between text recognition results and manual annotation results in an image dataset to be extracted, obtaining pattern alignment metadata. Based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model, key information extraction results are obtained. This allows for the combination of manual annotation results and model recognition results, and key information extraction through the model. This solves the technical problem in existing technologies where the accuracy of key information extraction is low due to the low precision of information annotation methods using manual annotation and / or detection model annotation methods.

[0069] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or key information extraction device capable of performing the above functions. The following description uses a key information extraction device (hereinafter referred to as the device) as an example to illustrate this embodiment and the subsequent embodiments.

[0070] Based on this, embodiments of this application provide a method for extracting key information, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the key information extraction method of this application.

[0071] In this embodiment, the key information extraction method includes steps S10 to S40:

[0072] Step S10: Perform text detection on the image dataset to be extracted and obtain the text detection results.

[0073] In practical applications, refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the overall process of OCR key information extraction in the key information extraction method of this application. For example... Figure 2 As shown, the process of OCR key information extraction typically includes three steps: text detection, text recognition, and information extraction. Information extraction is sometimes further divided into two steps: Semantic Entity Recognition (SER) and Relation Extraction (RE). Text detection identifies text-containing regions in an image; text recognition identifies the detected text regions as computer text; and semantic entity recognition categorizes the detected text content, typically into several fixed categories such as title, question, answer, and others. Specifically, "name" can be identified as a question, and "Zhang San" as an answer. Relation extraction matches the relationships between text to form information with specific, understandable meanings. For example, matching "name" and "Zhang San" reveals that "Zhang San" corresponds to the key information "name".

[0074] It should be noted that in real-world applications, many problems often arise that require specific processing methods to improve recognition accuracy. For example, excessive spacing between characters may cause content that should be merged to be recognized as multiple separate objects; blurry images may lead to characters being identified as similar-looking characters. Therefore, in practical industrial applications, a post-processing step is often added at the end, referring to... Figure 3 , Figure 3 This diagram illustrates the process of adding post-processing OCR key information extraction to the key information extraction method of this application. In practical applications, the detection results of text detection during key information extraction are usually several detection boxes, each of which may contain several characters. These detection boxes are not arranged and combined in a way that humans understand, but rather the detection model combines them according to certain positional rules. (Refer to...) Figure 4 , Figure 4 This is an example image of character detection in the key information extraction method of this application. Figure 4The content, when combined in a way that humans understand, should result in two detection boxes: "Certification Authority" and "XXX Company." However, models often fail to grasp this underlying meaning and can only combine them line by line or character spacing, resulting in four text boxes. To improve overall accuracy, a post-processing step can be added to recombine the content of these four detection boxes into appropriate content. However, due to the lack of user-defined prompts regarding the desired results, this customized post-processing often requires manual intervention, incurring significant manpower costs. Requiring user-provided prompts increases the learning curve, and if users cannot understand the specific post-processing techniques, accuracy may be compromised. Therefore, the key information extraction method proposed in this paper combines human annotation results with model recognition results for text detection in images, thereby reducing manual costs and improving the accuracy and efficiency of image detection.

[0075] In practical applications, refer to Figure 5 , Figure 5 This diagram illustrates the stages of OCR key information extraction in the key information extraction method of this application. Figure 5 As shown, the OCR key information extraction process can include three stages: annotation, modeling, and prediction. During annotation, users can select the key information to be extracted and provide the relationships between the information. This annotation information is then input into the modeling process, where two models are trained: a SER model, used to classify the recognized text information; and a RE model, used to match relationships between text information with category labels. In the prediction process, it will go through the following stages: Figure 3 The five processes shown ultimately yield the extracted results.

[0076] It should be understood that the aforementioned image dataset to be extracted can be a dataset composed of any images for which key information needs to be extracted. In this embodiment, K images for which key information needs to be extracted can be collected in advance to construct the image dataset S to be extracted, wherein each image in the image dataset S has a unique image ID number.

[0077] It should be noted that the above text detection results consist of image information (including image ID, detection box coordinates, etc.) obtained after text detection of the image. In this embodiment, a text detection model can be used to perform text detection on the image dataset S to be extracted, and the text detection results are obtained. The text detection results contain K detection entries, each containing several detection point records. Each detection point record is in dictionary format and records the image ID and the coordinates of the detection box.

[0078] Step S20: Based on the text detection results, perform text recognition on the image dataset to be extracted to obtain text recognition results.

[0079] It should be noted that text recognition of the image dataset to be extracted can be the process of recognizing the text content information in the images. Accordingly, the text recognition result can be a result composed of the text content information of each image in the image dataset to be extracted. In this embodiment, a text recognition model can be used to perform text recognition on the image dataset S to be extracted. The text recognition requires the text detection result in step S10 to obtain the text recognition result. The text recognition result can contain K recognition entries, and each recognition entry contains several text recognition result records. Each text recognition result record is in dictionary format and records the image ID, the coordinate position of the detection box, and the recognized text content information.

[0080] Step S30: Align the text recognition results with the manually labeled results corresponding to the image dataset to be extracted to obtain pattern alignment metadata.

[0081] It should be understood that manual annotation can be the process of human annotators labeling images according to human understanding. In practical applications, refer to... Figure 6 , Figure 6 This is an example image used in the key information extraction method of this application for manual image annotation. Manual annotation can be used to annotate the content as follows: Figure 6 The form shown, where, Figure 6 In this annotation pattern, the blue boxes indicate questions, the yellow boxes indicate answers, and the red connecting lines indicate the relationship between the two. The annotations in this pattern are grouped according to human understanding, even if the content within a single box spans multiple lines, is too widely spaced, or too close together. Furthermore, information that the user doesn't need to retrieve doesn't need to be annotated, for example... Figure 6The text is labeled as "irrelevant content." Therefore, using this type of labeled content for modeling is simple and intuitive, and the labeling workload is relatively small. However, because the information obtained during text detection during prediction does not combine in a way that humans understand, the text combination form used in modeling differs from that used in prediction, thus affecting the model's prediction performance. Simultaneously, because the annotator did not label unlabeled information, this information is missing during the modeling process. However, during prediction, text detection will detect all text in the image, so this information needs to be identified during prediction. The lack of this information during modeling leads to a decrease in the SER model's ability to determine categories. Furthermore, if users label information that does not need to be extracted, it will significantly increase the workload for the annotator, and this additional labeled information will also affect the model's prediction performance due to differences in the form of the text detection results.

[0082] It should be noted that, referring to Figure 7 , Figure 7 This is an example image used for model annotation in the key information extraction method of this application. The annotation method using a detection model can annotate the content to be annotated as follows: Figure 7 The form shown, where, Figure 7 In this annotation pattern, the blue boxes indicate the question category, the yellow boxes indicate the answer category, the purple boxes indicate "other," and the red connecting lines indicate the relationship between the two. The boxes in this annotation pattern are typically detected by a text detection model used during inference, and then annotators label and connect the detection results. Its advantage is that the annotation is based on the detection characteristics of the model itself, ensuring high consistency between the text combinations used in modeling and those used in prediction, thus improving model prediction performance. However, since the annotation information does not include the format of the final result, for example... Figure 7 The annotations did not provide information that "certification" and "organization" were considered as a single element, so the final model output required manual, customized post-processing to identify and recombine the content. Furthermore, the fragmented nature of the detection results necessitates extensive connector annotation by the annotators, reducing efficiency. Also, the fragmentation means that if any part is misidentified by the SER model, the entire result will be incorrect. For example... Figure 7 If any one of the five digits in the monetary value is incorrectly predicted, the location association information cannot be used to correct the result. Because the text detection model cannot recognize human intent, content that should not be merged may be merged into a single detection box, such as... Figure 7The "account" is used for this purpose. It is evident that neither manual nor model-based annotation methods are effective for image annotation. Therefore, this solution combines manual annotation results with model detection results to improve image annotation quality and, consequently, the extraction of key information from subsequent images.

[0083] It should be noted that the above-mentioned manual annotation results can be obtained by annotating the image according to a human-understandable annotation pattern. In this embodiment, the manual annotation results may contain K annotation entries, each annotation entry containing several annotation point records, wherein each annotation point record is in dictionary format, recording the image ID, the coordinate position of the annotation box, the category of the annotation box, and the link information between the annotation boxes. Since the annotation process is performed in a human-understandable way, the obtained annotation information contains the result format that the user expects to obtain in the final prediction output. Figure 6 Taking "Certification Authority" as an example, the user expects the result to be in the form of: {"Certification Authority": "XXX Company"}.

[0084] It should be noted that the above-mentioned pattern alignment metadata can be the metadata obtained after pattern alignment of the text content in the text recognition results and the manually labeled results.

[0085] Step S40: Based on the pattern alignment metadata, the preset text information classification model, and the preset text information relationship matching model, obtain the key information extraction results corresponding to the image dataset to be extracted.

[0086] It should be noted that the aforementioned preset text information classification model can be a model used to classify the identified text information, such as the SER model. The aforementioned preset text information relationship matching model can be a model used to match relationships between text information with category labels, such as the RE model.

[0087] In this embodiment, the metadata information in the pattern alignment metadata can be used to train the SER model and RE model, and the key information of the images in the image dataset to be extracted can be extracted through the trained SER model and RE model.

[0088] In the specific implementation, refer to Figure 8 , Figure 8 This is a flowchart illustrating the overall annotation and training process for the key information extraction method in this application. Figure 8As shown, when extracting key information from the image dataset S, the process begins with manual annotation of the dataset in a human-understandable pattern. Simultaneously, a text detection model is used to detect text in the dataset, yielding detection results. A text recognition model is then used to recognize text based on these detection results, resulting in text recognition result C. The manually annotated and recognized results are then pattern-aligned to obtain pattern alignment metadata M. Next, the device uses the metadata information in M ​​to segment the discrepancies in the text recognition result C, obtaining a segmentation correction record D. This D is then input into the SER model for training, resulting in the SER model. Furthermore, the metadata information in M ​​is used to correct discrepancies in the question-type text within the segmentation correction record D, resulting in a question correction record E. This E is then input into the RE model for training, resulting in the RE model. Finally, the trained SER and RE models can be used to extract the key information from the image dataset S, yielding the key information extraction result.

[0089] This embodiment provides a key information extraction method. The method discloses: performing text detection on a dataset of images to be extracted to obtain text detection results; performing text recognition on the dataset based on the text detection results to obtain text recognition results; aligning the text recognition results with the manually labeled results corresponding to the dataset to be extracted to obtain pattern alignment metadata; and obtaining key information extraction results corresponding to the dataset to be extracted based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model. Compared to existing technologies where information annotation through manual annotation and / or detection model annotation is not very accurate, resulting in low accuracy of key information extraction, this embodiment can align the text recognition results and manually labeled results of the dataset to be extracted to obtain pattern alignment metadata, and obtain key information extraction results based on the pattern alignment metadata, the preset text information classification model, and the preset text information relationship matching model. This allows for the combination of manually labeled results and model recognition results, and key information extraction through the model, thereby solving the technical problem in existing technologies where the accuracy of key information extraction is low due to the low accuracy of information annotation through manual annotation and / or detection model annotation.

[0090] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 9 , Figure 9This is a flowchart illustrating Embodiment 2 of the key information extraction method of this application.

[0091] In this embodiment, step S30 includes steps S301 to S304:

[0092] Step S301: Determine the first metadata based on the text recognition result and the manual annotation result corresponding to the image dataset to be extracted. The first metadata is the text content in the text recognition result and the manual annotation result where the text content is the same and the annotation box overlap is greater than a preset overlap threshold.

[0093] It should be noted that the aforementioned overlap of the annotation boxes can be a value used to characterize the degree of overlap between the annotation boxes in the text recognition results and the manual annotation results. The aforementioned preset overlap threshold can be a threshold used to characterize a large degree of overlap between the annotation boxes in the text recognition results and the manual annotation results. For example, in this embodiment, the preset overlap threshold can be set to 75%. At this time, the aforementioned first metadata, namely, the data where the text content recorded in the text recognition results and the manual annotation results are consistent and the overlap value of their annotation boxes is greater than 75%, is considered valid.

[0094] Specifically, step S301 includes: obtaining a first bounding box in the text recognition result and a second bounding box in the manual annotation result corresponding to the image dataset to be extracted; determining a first target bounding box and a second target bounding box that have the same text content in the first bounding box and the second bounding box; determining the area of ​​overlap and the area of ​​union of regions between the first target bounding box and the second target bounding box; determining the degree of overlap between the first target bounding box and the second target bounding box based on the area of ​​overlap and the area of ​​union of regions; if the degree of overlap is greater than a preset degree of overlap threshold, then recording the text content in the first target bounding box and the second target bounding box as first metadata.

[0095] It should be understood that the first and second annotation boxes mentioned above can be annotation boxes used to annotate text content in the text recognition results and manual annotation results, respectively. Correspondingly, the first and second target annotation boxes mentioned above can be annotation boxes with the same text content in the text recognition results and manual annotation results, respectively.

[0096] It is understandable that the overlapping area of ​​the above-mentioned regions can be the area of ​​the overlapping regions of the first target annotation box and the second target annotation box; the union area of ​​the above-mentioned regions can be the union area of ​​the first target annotation box and the second target annotation box.

[0097] In this embodiment, the overlap area of ​​the first target annotation box and the second target annotation box can be divided by the area of ​​their union to obtain the annotation box overlap degree. In other words, the overlap area can be divided by the area of ​​their union to obtain the annotation box overlap degree. Then, the device can compare the annotation box overlap degree with a preset overlap degree threshold. If the overlap degree is greater than the preset threshold, the text content in the first target annotation box and the second target annotation box can be recorded in the first metadata.

[0098] Step S302: Determine the second metadata in the text recognition result. The second metadata is the text content that exists in all image samples in the text recognition result and appears a number of times that meets the preset number requirement.

[0099] It should be understood that the aforementioned second metadata can be the text content that appears only once in each image sample in the text recognition result. That is, in this embodiment, the aforementioned preset number of times requirement can be set to once, so that the content that appears fixedly in the image can be saved as the second metadata.

[0100] Step S303: Obtain third-party metadata based on the difference between the text recognition result and the manual annotation result.

[0101] It is understandable that the aforementioned third-party metadata can be content that differs between the text recognition results and the manually labeled results.

[0102] Furthermore, step S303 includes:

[0103] Step S303a: Determine the difference recording method based on the differences between the text recognition results and the manual annotation results.

[0104] It should be understood that the above-mentioned difference recording method can be used to record the differences between text recognition results and manually labeled results. In practical applications, the difference recording method is related to the needs of the actual application scenario. Different application scenarios encounter different difference problems, so different recording methods are required. The level of detail in the difference recording is related to the level of detail required for prediction.

[0105] Specifically, step S303a includes: determining detection boxes in the image dataset to be extracted based on the text recognition result; determining the difference type corresponding to the difference content between the text recognition result and the manual annotation result based on the detection boxes and the annotation boxes corresponding to the manual annotation result; and determining the difference recording method according to the difference type.

[0106] It should be noted that, referring to Figure 10 , Figure 10This is a diagram illustrating the differences in the key information extraction method used in this application. Figure 10 The diagram shows several specific types of differences and recording methods. The blue boxes are the detection boxes obtained by the detection model, which are the detection boxes in the above image dataset to be extracted; the orange boxes are the annotation boxes obtained by manual annotation, which are the annotation boxes corresponding to the above manual annotation results.

[0107] It should be noted that, in this embodiment, the difference types corresponding to the differences between the text recognition results and the manual annotation results may include... Figure 10 The four types of discrepancies are shown below. Difference a indicates that the user-annotated content spans multiple lines, while the text detection result is an independent detection box for each line. Difference b indicates that the user-annotated content consists of two independent parts, while the text detection result is a single detection box containing both parts of the user-annotated content. Difference c indicates that the user-annotated content consists of horizontally continuous characters, but due to the large spacing between the characters, the text detection result is multiple independent character boxes. Difference d indicates that due to the small spacing between different content items, the detection box in the text detection results contains content that does not need to be extracted.

[0108] In practical applications, Figure 10 The difference types shown can be determined based on the overlap between the bounding boxes and the check boxes, as well as their specific content. For example, in differences a and c, the overlap can be defined as the overlapping area of ​​the two boxes divided by the area of ​​the check box. If the overlap is greater than 80%, it means that the content of the check box belongs to the content of the bounding box. If multiple different check boxes belong to the bounding box, then it meets the criteria. Figure 10 The situation is shown in difference a or difference c. Further analysis of whether the multiple annotation boxes are in different rows or the same row can distinguish between them. Figure 10 One of the cases shown in differences a and c.

[0109] Step S303b: Record the difference content using the difference recording method to obtain difference metadata records.

[0110] It should be noted that after determining the difference type corresponding to the difference between the text recognition result and the manual annotation result, the difference data can be recorded using the corresponding difference recording method to obtain the above-mentioned difference metadata. For example, taking difference type a as an example, the metadata record can be represented as: {"difference_type": "a", "annotation_type": "question", "user_annotation": "certification authority", "user_annotation_position": [0.1, 0.2, 0.15, 0.22], "model_detected": ["certification", "institution"], "user_annotation_position": [0.1, 0.2, 0.15, 0.22], "model_detected_position": [[0.1, 0.2, 0.15, 0.22], [0.1, 0.2, 0.15, 0.22]], "multi_lines": "true", "lines_spacing": 0.8}. The metadata includes: `difference_type` records the type of difference; `annotation_type` records the type of the annotation box, typically `header`, `question`, `answer`, and `other`, representing title, question, answer, and other types respectively; `user_annotation` records the text content entered by the user during annotation; `user_annotation_position` records the coordinates of the annotation box during user annotation; `model_detected` records the text content detected by the recognition model, which is in list form due to the existence of multiple text boxes; `model_detected_position` records the coordinates of the text boxes detected by the detection model, which is also in list form due to the existence of multiple detection boxes; `multi_lines` indicates whether the annotation box has multiple lines (true if multiple lines exist, false otherwise); `lines_spacing` represents the line spacing, in units of the average height of the detection boxes. Other types of differences can be constructed using similar metadata records.

[0111] Step S303c: Use a preset deduplication rule to deduplicatize the difference metadata records to obtain third metadata.

[0112] It should be noted that the differences between each image sample may vary, so these difference data can be deduplicated. In this embodiment, according to preset deduplication rules, difference metadata records with the same difference type, the same annotation box type, the same annotation content, and an IOU (Intersection over Union) value greater than 85% are deduplicated. During deduplication, the record with the larger difference value is retained to obtain the aforementioned third metadata.

[0113] Step S304: Merge the first metadata, the second metadata, and the third metadata to obtain pattern-aligned metadata.

[0114] It is understood that in this embodiment, the first metadata, the second metadata, and the third metadata can be merged to obtain a pattern-aligned metadata.

[0115] In this embodiment, a method is disclosed to determine first metadata based on the text recognition results and the corresponding manual annotation results of the image dataset to be extracted. The first metadata consists of text content that is identical in both the text recognition results and the manual annotation results, and whose bounding box overlap is greater than a preset overlap threshold. Second metadata is determined from the text recognition results. The second metadata consists of text content that exists in all image samples in the text recognition results and whose occurrence frequency meets a preset requirement. Third metadata is obtained based on the differences between the text recognition results and the manual annotation results. The first, second, and third metadata are merged to obtain pattern alignment metadata. Since this embodiment can perform pattern alignment between the manually annotated content and the model detection content to obtain pattern alignment metadata, it allows subsequent model prediction using the pattern alignment metadata. This ensures that while maintaining the model training and prediction effects, the output format of the pattern remains consistent with human-understandable patterns, greatly reducing the need for customized post-processing due to inconsistencies between the two patterns, and improving the accuracy of key information extraction.

[0116] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 11 , Figure 11 This is a flowchart illustrating the third embodiment of the key information extraction method of this application.

[0117] In this embodiment, step S40 includes steps S401 to S405:

[0118] Step S401: Based on the pattern alignment metadata, perform content splitting on the differences in the text recognition results to obtain splitting correction records.

[0119] In this embodiment, the metadata information in the pattern alignment metadata M can be used to identify characters in the text recognition result C that contain similar metadata. Figure 10 Contents with similar differences b and d are split into separate content, resulting in the aforementioned splitting and correction records.

[0120] Step S402: Input the split correction record into the preset text information classification model to obtain the information classification prediction result.

[0121] It should be understood that in this embodiment, the split correction record can be input into a preset text information classification model for model training. At this time, the preset text information classification model can output the prediction result, namely the above-mentioned information classification prediction result.

[0122] Step S403: Correct the content of the question type text of the information classification prediction result by means of the pattern alignment metadata, and obtain the question correction record.

[0123] In this embodiment, the metadata information in the pattern alignment metadata M can be used to identify text containing problem types in the split correction record. Figure 10 The content of differences a and c is corrected to obtain the above problem correction record.

[0124] Step S404: Input the problem correction record into the preset text information relationship matching model to obtain the information relationship matching prediction result.

[0125] It should be understood that in this embodiment, the problem correction record can be input into the preset text information relationship matching model for model training. At this time, the preset text information relationship matching model can output the prediction result, namely the above-mentioned information relationship matching prediction result.

[0126] Step S405: Correct the content of the answer type text of the information relationship matching prediction result by means of the pattern alignment metadata, so as to obtain the key information extraction result corresponding to the image dataset to be extracted.

[0127] In this embodiment, metadata information in the pattern alignment metadata M can be used to correct discrepancies in the text of the answer type in the information relationship matching prediction results, resulting in answer correction records. This enables customized post-processing and ultimately outputs the key information extraction processing corresponding to the image dataset to be extracted. This embodiment adds three stages—content splitting, question correction, and answer correction—to the model's conventional prediction process, applying pattern alignment metadata to each stage. This ensures that the final output matches a human-understandable output format, thereby improving model training effectiveness, maintaining consistency with human-understandable output formats, and reducing R&D investment in customized post-processing.

[0128] In the specific implementation, refer to Figure 12 , Figure 12 This is a flowchart illustrating the overall prediction process for the key information extraction method in this application. Figure 12 As shown, firstly, the device can use a text detection model to perform text detection on image P, obtaining text detection result B'. Text detection result B' contains several detection point records, each in dictionary format, recording the image ID and the coordinate position information of the detection box. Then, a text recognition model can use the text detection result B' to perform text recognition on image P, obtaining text recognition result C'. Text recognition result C' contains several text recognition result records, each in dictionary format, recording the image ID, the coordinate position of the detection box, and the recognized text content. Afterwards, the metadata information in the pattern alignment metadata M obtained during training can be used to align text in text recognition result C' with... Figure 10 Content with similar differences b and d is split into content correction records D', which are then input into the SER model for prediction, yielding the SER prediction result F'. Subsequently, metadata information in the pattern alignment metadata M can be used to further analyze the text of the question type in the SER prediction result F'. Figure 10 The content of differences a and c is corrected to obtain a problem correction record E'. This record E' is then input into the RE model for prediction, yielding the RE model prediction result G'. Finally, using the metadata information in the pattern alignment metadata M, the differing content in the answer type text of the RE model prediction result G' is corrected, resulting in an answer correction record H'. If some problems remain difficult to correct after the above steps, customized post-processing can be implemented based on the specific problem, ultimately yielding a customized post-processing output O', which is the final result of extracting key information from the image.

[0129] This embodiment discloses a method for content segmentation of discrepancies in text recognition results based on pattern alignment metadata, obtaining segmentation correction records; inputting the segmentation correction records into a preset text information classification model to obtain information classification prediction results; correcting the question type text in the information classification prediction results using pattern alignment metadata, obtaining question correction records; inputting the question correction records into a preset text information relationship matching model to obtain information relationship matching prediction results; and correcting the answer type text in the information relationship matching prediction results using pattern alignment metadata to obtain key information extraction results corresponding to the image dataset to be extracted. Since this embodiment applies pattern alignment metadata to the three stages of content segmentation, question correction, and answer correction in the model training process, it can correct some recognized content, thereby improving the effectiveness of model training. Furthermore, the automated correction and matching methods used in this embodiment reduce human intervention.

[0130] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the key information extraction method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0131] This application also provides a key information extraction device, please refer to... Figure 13 The key information extraction device includes:

[0132] The text detection module 10 is used to perform text detection on the image dataset to be extracted and obtain text detection results;

[0133] The text recognition module 20 is used to perform text recognition on the image dataset to be extracted based on the text detection results, and obtain text recognition results;

[0134] The pattern alignment module 30 is used to perform pattern alignment between the text recognition result and the manually labeled result corresponding to the image dataset to be extracted, and obtain pattern alignment metadata.

[0135] The information extraction module 40 is used to obtain the key information extraction results corresponding to the image dataset to be extracted based on the pattern-aligned metadata, the preset text information classification model, and the preset text information relationship matching model.

[0136] The key information extraction device provided in this application, employing the key information extraction method described in the above embodiments, can solve the technical problem in the prior art where the accuracy of key information extraction is low due to the low precision of information annotation through manual annotation and / or detection model annotation methods during OCR key information extraction. Compared with the prior art, the beneficial effects of the key information extraction device provided in this application are the same as those of the key information extraction method provided in the above embodiments, and other technical features in the key information extraction device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0137] This application provides a key information extraction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the key information extraction method in the first embodiment described above.

[0138] The following is for reference. Figure 14 The diagram illustrates a structural schematic of a key information extraction device suitable for implementing embodiments of this application. The key information extraction device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 14 The key information extraction device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0139] like Figure 14As shown, the critical information extraction device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the critical information extraction device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the critical information extraction device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows critical information extraction devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0140] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0141] The key information extraction device provided in this application, employing the key information extraction method described in the above embodiments, can solve the technical problem of key information extraction. Compared with the prior art, the beneficial effects of the key information extraction device provided in this application are the same as those of the key information extraction method described in the above embodiments, and other technical features of this key information extraction device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0142] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0143] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0144] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the key information extraction method in the above embodiments.

[0145] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0146] The aforementioned computer-readable storage medium may be included in the critical information extraction device; or it may exist independently and not be assembled into the critical information extraction device.

[0147] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the key information extraction device, the key information extraction device performs the following actions: performs text detection on the image dataset to be extracted, and obtains text detection results; performs text recognition on the image dataset to be extracted based on the text detection results, and obtains text recognition results; performs pattern alignment between the text recognition results and the manually labeled results corresponding to the image dataset to be extracted, and obtains pattern alignment metadata; and obtains the key information extraction results corresponding to the image dataset to be extracted based on the pattern alignment metadata, a preset text information classification model, and a preset text information relationship matching model.

[0148] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0151] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described key information extraction method. This solves the technical problem in the prior art where the accuracy of key information extraction is low due to the low precision of information annotation achieved through manual annotation and / or detection model annotation methods during OCR key information extraction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the key information extraction method provided in the above embodiments, and will not be repeated here.

[0152] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for extracting key information, characterized in that, The method includes: Perform text detection on the image dataset to be extracted and obtain the text detection results; Based on the text detection results, text recognition is performed on the image dataset to be extracted to obtain text recognition results; The text recognition results and the manually labeled results corresponding to the image dataset to be extracted are pattern aligned to obtain pattern alignment metadata. Based on the pattern alignment metadata, the preset text information classification model, and the preset text information relationship matching model, the key information extraction results corresponding to the image dataset to be extracted are obtained. The step of performing pattern alignment between the text recognition result and the manually labeled result corresponding to the image dataset to be extracted, and obtaining pattern alignment metadata, includes: The first metadata is determined based on the text recognition result and the manual annotation result corresponding to the image dataset to be extracted. The first metadata is the text content in the text recognition result and the manual annotation result where the text content is the same and the annotation box overlap is greater than a preset overlap threshold. Determine the second metadata in the text recognition result, wherein the second metadata is the text content that exists in each image sample of the text recognition result and appears a preset number of times; Third-party metadata is obtained based on the differences between the text recognition results and the manually labeled results; The first metadata, the second metadata, and the third metadata are merged to obtain pattern-aligned metadata.

2. The method as described in claim 1, characterized in that, The step of determining the first metadata based on the text recognition result and the manually labeled result corresponding to the image dataset to be extracted includes: Obtain the first bounding box from the text recognition result and the second bounding box from the manual annotation result corresponding to the image dataset to be extracted; Identify a first target annotation box and a second target annotation box that contain the same text content as the first annotation box and the second annotation box; Determine the area of ​​overlap and the area of ​​union of the regions between the first target bounding box and the second target bounding box; The overlap ratio between the first target bounding box and the second target bounding box is determined based on the overlapping area of ​​the regions and the union area of ​​the regions. If the overlap of the annotation boxes is greater than a preset overlap threshold, then the text content in the first target annotation box and the second target annotation box is recorded as first metadata.

3. The method as described in claim 1, characterized in that, The step of obtaining third-party metadata based on the difference between the text recognition result and the manually labeled result includes: The method for recording differences is determined based on the differences between the text recognition results and the manual annotation results. The difference content is recorded using the difference recording method to obtain difference metadata records; The differential metadata records are deduplicated using a preset deduplication rule to obtain third metadata.

4. The method as described in claim 3, characterized in that, The step of determining the difference recording method based on the difference between the text recognition result and the manual annotation result includes: Based on the text recognition results, the detection boxes in the image dataset to be extracted are determined; Based on the detection box and the annotation box corresponding to the manual annotation result, determine the difference type corresponding to the difference content between the text recognition result and the manual annotation result; The method of recording differences is determined based on the type of difference.

5. The method according to any one of claims 1 to 4, characterized in that, The step of obtaining the key information extraction results corresponding to the image dataset to be extracted based on the pattern alignment metadata, the preset text information classification model, and the preset text information relationship matching model includes: Based on the pattern alignment metadata, the differences in the text recognition results are split to obtain splitting correction records; The splitting and correcting records are input into a preset text information classification model to obtain information classification prediction results; The problem type text of the information classification prediction result is corrected by aligning the metadata with the pattern, and a problem correction record is obtained. The problem correction record is input into a preset text information relationship matching model to obtain the information relationship matching prediction result; The answer type text of the information relationship matching prediction result is corrected by using the pattern alignment metadata to obtain the key information extraction result corresponding to the image dataset to be extracted.

6. The method according to any one of claims 1 to 4, characterized in that, Before the step of performing text detection on the image dataset to be extracted and obtaining the text detection results, the method further includes: Determine the coordinates, categories, and link information of the bounding boxes in the image dataset to be extracted; Several annotation point records are generated based on the image identifiers, the coordinate positions of the annotation boxes, the categories of the annotation boxes, and the link information of the annotation boxes in the image dataset to be extracted; The image dataset to be extracted is annotated based on each annotation point record to obtain the manual annotation results corresponding to the image dataset to be extracted.

7. A key information extraction device, characterized in that, The device includes: The text detection module is used to perform text detection on the image dataset to be extracted and obtain text detection results; The text recognition module is used to perform text recognition on the image dataset to be extracted based on the text detection results, and obtain text recognition results; The pattern alignment module is used to perform pattern alignment between the text recognition results and the manually labeled results corresponding to the image dataset to be extracted, and to obtain pattern alignment metadata. The information extraction module is used to obtain the key information extraction results corresponding to the image dataset to be extracted based on the pattern-aligned metadata, the preset text information classification model, and the preset text information relationship matching model. The pattern alignment module is further configured to determine first metadata based on the text recognition result and the manual annotation result corresponding to the image dataset to be extracted, wherein the first metadata is text content in the text recognition result and the manual annotation result where the text content is the same and the annotation box overlap is greater than a preset overlap threshold; determine second metadata in the text recognition result, wherein the second metadata is text content in the text recognition result where it exists in all image samples and the number of occurrences meets a preset number requirement; obtain third metadata based on the differences between the text recognition result and the manual annotation result; and merge the first metadata, the second metadata, and the third metadata to obtain pattern alignment metadata.

8. A key information extraction device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the key information extraction method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the key information extraction method as described in any one of claims 1 to 6.

Images