Material information extraction method and device based on template alignment and electronic equipment

Abstract

The invention provides a material information extraction method and device based on template alignment and electronic equipment, and relates to the technical field of information extraction, and the method comprises the steps: obtaining a material template; the material template comprises a target reference area information list, and the target reference area information list comprises target reference area information and a selection box thereof; performing full-text character recognition on the to-be-recognized material to obtain character information of the to-be-recognized material; the character information of the material to be recognized is compared with the target reference area information list, a reference area information list meeting the target similarity is obtained, and the reference area information list comprises reference area information and a selection box of the reference area information; determining the scaling of the to-be-identified material based on the selection box of the reference area information and the selection box of the target reference area information; and aligning the to-be-identified material with the material template based on the scaling of the to-be-identified material to complete material extraction of the to-be-identified material. According to the method, a large amount of labeling training is not needed, one template is configured for one material, and accurate extraction of material information can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to a material information extraction method, device and electronic equipment based on template alignment. Background Art

[0002] With the rapid development of information technology, especially the continuous advancement of digitalization and artificial intelligence technology, the processing and utilization of paper materials are undergoing unprecedented changes. Traditionally, the organization, summary, key information extraction and statistics of paper materials need to be done manually, which is not only inefficient but also prone to errors. With the popularization of electronic documents, people have begun to explore the use of computers and artificial intelligence technology to automatically complete these tasks.

[0003] In the field of information extraction, a commonly used method is to combine optical character recognition (OCR) technology with an information extraction model based on deep learning. This method first uses OCR technology to recognize the text in the paper material image, and then uses the deep learning model to structure the recognized text, extract the required key information, and output it in a table form. However, this method faces many challenges in practical applications.

[0004] First, deep learning-based information extraction models require a large number of high-quality labeled samples for training. In practice, obtaining a sufficient number of labeled samples is often a time-consuming and costly task. In addition, deep learning models are highly dependent on training data. If the training data is insufficient or of low quality, the extraction effect of the model will be seriously affected.

[0005] Secondly, existing information extraction techniques usually rely on contextual dialogue. This means that when the writing style of new samples changes, the extraction effect of the model may be seriously affected. For example, if the format or content of the template material changes slightly, the model may not be able to correctly identify and extract the required key information.

[0006] Finally, the existing information extraction technology usually can only process one type of template material per model. When it is necessary to process multiple types of template materials, multiple models need to be trained, which not only increases the development cost but also reduces the generalization ability of the system.

[0007] Therefore, a material information extraction method, device and electronic device based on template alignment are proposed. Summary of the invention

[0008] This specification provides a material information extraction method, device and electronic device based on template alignment. It does not require a large amount of annotation training. Just configure one template for one material to achieve accurate extraction of material information.

[0009] This specification provides a material information extraction method based on template alignment, comprising:

[0010] Acquire a material template; the material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof;

[0011] Perform full text recognition on the material to be identified to obtain text information of the material to be identified;

[0012] Compare the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box;

[0013] Determining a scaling ratio of the material to be identified based on a selection box of the reference area information and a selection box of the target reference area information;

[0014] The material to be identified is aligned with the material template based on the scaling ratio of the material to be identified, thereby completing the material extraction of the material to be identified.

[0015] Optionally, the obtaining of a material template includes:

[0016] Acquire a standard material image marked with target reference area information and its selection box, and target identification area selection box;

[0017] Obtaining a target reference area information list based on the target reference area information and its selection box;

[0018] Binding the keyword of the target identification area and the target identification area selection box to form a dictionary relationship;

[0019] A material template is obtained based on the target reference area information list and the dictionary relationship.

[0020] Optionally, the determining the scaling ratio of the material to be identified based on the selection box of the reference area information and the selection box of the target reference area information includes:

[0021] The material to be identified includes a plurality of selection boxes of reference area information, and the material template includes a plurality of selection boxes of target reference area information;

[0022] Respectively acquiring coordinate information of the frame selection boxes of the plurality of reference area information and coordinate information of the frame selection boxes of the plurality of target reference area information;

[0023] Determine, based on the coordinate information of the frame selection box of the reference area information, the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0024] Determine a first minimum circumscribed rectangular frame of the material to be identified based on the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0025] Determine, based on the coordinate information of the frame selection box of the target reference area information, the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0026] Determine a second minimum circumscribed rectangular frame of the material template based on the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0027] The scaling ratio of the to-be-identified material is determined based on the first minimum circumscribed rectangular frame and the second minimum circumscribed rectangular frame.

[0028] Optionally, aligning the material to be identified with the material template based on the scaling ratio of the material to be identified to complete the material extraction of the material to be identified includes:

[0029] Calculate the distance values ​​from each target recognition area selection box to the four sides of the first minimum circumscribed rectangular box respectively;

[0030] Determine the identification area selection box and coordinate information of the material to be identified based on the scaling ratio of the material to be identified and the distance value;

[0031] Determine the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information, and merge the optical character recognition detection frames located in the same recognition area selection frame;

[0032] The material extraction of the material to be identified is completed through the combined optical character recognition detection frame.

[0033] Optionally, determining the recognition area of ​​the optical character recognition detection box based on the recognition area selection box and its coordinate information includes:

[0034] When the center point of the optical character recognition detection frame is located within the recognition area selection frame;

[0035] or,

[0036] When any point of the optical character recognition detection frame is set within the recognition area selection frame, and the height of the overlapping area between the optical character recognition detection frame and the recognition area selection frame is greater than half of the height of the optical character recognition detection frame.

[0037] Optionally, the determination of the target reference area information and its selection box includes:

[0038] The selection criteria of the target reference area information include: the width of the text interval of the target reference area information is smaller than the width of the text; the text of the target reference area information does not cross rows or columns; the text of the target reference area information does not appear repeatedly in the standard material image;

[0039] The setting standard of the selection box of the target reference area information includes: setting it close to the target reference area information, and all the target reference area information needs to be selected.

[0040] Optionally, the method further includes: entering the extracted material into an information database based on the dictionary relationship.

[0041] This specification provides a material information extraction device based on template alignment, comprising:

[0042] An acquisition module, used for acquiring a material template; the material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof;

[0043] A recognition module is used to perform full text recognition on the material to be recognized to obtain text information of the material to be recognized;

[0044] A comparison module, used for comparing the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box;

[0045] A determination module, configured to determine a scaling ratio of the material to be identified based on a selection box of the reference area information and a selection box of the target reference area information;

[0046] An alignment module is used to align the material to be identified with the material template based on the scaling ratio of the material to be identified, so as to complete the material extraction of the material to be identified.

[0047] Optionally, the acquisition module includes:

[0048] Acquire a standard material image marked with target reference area information and its selection box, and target identification area selection box;

[0049] Obtaining a target reference area information list based on the target reference area information and its selection box;

[0050] Binding the keyword of the target identification area and the target identification area selection box to form a dictionary relationship;

[0051] A material template is obtained based on the target reference area information list and the dictionary relationship.

[0052] Optionally, the determining module includes:

[0053] The material to be identified includes a plurality of selection boxes of reference area information, and the material template includes a plurality of selection boxes of target reference area information;

[0054] Respectively acquiring coordinate information of the frame selection boxes of the plurality of reference area information and coordinate information of the frame selection boxes of the plurality of target reference area information;

[0055] Determine, based on the coordinate information of the frame selection box of the reference area information, the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0056] Determine a first minimum circumscribed rectangular frame of the material to be identified based on the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0057] Determine, based on the coordinate information of the frame selection box of the target reference area information, the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0058] Determine a second minimum circumscribed rectangular frame of the material template based on the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0059] The scaling ratio of the to-be-identified material is determined based on the first minimum circumscribed rectangular frame and the second minimum circumscribed rectangular frame.

[0060] Optionally, the alignment module includes:

[0061] Calculate the distance values ​​from each target recognition area selection box to the four sides of the first minimum circumscribed rectangular box respectively;

[0062] Determine the identification area selection box and coordinate information of the material to be identified based on the scaling ratio of the material to be identified and the distance value;

[0063] Determine the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information, and merge the optical character recognition detection frames located in the same recognition area selection frame;

[0064] The material extraction of the material to be identified is completed through the combined optical character recognition detection frame.

[0065] Optionally, determining the recognition area of ​​the optical character recognition detection box based on the recognition area selection box and its coordinate information includes:

[0066] When the center point of the optical character recognition detection frame is located within the recognition area selection frame;

[0067] or,

[0068] When any point of the optical character recognition detection frame is set within the recognition area selection frame, and the height of the overlapping area between the optical character recognition detection frame and the recognition area selection frame is greater than half of the height of the optical character recognition detection frame.

[0069] Optionally, the determination of the target reference area information and its selection box includes:

[0070] The selection criteria of the target reference area information include: the width of the text interval of the target reference area information is smaller than the width of the text; the text of the target reference area information does not cross rows or columns; the text of the target reference area information does not appear repeatedly in the standard material image;

[0071] The setting standard of the selection box of the target reference area information includes: setting it close to the target reference area information, and all the target reference area information needs to be selected.

[0072] Optionally, the method further includes: entering the extracted material into an information database based on the dictionary relationship.

[0073] This specification also provides an electronic device, wherein the electronic device includes:

[0074] processor; and,

[0075] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.

[0076] The present specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, any of the above methods is implemented.

[0077] In the present invention, a large number of labeled samples are not required for training, which greatly reduces the cost of data collection and annotation, and also shortens the cycle of model development and optimization. By configuring different templates for different material types, the system can handle multiple types of template materials without the need to develop a separate model for each type. When a new material type needs to be processed, the user only needs to define a new template configuration without retraining the model, which greatly reduces the cost of development and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 A schematic diagram of the principle of a material information extraction method based on template alignment provided in an embodiment of this specification;

[0080] Figure 2 A schematic diagram of the structure of a material information extraction device based on template alignment provided in an embodiment of this specification;

[0081] Figure 3 A schematic diagram of a material template and a material to be identified provided in the embodiments of this specification;

[0082] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0083] Figure 5 A schematic diagram of a computer-readable medium provided for an embodiment of this specification. DETAILED DESCRIPTION

[0084] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0085] The following is combined with Figure 1-5 The exemplary embodiments of the present invention are described more fully. However, the exemplary embodiments can be implemented in various forms, and it should not be understood that the present invention is limited to the embodiments set forth herein. On the contrary, providing these exemplary embodiments can make the present invention more comprehensive and complete, and it is more convenient to fully convey the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0086] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0087] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0088] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0089] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0090] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0091] Figure 1 A schematic diagram of the principle of a material information extraction method based on template alignment provided in an embodiment of this specification, the method may include:

[0092] S110: Acquire a material template; the material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof;

[0093] Optionally, the S110 includes:

[0094] Acquire a standard material image marked with target reference area information and its selection box, and target identification area selection box;

[0095] Obtaining a target reference area information list based on the target reference area information and its selection box;

[0096] Binding the keyword of the target identification area and the target identification area selection box to form a dictionary relationship;

[0097] A material template is obtained based on the target reference area information list and the dictionary relationship.

[0098] In a specific implementation of the present specification, the standard material image is annotated by an image annotation tool (such as labelImg, COCO Annotator, etc.), the annotated image is stored in a designated folder, and it is ensured that the image file and the corresponding annotation file (such as XML, JSON, etc.) have the same file name or can be associated with each other in some way.

[0099] Arrange all the target reference area information and its selection box into a separate list, where each element is a dictionary or object containing the target reference area information. Create a dictionary to store the keywords of the target identification area and the binding relationship between the target identification area selection box. The key of the dictionary can be the target reference area information or a unique identifier, and the value is an object or dictionary containing all the annotation information. Determine the basic structure and layout of the material template based on the target reference area information list and the dictionary relationship.

[0100] S120: Perform full text recognition on the material to be recognized to obtain text information of the material to be recognized;

[0101] In the specific implementation of this specification, noise and impurities in the image are reduced to make the text part clearer. The color or grayscale image is converted into a black and white binary image to simplify the processing process. If the material to be identified is tilted, it needs to be corrected to a horizontal position. The unnecessary parts are cut off. The OCR engine (such as Tesseract OCR, Google CloudVision, etc.) is used to perform character recognition on the material to be identified, and the OCR engine converts the characters in the image into computer-readable text.

[0102] S130: Compare the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box;

[0103] S140: Determine a scaling ratio of the material to be identified based on the selection box of the reference area information and the selection box of the target reference area information;

[0104] Optionally, the S140 includes:

[0105] The material to be identified includes a plurality of selection boxes of reference area information, and the material template includes a plurality of selection boxes of target reference area information;

[0106] Respectively acquiring coordinate information of the frame selection boxes of the plurality of reference area information and coordinate information of the frame selection boxes of the plurality of target reference area information;

[0107] Determine, based on the coordinate information of the frame selection box of the reference area information, the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0108] Determine a first minimum circumscribed rectangular frame of the material to be identified based on the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0109] Determine, based on the coordinate information of the frame selection box of the target reference area information, the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0110] Determine a second minimum circumscribed rectangular frame of the material template based on the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0111] The scaling ratio of the to-be-identified material is determined based on the first minimum circumscribed rectangular frame and the second minimum circumscribed rectangular frame.

[0112] In the specific implementation of this specification, Figure 2 As shown, among all the reference area information selection boxes in the material to be identified, the coordinate information of the reference area information selection box with the largest horizontal coordinate, the coordinate information of the reference area information selection box with the smallest horizontal coordinate, the coordinate information of the reference area information selection box with the largest vertical coordinate, and the coordinate information of the reference area information selection box with the smallest vertical coordinate are found. Among all the target reference area information selection boxes in the material template, the coordinate information of the target reference area information selection box with the largest horizontal coordinate, the coordinate information of the target reference area information selection box with the smallest horizontal coordinate, the coordinate information of the target reference area information selection box with the largest vertical coordinate, and the coordinate information of the target reference area information selection box with the smallest vertical coordinate are found.

[0113] According to the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate, the minimum circumscribed rectangular frame of the frame selection box containing all the reference area information in the material to be identified is drawn. According to the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate, the minimum circumscribed rectangular frame of the frame selection box containing all the target reference area information in the material template is drawn.

[0114] Compare the width and height of the two minimum bounding rectangles. Width scaling ratio = width of the first minimum bounding rectangle / width of the second minimum bounding rectangle. Height scaling ratio = height of the first minimum bounding rectangle / height of the second minimum bounding rectangle.

[0115] S150: Aligning the material to be identified with the material template based on the scaling ratio of the material to be identified, thereby completing material extraction of the material to be identified.

[0116] Optionally, the S150 includes:

[0117] Calculate the distance values ​​from each target recognition area selection box to the four sides of the first minimum circumscribed rectangular box respectively;

[0118] Determine the identification area selection box and coordinate information of the material to be identified based on the scaling ratio of the material to be identified and the distance value;

[0119] Determine the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information, and merge the optical character recognition detection frames located in the same recognition area selection frame;

[0120] The material extraction of the material to be identified is completed through the combined optical character recognition detection frame.

[0121] In the specific implementation of this specification, all OCR detection frames are traversed and their recognition areas are compared. If the recognition areas of two or more detection frames are the same or overlapped, they are merged into one detection frame.

[0122] Optionally, determining the recognition area of ​​the optical character recognition detection box based on the recognition area selection box and its coordinate information includes:

[0123] When the center point of the optical character recognition detection frame is located within the recognition area selection frame;

[0124] or,

[0125] When any point of the optical character recognition detection frame is set within the recognition area selection frame, and the height of the overlapping area between the optical character recognition detection frame and the recognition area selection frame is greater than half of the height of the optical character recognition detection frame.

[0126] Optionally, the determination of the target reference area information and its selection box includes:

[0127] The selection criteria of the target reference area information include: the width of the text interval of the target reference area information is smaller than the width of the text; the text of the target reference area information does not cross rows or columns; the text of the target reference area information does not appear repeatedly in the standard material image;

[0128] The setting standard of the selection box of the target reference area information includes: setting it close to the target reference area information, and all the target reference area information needs to be selected.

[0129] Optionally, the method further includes: entering the extracted material into an information database based on the dictionary relationship.

[0130] In the present invention, a large number of labeled samples are not required for training, which greatly reduces the cost of data collection and annotation, and also shortens the cycle of model development and optimization. By configuring different templates for different material types, the system can handle multiple types of template materials without the need to develop a separate model for each type. When a new material type needs to be processed, the user only needs to define a new template configuration without retraining the model, which greatly reduces the cost of development and maintenance.

[0131] Figure 3 A schematic diagram of a material information extraction device based on template alignment provided in an embodiment of this specification, the device may include:

[0132] An acquisition module 10 is used to acquire a material template; the material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof;

[0133] The recognition module 20 is used to perform full text recognition on the material to be recognized to obtain text information of the material to be recognized;

[0134] A comparison module 30 is used to compare the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box;

[0135] A determination module 40, configured to determine a scaling ratio of the material to be identified based on a selection box of the reference area information and a selection box of the target reference area information;

[0136] The alignment module 50 is used to align the material to be identified with the material template based on the scaling ratio of the material to be identified, so as to complete the material extraction of the material to be identified.

[0137] Optionally, the acquisition module 10 includes:

[0138] Acquire a standard material image marked with target reference area information and its selection box, and target identification area selection box;

[0139] Obtaining a target reference area information list based on the target reference area information and its selection box;

[0140] Binding the keyword of the target identification area and the target identification area selection box to form a dictionary relationship;

[0141] A material template is obtained based on the target reference area information list and the dictionary relationship.

[0142] Optionally, the determining module 40 includes:

[0143] The material to be identified includes a plurality of selection boxes of reference area information, and the material template includes a plurality of selection boxes of target reference area information;

[0144] Respectively acquiring coordinate information of the frame selection boxes of the plurality of reference area information and coordinate information of the frame selection boxes of the plurality of target reference area information;

[0145] Determine, based on the coordinate information of the frame selection box of the reference area information, the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0146] Determine a first minimum circumscribed rectangular frame of the material to be identified based on the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate;

[0147] Determine, based on the coordinate information of the frame selection box of the target reference area information, the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0148] Determine a second minimum circumscribed rectangular frame of the material template based on the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate;

[0149] The scaling ratio of the to-be-identified material is determined based on the first minimum circumscribed rectangular frame and the second minimum circumscribed rectangular frame.

[0150] Optionally, the alignment module 50 includes:

[0151] Calculate the distance values ​​from each target recognition area selection box to the four sides of the first minimum circumscribed rectangular box respectively;

[0152] Determine the identification area selection box and coordinate information of the material to be identified based on the scaling ratio of the material to be identified and the distance value;

[0153] Determine the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information, and merge the optical character recognition detection frames located in the same recognition area selection frame;

[0154] The material extraction of the material to be identified is completed through the combined optical character recognition detection frame.

[0155] Optionally, determining the recognition area of ​​the optical character recognition detection box based on the recognition area selection box and its coordinate information includes:

[0156] When the center point of the optical character recognition detection frame is located within the recognition area selection frame;

[0157] or,

[0158] When any point of the optical character recognition detection frame is set within the recognition area selection frame, and the height of the overlapping area between the optical character recognition detection frame and the recognition area selection frame is greater than half of the height of the optical character recognition detection frame.

[0159] Optionally, the determination of the target reference area information and its selection box includes:

[0160] The selection criteria of the target reference area information include: the width of the text interval of the target reference area information is smaller than the width of the text; the text of the target reference area information does not cross rows or columns; the text of the target reference area information does not appear repeatedly in the standard material image;

[0161] The setting standard of the selection box of the target reference area information includes: setting it close to the target reference area information, and all the target reference area information needs to be selected.

[0162] Optionally, the method further includes: entering the extracted material into an information database based on the dictionary relationship.

[0163] The functions of the device in the embodiment of the present invention have been described in the above method embodiment, so for details not provided in the description of this embodiment, please refer to the relevant description in the above embodiment, and no further description will be given here.

[0164] Based on the same inventive concept, an embodiment of this specification also provides an electronic device.

[0165] The electronic device embodiment of the present invention is described below, and the electronic device can be regarded as a specific physical implementation of the method and device embodiments of the present invention described above. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0166] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 4 The electronic device 300 according to the embodiment of the present invention is described. Figure 4 The electronic device 300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0167] like Figure 4 As shown, the electronic device 300 is in the form of a general computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including the storage unit 320 and the processing unit 310), a display unit 340, etc.

[0168] The storage unit stores program codes, which can be executed by the processing unit 310, so that the processing unit 310 performs the steps according to various exemplary embodiments of the present invention described in the above processing method section of this specification. For example, the processing unit 310 can perform the following steps: Figure 1 Steps shown.

[0169] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit 3202 , and may further include a read-only memory unit (ROM) 3203 .

[0170] The storage unit 320 may also include a program / utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0171] Bus 330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0172] The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable viewers to interact with the electronic device 300, and / or any device that enables the electronic device 300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 350. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 360. The network adapter 360 may communicate with other modules of the electronic device 300 through the bus 330. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0173] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: Figure 1 The method shown.

[0174] Figure 5 A schematic diagram of a computer-readable medium provided for an embodiment of this specification.

[0175] accomplish Figure 5 The computer program of the method shown can be stored on one or more computer readable media. The computer readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0176] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0177] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the viewer computing device, partially on the viewer device, as a stand-alone software package, partially on the viewer computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the viewer computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0178] In summary, the present invention can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that general data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0179] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0180] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0181] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A material information extraction method based on template alignment, characterized in that: include: Get material templates; The material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof; Perform full text recognition on the material to be identified to obtain text information of the material to be identified; Compare the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box; Determining a scaling ratio of the material to be identified based on a selection box of the reference area information and a selection box of the target reference area information; The material to be identified is aligned with the material template based on the scaling ratio of the material to be identified, thereby completing the material extraction of the material to be identified.

2. The material information extraction method based on template alignment according to claim 1, characterized in that: The obtaining of the material template comprises: Acquire a standard material image marked with target reference area information and its selection box, and target identification area selection box; Obtaining a target reference area information list based on the target reference area information and its selection box; Binding the keyword of the target identification area and the target identification area selection box to form a dictionary relationship; A material template is obtained based on the target reference area information list and the dictionary relationship.

3. The material information extraction method based on template alignment as claimed in claim 2, characterized in that: The determining the scaling ratio of the material to be identified based on the selection box of the reference area information and the selection box of the target reference area information includes: The material to be identified includes a plurality of selection boxes of reference area information, and the material template includes a plurality of selection boxes of target reference area information; Respectively acquiring coordinate information of the frame selection boxes of the plurality of reference area information and coordinate information of the frame selection boxes of the plurality of target reference area information; Determine, based on the coordinate information of the frame selection box of the reference area information, the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate; Determine a first minimum circumscribed rectangular frame of the material to be identified based on the coordinate information of the frame selection box of the reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the reference area information with the smallest vertical coordinate; Determine, based on the coordinate information of the frame selection box of the target reference area information, the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate; Determine a second minimum circumscribed rectangular frame of the material template based on the coordinate information of the frame selection box of the target reference area information with the largest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the smallest horizontal coordinate, the coordinate information of the frame selection box of the target reference area information with the largest vertical coordinate, and the coordinate information of the frame selection box of the target reference area information with the smallest vertical coordinate; The scaling ratio of the to-be-identified material is determined based on the first minimum circumscribed rectangular frame and the second minimum circumscribed rectangular frame.

4. The material information extraction method based on template alignment as claimed in claim 3, characterized in that: The step of aligning the material to be identified with the material template based on the scaling ratio of the material to be identified to complete the material extraction of the material to be identified includes: Calculate the distance values ​​from each target recognition area selection box to the four sides of the first minimum circumscribed rectangular box respectively; Determine the identification area selection box and coordinate information of the material to be identified based on the scaling ratio of the material to be identified and the distance value; Determine the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information, and merge the optical character recognition detection frames located in the same recognition area selection frame; The material extraction of the material to be identified is completed through the combined optical character recognition detection frame.

5. The material information extraction method based on template alignment according to claim 4, characterized in that: The determining of the recognition area of ​​the optical character recognition detection frame based on the recognition area selection frame and its coordinate information includes: When the center point of the optical character recognition detection frame is located within the recognition area selection frame; or, When any point of the optical character recognition detection frame is set within the recognition area selection frame, and the height of the overlapping area between the optical character recognition detection frame and the recognition area selection frame is greater than half of the height of the optical character recognition detection frame.

6. The material information extraction method based on template alignment according to claim 5, characterized in that: The determination of the target reference area information and its selection box includes: The selection criteria of the target reference area information include: the width of the text interval of the target reference area information is smaller than the width of the text; the text of the target reference area information does not cross rows or columns; the text of the target reference area information does not appear repeatedly in the standard material image; The setting standard of the selection box of the target reference area information includes: setting it close to the target reference area information, and all the target reference area information needs to be selected.

7. The material information extraction method based on template alignment according to claim 6, characterized in that: Also includes: The extracted materials are entered into the information base based on the dictionary relationship.

8. A material information extraction device based on template alignment, characterized in that: include: An acquisition module, used to acquire material templates; The material template includes a target reference area information list, and the target reference area information list includes target reference area information and a selection box thereof; A recognition module is used to perform full text recognition on the material to be recognized to obtain text information of the material to be recognized; A comparison module, used for comparing the text information of the material to be identified with the target reference area information list to obtain a reference area information list that meets the target similarity, wherein the reference area information list includes reference area information and its selection box; A determination module, configured to determine a scaling ratio of the material to be identified based on a selection box of the reference area information and a selection box of the target reference area information; An alignment module is used to align the material to be identified with the material template based on the scaling ratio of the material to be identified, so as to complete the material extraction of the material to be identified.

9. An electronic device, wherein: The electronic device includes: processor; and, A memory storing computer executable instructions which, when executed, cause the processor to perform a method according to any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.

Images