Document image field information matching method, device, equipment and readable storage medium

Abstract

The present invention relates to the field of image processing technology, and more specifically, to a document image field information matching method, device, equipment and readable storage medium, the method comprising obtaining document image information; preprocessing the document image information, inputting the preprocessed document image information into an image slice classification module and a feature coding module for classification processing and feature coding processing, respectively, to obtain image classification information and image feature coding information; sending the image classification information and image feature coding information to an information matching degree calculation model, matching different types of text information in the image classification information, and obtaining matching results of text information in the document image information. The present invention improves the accuracy of post-recognition through preprocessing, alleviates abnormal image interference, extracts image layout features, matches multiple features, and increases the matching efficiency of text information in documents.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a document image field information matching method, device, equipment and readable storage medium. Background Art

[0002] Due to the non-fixed layout and the hundreds or even thousands of document types, it becomes very time-consuming and laborious to make templates for column matching. In addition, how to manage and classify different types of templates also becomes a problem. Now we need a system that can perform high-precision column matching of information in document images in different types of templates, so as to reduce errors and save manpower and material resources. Summary of the invention

[0003] The purpose of the present invention is to provide a document image field information matching method, device, equipment and readable storage medium to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0004] In one aspect, the present application provides a document image field information matching method, the method comprising:

[0005] Acquire document image information, where the document image information is scanned image information of a paper document;

[0006] Preprocessing the document image information, inputting the preprocessed document image information into the image slice classification module and the feature coding module for classification processing and feature coding processing respectively, to obtain image classification information and image feature coding information;

[0007] The classification information of the image and the feature coding information of the image are sent to the information matching degree calculation model, and different types of text information in the classification information of the image are matched to obtain the matching result of the text information in the document image information.

[0008] Optionally, after acquiring the document image information, the method further includes:

[0009] Get historical document image information;

[0010] Sending the business images in the historical document image information and the abnormal images in the historical document image information to a classification training model for training to obtain a document image information classification model;

[0011] The document image information is sent to the document image information classification model for business classification and abnormality classification, and it is determined whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, so as to obtain a normal business image.

[0012] Optionally, the preprocessing of the document image information includes:

[0013] Deleting unnecessary background from the business image in the document image information to obtain first image information;

[0014] Performing tilt correction on the first image information according to a perspective transformation method to obtain second image information;

[0015] The second image information is processed by grayscale conversion and noise reduction methods to obtain preprocessed image information.

[0016] Optionally, the pre-processed document image information is input into the image slice classification module and the feature coding module for classification processing and feature coding processing respectively, to obtain the image classification information and the image feature coding information, including:

[0017] Sending the preprocessed document image information to the recognition and classification module for recognition and classification to obtain classification information of the image;

[0018] Sending the preprocessed document image information to a convolutional neural network model for layout extraction and encoding to obtain image layout encoding information;

[0019] The classification information of the image is sent to a position feature extraction module and a semantic feature extraction module for feature extraction and encoding to obtain position feature encoding information and semantic feature encoding information.

[0020] Optionally, sending the pre-processed document image information to a recognition and classification module for recognition and classification to obtain classification information of the image includes:

[0021] Inputting the preprocessed document image information into an OCR recognition module for OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes angular coordinate information of the text slice and text information of the text slice;

[0022] Determining a region where the text slice is located according to the angular coordinate information of the text slice;

[0023] The text slices are classified according to the character sequence information in the text information of the text slices to obtain classification information of the image, and the classification information of the image includes first image slice information and second image slice information.

[0024] Optionally, sending the classification information of the image to a position feature extraction module and a semantic feature extraction module for feature extraction and encoding includes:

[0025] Inputting the coordinate information of the first image slice in the classification information of the image and the coordinate information of the second image slice in the classification information of the image into a position feature module for processing to obtain a positional relationship between the first image slice and the second image slice;

[0026] Converting the positional relationship between the first image slice and the second image slice into positional feature coding information;

[0027] Inputting the classification information of the image into a semantic feature module, processing the classification information of the image through a bidirectional long short-term memory network, and obtaining semantic feature information of the second image slice information;

[0028] The semantic feature information of the second image slice information is converted to obtain semantic feature coding information.

[0029] Optionally, the matching of different types of text information in the classification information of the image to obtain a matching result of the text information in the document image information includes:

[0030] The semantic feature encoding information, the position feature encoding information and the layout feature encoding information are input into the fully connected layer for concatenation and random inactivation processing to obtain the feature processing result;

[0031] Calculate the matching degree between at least one first image slice information and at least one second slice information respectively on the feature processing result to obtain at least one calculation result;

[0032] The at least one calculation result is compared with a preset first threshold value to determine whether the at least one calculation result is greater than the preset first threshold value, and the matching result of the text information of the document image information is determined based on the judgment result of whether the calculation result is greater than the preset first threshold value.

[0033] In a second aspect, an embodiment of the present application provides a document image field information matching device, the device comprising:

[0034] A first acquisition unit, used to acquire document image information, wherein the document image information is scanned image information of a paper document;

[0035] A first processing unit is used to pre-process the document image information, and input the pre-processed document image information into the image slice classification module and the feature coding module for classification processing and feature coding processing respectively, so as to obtain image classification information and image feature coding information;

[0036] The first matching unit is used to send the classification information of the image and the feature coding information of the image to the information matching degree calculation model, match different types of text information in the classification information of the image, and obtain the matching result of the text information in the document image information.

[0037] Optionally, the device further comprises:

[0038] A second acquisition unit, used to acquire historical document image information;

[0039] A first training unit is used to send the business images in the historical document image information and the abnormal images in the historical document image information to the classification training model for training to obtain a document image information classification model;

[0040] The first judgment unit is used to send the document image information to the document image information classification model for business classification and abnormality classification, to judge whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, and to obtain a normal business image.

[0041] Optionally, the device comprises:

[0042] A first processing subunit is used to delete unnecessary background in the business image in the document image information to obtain first image information;

[0043] A second processing subunit is used to perform tilt correction on the first image information according to a perspective transformation method to obtain second image information;

[0044] The third processing subunit is used to process the second image information by grayscale conversion and noise reduction methods to obtain pre-processed image information.

[0045] Optionally, the device comprises:

[0046] A first classification subunit is used to send the pre-processed document image information to the recognition and classification module for recognition and classification to obtain classification information of the image;

[0047] A first sending subunit is used to send the preprocessed document image information to a convolutional neural network model for layout extraction and encoding to obtain image layout encoding information;

[0048] The first extraction subunit is used to send the classification information of the image to the position feature extraction module and the semantic feature extraction module for feature extraction and encoding to obtain position feature encoding information and semantic feature encoding information.

[0049] Optionally, the device comprises:

[0050] A first recognition subunit is used to input the pre-processed document image information into an OCR recognition module for OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes angular coordinate information of the text slice and text information of the text slice;

[0051] A fourth processing subunit, configured to determine a region where the text slice is located according to the angular coordinate information of the text slice;

[0052] The second classification subunit is used to classify the text slice according to the character sequence information in the text information of the text slice to obtain image classification information, wherein the image classification information includes first image slice information and second image slice information.

[0053] Optionally, the device comprises:

[0054] a fifth processing subunit, configured to input the coordinate information of the first image slice in the classification information of the image and the coordinate information of the second image slice in the classification information of the image into a position feature module for processing, so as to obtain a positional relationship between the first image slice and the second image slice;

[0055] A sixth processing subunit, configured to convert the positional relationship between the first image slice and the second image slice into positional feature coding information;

[0056] a seventh processing subunit, configured to input the classification information of the image into a semantic feature module, and process the classification information of the image through a bidirectional long short-term memory network to obtain semantic feature information of the second image slice information;

[0057] The eighth processing subunit is used to convert the semantic feature information of the second image slice information to obtain semantic feature coding information.

[0058] Optionally, the device comprises:

[0059] A ninth processing subunit, used for inputting the semantic feature coding information, the position feature coding information and the layout feature coding information into the fully connected layer for splicing processing and random inactivation processing to obtain a feature processing result;

[0060] A first calculation subunit is used to calculate the matching degree of at least one first image slice information and at least one second slice information respectively on the feature processing result to obtain at least one calculation result;

[0061] The first matching subunit is used to compare the at least one calculation result with a preset first threshold value, determine whether the at least one calculation result is greater than the preset first threshold value, and determine the matching result of the text information of the document image information according to the judgment result of whether the calculation result is greater than the preset first threshold value.

[0062] In a third aspect, an embodiment of the present application provides a document image field information matching device, the device comprising a memory and a processor. The memory is used to store a computer program; the processor is used to implement the steps of the document image field information matching method when executing the computer program.

[0063] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned document image field information matching method are implemented.

[0064] The beneficial effects of the present invention are:

[0065] The present invention takes the entire document image as input, improves the later recognition accuracy through preprocessing, alleviates the interference of abnormal images, and extracts image layout features through convolutional neural networks to increase matching efficiency. The present invention also uses a bidirectional long short-term memory network to encode the semantic features of sliced ​​text information, thereby achieving the effect of semantic matching. The present invention can process different templates and obtain the area where the valid column slices in the image are located through a layout feature extraction model; and the matching degree calculation model integrates various features, thereby achieving high-precision matching of the columns in the document image information and reducing the input of manpower and material resources of the staff.

[0066] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A schematic diagram of a flow chart of a document image field information matching method according to an embodiment of the present invention;

[0069] Figure 2It is a schematic diagram of the structure of a document image field information matching device according to an embodiment of the present invention;

[0070] Figure 3 It is a schematic diagram of the structure of a document image field information matching device described in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0072] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0073] like Figure 1 As shown, this embodiment provides a document image field information matching method, which includes step S1, step S2 and step S3.

[0074] Step S1, obtaining document image information, wherein the document image information is scanned image information of a paper document;

[0075] Step S2, preprocessing the document image information, and inputting the preprocessed document image information into the image slice classification module and the feature coding module for classification processing and feature coding processing respectively, to obtain image classification information and image feature coding information;

[0076] Step S3: Send the classification information of the image and the feature coding information of the image to the information matching degree calculation model, match different types of text information in the classification information of the image, and obtain the matching result of the text information in the document image information.

[0077] The present invention takes the entire document image as input, improves the later recognition accuracy through preprocessing, alleviates the interference of abnormal images, and extracts image layout features through convolutional neural networks to increase matching efficiency. The present invention also uses a bidirectional long short-term memory network to encode the semantic features of sliced ​​text information, thereby achieving the effect of semantic matching. The present invention can process different templates and obtain the area where the valid column slices in the image are located through a layout feature extraction model; and the matching degree calculation model integrates various features, thereby achieving high-precision matching of the columns in the document image information and reducing the input of manpower and material resources of the staff.

[0078] In a specific implementation of the present disclosure, step S1 includes step S4, step S5 and step S6.

[0079] Step S4, obtaining historical document image information;

[0080] Step S5, sending the business images in the historical document image information and the abnormal images in the historical document image information to the classification training model for training to obtain a document image information classification model;

[0081] Step S6: Send the document image information to the document image information classification model for business classification and abnormality classification, determine whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, and obtain a normal business image.

[0082] It can be understood that the present invention trains the classification training model through historical document image information to obtain an information classification model, and then eliminates business-irrelevant pictures and abnormal pictures, reduces subsequent calculations, and saves overall process time.

[0083] In a specific implementation of the present disclosure, step S2 includes step S21, step S22 and step S23.

[0084] Step S21, deleting unnecessary background in the business image in the document image information to obtain first image information;

[0085] Step S22: performing tilt correction on the first image information according to a perspective transformation method to obtain second image information;

[0086] Step S23: grayscale and reduce noise on the second image information to obtain pre-processed image information.

[0087] It can be understood that the present invention makes the business images in the document image information more uniform and regular by preprocessing the document image information, thereby improving the accuracy of subsequent recognition and prediction.

[0088] In a specific implementation of the present disclosure, step S2 includes step S24, step S25 and step S26.

[0089] Step S24, sending the pre-processed document image information to the recognition and classification module for recognition and classification to obtain classification information of the image;

[0090] Step S25, sending the preprocessed document image information to a convolutional neural network model for layout extraction and encoding to obtain image layout encoding information;

[0091] Step S26: Send the classification information of the image to the position feature extraction module and the semantic feature extraction module for feature extraction and encoding to obtain position feature encoding information and semantic feature encoding information.

[0092] It can be understood that the present invention improves the recognition accuracy of the image by extracting different features of the image, and the present invention integrates various features to achieve high-precision matching of fields in the document image information, thereby reducing the input of manpower and material resources of the staff.

[0093] In a specific implementation of the present disclosure, step S24 includes step S241, step S242 and step S243.

[0094] Step S241: input the pre-processed document image information into an OCR recognition module for OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes angular coordinate information of the text slice and text information of the text slice;

[0095] Step S242: determining the region where the text slice is located according to the angular coordinate information of the text slice;

[0096] Step S243: classify the text slice according to the character sequence information in the text information of the text slice to obtain image classification information, wherein the image classification information includes first image slice information and second image slice information.

[0097] It can be understood that the present invention uses an OCR recognition module to recognize and classify images, and divides the image information into two categories that need to be recognized according to the character sequence information, so as to classify the subsequent column matching, wherein the two categories of images are column header slice images and candidate slice images, the column header slice images are image slices containing labels such as name and time, and the candidate slices are image slices with name content and time content.

[0098] In a specific implementation of the present disclosure, step S24 includes step S244, step S245, step S246 and step S247.

[0099] Step S244: input the coordinate information of the first image slice in the classification information of the image and the coordinate information of the second image slice in the classification information of the image into a position feature module for processing to obtain a positional relationship between the first image slice and the second image slice;

[0100] Step S245, converting the positional relationship between the first image slice and the second image slice into positional feature coding information;

[0101] Step S246, inputting the classification information of the image into a semantic feature module, processing the classification information of the image through a bidirectional long short-term memory network, and obtaining semantic feature information of the second image slice information;

[0102] Step S247: converting the semantic feature information of the second image slice information to obtain semantic feature coding information.

[0103] It can be understood that the present invention reduces the probability of slice matching errors, increases matching efficiency and accuracy, and reduces costs by encoding multiple features of the first image slice and the second image slice.

[0104] In a specific implementation of the present disclosure, step S3 includes step S31, step S32 and step S33.

[0105] The semantic feature encoding information, the position feature encoding information and the layout feature encoding information are input into the fully connected layer for concatenation and random inactivation processing to obtain the feature processing result;

[0106] Calculate the matching degree between at least one first image slice information and at least one second slice information respectively on the feature processing result to obtain at least one calculation result;

[0107] The at least one calculation result is compared with a preset first threshold value to determine whether the at least one calculation result is greater than the preset first threshold value, and the matching result of the text information of the document image information is determined based on the judgment result of whether the calculation result is greater than the preset first threshold value.

[0108] It can be understood that the present invention concatenates and randomly inactivates the above-mentioned encoded information, sends the processed encoded information to the fully connected layer for processing, and then calculates the matching degree of two different types of slices, and then determines whether the text information in the two slices matches based on the matching degree.

[0109] It can be understood that the present invention prevents overfitting and increases matching accuracy and efficiency by splicing and randomly inactivating the coded information of multiple features. Moreover, the present invention does not require a template and can perform recognition and matching in different situations.

[0110] Example 2

[0111] like Figure 2 As shown, this embodiment provides a device for multiple people to edit code online, and the device includes a first acquisition unit 701, a first processing unit 702 and a first matching unit 703.

[0112] The first acquisition unit 701 is used to acquire document image information, where the document image information is scanned image information of a paper document;

[0113] The first processing unit 702 is used to pre-process the document image information, and input the pre-processed document image information into the image slice classification module and the feature coding module for classification processing and feature coding processing respectively, so as to obtain image classification information and image feature coding information;

[0114] The first matching unit 703 is used to send the classification information of the image and the feature coding information of the image to the information matching degree calculation model, match different types of text information in the classification information of the image, and obtain the matching result of the text information in the document image information.

[0115] In a specific embodiment of the present disclosure, the first acquisition unit 701 further includes a second acquisition unit 704, a first training unit 705 and a first judgment unit 706.

[0116] The second acquisition unit 704 is used to acquire historical document image information;

[0117] The first training unit 705 is used to send the business image in the historical document image information and the abnormal image in the historical document image information to the classification training model for training to obtain a document image information classification model;

[0118] The first judgment unit 706 is used to send the document image information to the document image information classification model for business classification and abnormality classification, to judge whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, and to obtain a normal business image.

[0119] In a specific implementation of the present disclosure, the first processing unit 702 includes a first processing sub-unit 7021 , a second processing sub-unit 7022 , and a third processing sub-unit 7023 .

[0120] The first processing subunit 7021 is used to delete unnecessary background in the business image in the document image information to obtain first image information;

[0121] The second processing subunit 7022 is used to perform tilt correction on the first image information according to a perspective transformation method to obtain second image information;

[0122] The third processing subunit 7023 is used to process the second image information by grayscale conversion and noise reduction methods to obtain pre-processed image information.

[0123] In a specific implementation of the present disclosure, the first processing unit 702 further includes a first classification subunit 7024 , a first sending subunit 7025 and a first extraction subunit 7026 .

[0124] The first classification subunit 7024 is used to send the pre-processed document image information to the recognition and classification module for recognition and classification to obtain classification information of the image;

[0125] The first sending subunit 7025 is used to send the pre-processed document image information to the convolutional neural network model for layout extraction and encoding to obtain image layout encoding information;

[0126] The first extraction subunit 7026 is used to send the classification information of the image to the position feature extraction module and the semantic feature extraction module for feature extraction and encoding to obtain position feature encoding information and semantic feature encoding information.

[0127] In a specific embodiment of the present disclosure, the first classification subunit 7024 includes a first identification subunit 70241 , a fourth processing subunit 70242 and a second classification subunit 70243 .

[0128] The first recognition subunit 70241 is used to input the pre-processed document image information into the OCR recognition module for OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes the angular coordinate information of the text slice and the text information of the text slice;

[0129] The fourth processing subunit 70242 is used to determine the region where the text slice is located according to the angular coordinate information of the text slice;

[0130] The second classification subunit 70243 is used to classify the text slice according to the character sequence information in the text information of the text slice to obtain image classification information, and the image classification information includes first image slice information and second image slice information.

[0131] In a specific embodiment of the present disclosure, the first classification subunit 7024 also includes a fifth processing subunit 70244 , a sixth processing subunit 70245 , a seventh processing subunit 70246 and an eighth processing subunit 70247 .

[0132] a fifth processing subunit 70244, configured to input the coordinate information of the first image slice in the classification information of the image and the coordinate information of the second image slice in the classification information of the image into a position feature module for processing, so as to obtain a positional relationship between the first image slice and the second image slice;

[0133] A sixth processing subunit 70245 is configured to convert the positional relationship between the first image slice and the second image slice into positional feature coding information;

[0134] The seventh processing subunit 70246 is used to input the classification information of the image into the semantic feature module, and process the classification information of the image through a bidirectional long short-term memory network to obtain semantic feature information of the second image slice information;

[0135] The eighth processing subunit 70247 is used to transform the semantic feature information of the second image slice information to obtain semantic feature coding information.

[0136] In a specific embodiment of the present disclosure, the first matching unit 703 includes a ninth processing subunit 7031, a first calculation subunit 7032 and a first matching subunit 7033. The ninth processing subunit 7031 is used to input the semantic feature encoding information, the position feature encoding information and the layout feature encoding information into the fully connected layer for splicing processing and random inactivation processing to obtain a feature processing result;

[0137] A first calculation subunit 7032 is used to calculate the matching degree between at least one first image slice information and at least one second slice information respectively on the feature processing result to obtain at least one calculation result;

[0138] The first matching subunit 7033 is used to compare the at least one calculation result with a preset first threshold value, determine whether the at least one calculation result is greater than the preset first threshold value, and determine the matching result of the text information of the document image information based on the judgment result of whether the calculation result is greater than the preset first threshold value.

[0139] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] Example 3

[0141] Corresponding to the above method embodiment, the embodiment of the present disclosure further provides a document image field information matching device. The document image field information matching device described below and the document image field information matching method described above can be referred to each other.

[0142] Figure 3 FIG. 8 is a block diagram of a document image field information matching device 800 according to an exemplary embodiment. Figure 3 As shown, the document image field information matching device 800 may include: a processor 801, a memory 802. The document image field information matching device 800 may also include one or more of a multimedia component 803, an input / output (I / O) interface 804, and a communication component 805.

[0143] The processor 801 is used to control the overall operation of the document image field information matching device 800 to complete all or part of the steps in the document image field information matching method. The memory 802 is used to store various types of data to support the operation of the document image field information matching device 800, and these data may include, for example, instructions for any application or method operating on the document image field information matching device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the document image field information matching device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include: Wi-Fi module, Bluetooth module, NFC module.

[0144] In an exemplary embodiment, the document image field information matching device 800 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned document image field information matching method.

[0145] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the document image field information matching method described above are implemented. For example, the computer-readable storage medium may be the memory 802 including the program instructions, and the program instructions may be executed by the processor 801 of the document image field information matching device 800 to complete the document image field information matching method described above.

[0146] Example 4

[0147] Corresponding to the above method embodiment, the embodiment of the present disclosure further provides a readable storage medium. The readable storage medium described below and the document image field information matching method described above can be referred to each other.

[0148] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the document image field information matching method of the above method embodiment.

[0149] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0151] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A document image field information matching method, characterized in that: include: Acquire document image information, where the document image information is scanned image information of a paper document; Preprocessing the document image information to obtain preprocessed document image information; Using the recognition and classification module to recognize and classify the preprocessed document image information, including using the OCR recognition module to perform OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes the angular coordinate information of the text slice and the text information of the text slice; Classify the text slices according to the character sequence information in the text information of the text slices to obtain first image slice information and second image slice information; The first image slice information is a column header slice image, which includes an image slice with a name tag and a time tag; the second image slice information is a candidate slice image, which includes a name content and a time content; The convolutional neural network model is used to extract and encode the preprocessed document image information to obtain the image layout encoding information; Inputting the angular coordinate information of the first image slice information and the angular coordinate information of the second image slice information into a position feature module for processing to obtain a positional relationship between the first image slice and the second image slice; Converting the positional relationship between the first image slice and the second image slice into positional feature coding information; Inputting the second image slice information into a semantic feature module, processing the second image slice information through a bidirectional long short-term memory network, and obtaining semantic feature information of the second image slice information; Converting the semantic feature information of the second image slice information to obtain semantic feature coding information; The position feature coding information, semantic feature coding information and image format coding information are sent to the information matching degree calculation model, and different types of text information in the first image slice information and the second image slice information are matched to obtain the matching result of the text information in the document image information.

2. The document image field information matching method according to claim 1, characterized in that: After obtaining the document image information, the method further includes: Get historical document image information; Sending the business images in the historical document image information and the abnormal images in the historical document image information to a classification training model for training to obtain a document image information classification model; The document image information is sent to the document image information classification model for business classification and abnormality classification, and it is determined whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, so as to obtain a normal business image.

3. The document image field information matching method according to claim 1, characterized in that: The preprocessing of the document image information comprises: Deleting unnecessary background from the business image in the document image information to obtain first image information; Performing tilt correction on the first image information according to a perspective transformation method to obtain second image information; The second image information is processed by grayscale conversion and noise reduction methods to obtain preprocessed image information.

4. The document image field information matching method according to claim 1, characterized in that: The matching of different types of text information in the first image slice information and the second image slice information to obtain a matching result of the text information in the document image information includes: The semantic feature coding information, the position feature coding information and the image layout coding information are input into the fully connected layer for splicing and random inactivation processing to obtain the feature processing result; Calculate the matching degree between at least one first image slice information and at least one second image slice information respectively on the feature processing result to obtain at least one calculation result; The at least one calculation result is compared with a preset first threshold value to determine whether the at least one calculation result is greater than the preset first threshold value, and the matching result of the text information of the document image information is determined based on the judgment result of whether the calculation result is greater than the preset first threshold value.

5. A document image field information matching device, characterized in that: include: A first acquisition unit, used to acquire document image information, wherein the document image information is scanned image information of a paper document; A first processing unit, used for preprocessing the document image information to obtain preprocessed document image information; Using the recognition and classification module to recognize and classify the preprocessed document image information, including using the OCR recognition module to perform OCR recognition to obtain text slice information of the image, wherein the text slice information of the image includes the angular coordinate information of the text slice and the text information of the text slice; Classify the text slices according to the character sequence information in the text information of the text slices to obtain first image slice information and second image slice information; The first image slice information is a column header slice image, which includes an image slice with a name tag and a time tag; the second image slice information is a candidate slice image, which includes a name content and a time content; The convolutional neural network model is used to extract and encode the preprocessed document image information to obtain the image layout encoding information; The device comprises: A fifth processing subunit, configured to input the angular coordinate information of the first image slice information and the angular coordinate information of the second image slice information into a position feature module for processing to obtain a positional relationship between the first image slice and the second image slice; A sixth processing subunit, configured to convert the positional relationship between the first image slice and the second image slice into positional feature coding information; A seventh processing subunit, configured to input the second image slice information into a semantic feature module, and process the second image slice information through a bidirectional long short-term memory network to obtain semantic feature information of the second image slice information; An eighth processing subunit, configured to convert the semantic feature information of the second image slice information to obtain semantic feature coding information; The first matching unit is used to send the position feature coding information, the semantic feature coding information and the image format coding information to the information matching degree calculation model, match different types of text information in the first image slice information and the second image slice information, and obtain the matching result of the text information in the document image information.

6. The document image field information matching device according to claim 5, characterized in that: The device also includes: A second acquisition unit, used to acquire historical document image information; A first training unit is used to send the business images in the historical document image information and the abnormal images in the historical document image information to the classification training model for training to obtain a document image information classification model; The first judgment unit is used to send the document image information to the document image information classification model for business classification and abnormality classification, to judge whether the image of the document image information is a business image that needs to be processed and whether the image of the document image information is abnormal, and to obtain a normal business image.

7. The document image field information matching device according to claim 5, characterized in that: The device comprises: A first processing subunit is used to delete unnecessary background in the business image in the document image information to obtain first image information; A second processing subunit is used to perform tilt correction on the first image information according to a perspective transformation method to obtain second image information; The third processing subunit is used to process the second image information by grayscale conversion and noise reduction methods to obtain pre-processed image information.

8. The document image field information matching device according to claim 5, characterized in that: The device comprises: A ninth processing subunit, used for inputting the semantic feature coding information, the position feature coding information and the image layout coding information into the fully connected layer for splicing processing and random inactivation processing to obtain a feature processing result; A first calculation subunit is used to calculate the matching degree of at least one first image slice information and at least one second image slice information respectively on the feature processing result to obtain at least one calculation result; The first matching subunit is used to compare the at least one calculation result with a preset first threshold value, determine whether the at least one calculation result is greater than the preset first threshold value, and determine the matching result of the text information of the document image information according to the judgment result of whether the calculation result is greater than the preset first threshold value.

9. A document image field information matching device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the document image field information matching method as claimed in any one of claims 1 to 4 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the document image field information matching method as claimed in any one of claims 1 to 4 are implemented.

Images