Automatic processing method and device and electronic equipment

Abstract

The invention discloses an automatic processing method and device and electronic equipment, and relates to the field of data processing. The method comprises the following steps: acquiring an invoice image; performing feature extraction on the invoice image to obtain layout features corresponding to the invoice image; matching a corresponding target invoice template according to the layout characteristics; determining region segmentation parameters for the invoice image, and performing key information extraction on the invoice image according to the region segmentation parameters to obtain a plurality of key information regions; performing OCR (Optical Character Recognition) on each key information area to obtain an invoice information text corresponding to each key information area; if it is determined that the invoice information text is correct, key field extraction is conducted on the invoice information text, invoice keywords are obtained, and structured invoice information data are constructed according to the invoice keywords; and associating the invoice image with the invoice information data, and generating an invoice identification code. By implementing the technical scheme provided by the invention, the workload of manual input and verification is reduced, and the efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to an automated processing method, device and electronic equipment. Background Art

[0002] With the surge in economic activities and the increasing frequency of corporate transactions, invoice processing has become an indispensable part of daily business operations.

[0003] In order to solve the problem of invoice processing, the current relevant technologies generally adopt the method of manual review and input, supplemented by some automated tools to improve efficiency. These automated tools usually include optical character recognition (OCR) software with a single function, which can automatically recognize text data, but still requires a lot of manual intervention in the complete invoice processing process to ensure the accuracy and completeness of the data. However, when processing a large number of invoices, manual input of invoice information is time-consuming, resulting in low efficiency.

[0004] Therefore, there is an urgent need for an automated processing method, device and electronic equipment. Summary of the invention

[0005] The present application provides an automated processing method, device and electronic device, which reduces the workload of manual input and verification and improves efficiency.

[0006] In a first aspect of the present application, an automated processing method is provided, the method comprising: acquiring an invoice image, the invoice image being a digital image containing invoice information; performing feature extraction on the invoice image to obtain layout features corresponding to the invoice image; matching a corresponding target invoice template in a preset invoice template library according to the layout features; determining region segmentation parameters for the invoice image based on the target invoice template, and performing key information extraction on the invoice image according to the region segmentation parameters to obtain a plurality of key information regions; performing OCR recognition on each of the key information regions to obtain invoice information text corresponding to each of the key information regions; judging whether the invoice information text is correct according to preset semantic relevance and preset logical rules; if it is determined that the invoice information text is correct, performing key field extraction on the invoice information text to obtain invoice keywords, and constructing structured invoice information data according to the invoice keywords; associating the invoice image with the invoice information data, and generating an invoice identification code.

[0007] Optionally, the feature extraction of the invoice image to obtain the layout features corresponding to the invoice image specifically includes: dividing the invoice image into a plurality of uniform image blocks; extracting local texture features, local color features and local edge features for each of the image blocks; and constructing a feature topology map of the invoice image based on the spatial position relationship of each of the image blocks, the local texture features, the local color features and the local edge features as the layout features of the invoice image.

[0008] Optionally, matching the corresponding target invoice template in the preset invoice template library according to the layout features specifically includes: extracting the corresponding template layout features for each invoice template in the preset invoice template library; calculating the similarity between the layout features of the invoice image and the layout features of each template; sorting each similarity from large to small to obtain the maximum similarity, and using the invoice template corresponding to the maximum similarity as the target invoice template.

[0009] Optionally, based on the target invoice template, determining the regional segmentation parameters for the invoice image, and extracting key information from the invoice image according to the regional segmentation parameters to obtain multiple key information regions, specifically includes: mapping the key field boundaries pre-marked in the target invoice template onto the invoice image as an initial segmentation region; based on the initial segmentation region, performing regional expansion through a regional growing algorithm until the region is expanded to the invoice edge of the invoice image to obtain the regional segmentation parameters; and performing regional segmentation on the invoice image according to the regional segmentation parameters to obtain multiple key information regions.

[0010] Optionally, performing OCR recognition on each of the key information areas to obtain the invoice information text corresponding to each of the key information areas specifically includes: performing image preprocessing on each of the key information areas, the image preprocessing including denoising, binarization and correction; calling a preset OCR model to perform text recognition on each of the preprocessed key information areas to obtain the invoice information text corresponding to each of the key information areas.

[0011] Optionally, judging whether the invoice information text is correct based on preset semantic relevance and preset logical rules specifically includes: constructing a semantic network including the invoice information texts corresponding to each of the key information areas; in the semantic network, calculating the semantic similarity between each node to judge whether the preset semantic relevance is satisfied; matching each invoice information text with the preset logical rules to judge whether it complies with the preset logical rules; if each node in the semantic network satisfies the preset semantic relevance and each invoice information text complies with the preset logical rules, then determining that the invoice information text is correct.

[0012] Optionally, after associating the invoice image and the invoice information data and generating an invoice identification code, the method further includes: obtaining an invoice query request input by a user, the invoice query request including a target invoice identification code; determining the corresponding target invoice image and target invoice information data based on the target invoice identification code, and displaying the target invoice image and the target invoice information data to the user.

[0013] In a second aspect of the present application, an automated processing device is provided, the device comprising: an acquisition module and a processing module, wherein: the acquisition module is used to acquire an invoice image, the invoice image being a digital image containing invoice information; the processing module is used to perform feature extraction on the invoice image to obtain layout features corresponding to the invoice image; the processing module is also used to match the corresponding target invoice template in a preset invoice template library according to the layout features; the processing module is also used to determine a region segmentation parameter for the invoice image based on the target invoice template, and perform key information extraction on the invoice image according to the region segmentation parameter to obtain a plurality of key information regions; the processing module is also used to perform OCR recognition on each of the key information regions to obtain invoice information text corresponding to each of the key information regions; the processing module is also used to determine whether the invoice information text is correct according to preset semantic relevance and preset logical rules; the processing module is also used to perform key field extraction on the invoice information text to obtain invoice keywords if it is determined that the invoice information text is correct, and construct structured invoice information data according to the invoice keywords; the processing module is also used to associate the invoice image with the invoice information data and generate an invoice identification code.

[0014] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By acquiring the invoice image, performing feature extraction and template matching on the invoice image, determining the regional segmentation parameters and extracting the key information area, and then performing OCR recognition, semantic relevance and logical rule judgment on the key information area, finally obtaining structured invoice information data and associating it with the invoice image, and generating a unique invoice identification code. This automated and intelligent invoice processing method can efficiently and accurately complete the information extraction and structuring of the invoice, reduce the workload of manual entry and verification, and reduce the error rate. At the same time, through the association of invoice images and structured information, the management and query of invoices are facilitated, and the efficiency and availability of invoice processing are improved. The generation of invoice identification codes provides each invoice with a unique identification code, which is convenient for the identification, traceability and verification of invoices. The entire method realizes the automation and paperless processing of invoices, and improves the modernization level of financial management.

[0017] 2. By dividing the invoice image into uniform image blocks, extracting the local texture features, local color features and local edge features of the image blocks, and combining the spatial position relationship of the image blocks, the feature topology map of the invoice image is constructed as the layout feature. This multi-granularity, multi-feature layout feature extraction method can comprehensively and meticulously characterize the visual characteristics and spatial structure of the invoice image, and improve the expressiveness and distinguishability of the layout features. The feature topology map not only contains the local features of the image blocks, but also reflects the topological relationship between the image blocks.

[0018] 3. Pre-extract the template layout features of each invoice template in the preset invoice template library, calculate the similarity between the layout features of the invoice image and the template layout features, and determine the most matching target invoice template based on similarity sorting. This template matching method based on layout feature similarity transforms complex image matching into structured feature matching, simplifies the calculation process, and improves matching efficiency. At the same time, the best match is obtained through similarity sorting, which enhances the accuracy and reliability of matching. After finding a suitable target invoice template, the key field annotation information in the target invoice template can be used to guide the extraction of key information areas in a targeted manner, reduce the interference of invalid areas, and improve the accuracy of area extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of an automated processing method disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of an automated processing device disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0020] Explanation of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0021] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0022] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0023] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0024] This application provides an automated processing method, referring to Figure 1 , Figure 1 1 is a flow chart of an automated processing method provided in an embodiment of the present application. The method is applied to a server, which is a server that executes an automated processing program for invoices. The server may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center. The method includes steps S101 to S108, which are as follows: Step S101: Acquire an invoice image, which is a digital image containing invoice information.

[0025] In step S101, the server connects with other systems within the enterprise and automatically receives invoice images from these systems. For example, the server can establish a data channel with the enterprise's ERP system, financial management system, procurement system, etc. When these systems generate or receive new invoice images, they immediately transmit the invoice images to the server.

[0026] Step S102: extract features from the invoice image to obtain layout features corresponding to the invoice image.

[0027] In step S102, feature extraction is performed on the invoice image to obtain layout features corresponding to the invoice image, specifically including: dividing the invoice image into a plurality of uniform image blocks; extracting local texture features, local color features and local edge features from each image block; and constructing a feature topology map of the invoice image based on the spatial position relationship, local texture features, local color features and local edge features of each image block as the layout features of the invoice image.

[0028] Specifically, the server divides the invoice image into multiple uniform image blocks. The division method uses a fixed-size sliding window to traverse the image from left to right and from top to bottom with a certain step length, and each time captures an image block of a fixed size (such as 32x32 pixels). The size and step length of the image block can be dynamically adjusted according to parameters such as the overall size and resolution of the invoice image to balance the computational efficiency and feature granularity, which is not limited in this application.

[0029] Next, the server extracts multiple local features for each image block, including local texture features, local color features, and local edge features. Among them, texture features characterize the regularity and directionality of the grayscale distribution of pixels in the image block, and the server extracts local texture features using the local binary pattern algorithm. Color features characterize the color distribution and main color of pixels in the image block, and the server uses the color moment algorithm to extract local color features. Edge features characterize the gradient information of the grayscale changes of pixels in the image block, and the server uses the Canny edge algorithm to extract local edge features; the server uses the corresponding feature extraction algorithm to calculate the feature vector of each image block.

[0030] After extracting the local features of all image blocks, the server further analyzes the spatial position relationship between the image blocks and establishes a topological connection between the image blocks. Specifically, the server regards each image block as a node in the topological graph, and the spatial position of the image block in the invoice image (such as the coordinates of the upper left corner) corresponds to the geometric properties of the node. Then, the server calculates the feature similarity between the image blocks. For image block pairs whose similarity exceeds the preset similarity threshold, a topological edge is established between the corresponding nodes, and the weight of the edge is the similarity of the two image blocks. The similarity can be calculated using measurement functions such as Euclidean distance and cosine similarity.

[0031] Through the above steps, the server obtains a feature topology map that reflects the layout structure of the invoice image. In the feature topology map, nodes encode the local features of image blocks, and edges encode the relative positions and similarities between image blocks. The feature topology map combines the global layout information and local detail features of the invoice image, and is a high-level semantic representation of the original invoice image, which can be used for subsequent layout analysis, region segmentation and other tasks.

[0032] For example, for an A4-sized VAT invoice image, the server uses a 64x64 pixel sliding window with a step size of 16 pixels to divide the image into blocks, resulting in a total of 240 image blocks. Then, the server extracts 64-dimensional LBP texture features, 128-dimensional HSV color histogram features, and 32-dimensional Canny edge features for each image block, and concatenates these features into a 224-dimensional feature vector. Next, the server calculates the Euclidean distance between all image blocks, and for image block pairs with a distance less than 0.5, adds a topological edge with a weight equal to 1-distance between the corresponding nodes. Finally, the server obtains a feature topology map containing 240 nodes and 1,320 weighted edges, which serves as the layout feature of the invoice image for subsequent template matching and region segmentation steps.

[0033] Step S103: In the preset invoice template library, the corresponding target invoice template is matched according to the layout characteristics.

[0034] In step S103, for each invoice template in the preset invoice template library, the corresponding template layout features are extracted; the similarity between the layout features of the invoice image and the layout features of each template is calculated; the similarities are sorted from large to small to obtain the maximum similarity, and the invoice template corresponding to the maximum similarity is used as the target invoice template.

[0035] Specifically, the server matches the most similar target invoice template in the preset invoice template library according to the layout features of the extracted invoice image. The server pre-builds a preset invoice template library, which contains various common invoice templates, such as general VAT invoices, special invoices, fixed-amount invoices, etc. The invoice template can be obtained by manual annotation. Each invoice template has a unique number that records its layout structure, field location and other information. At the same time, each invoice template also corresponds to a template layout feature, which uses the same feature representation method as the invoice image in step S102, that is, the structured layout of the invoice template is encoded in the form of a feature topology map. The server uses the same image segmentation, local feature extraction and topology map construction method as the invoice image to be processed to calculate the template layout features of each invoice template and store them in the preset invoice template library.

[0036] In the matching stage, the server loads the layout features of the invoice image to be processed and the template layout features of all invoice templates. Next, the server calculates the similarity between the feature topology of the invoice image and the feature topology of each template. The similarity is calculated using a graph matching algorithm, which takes into account the similarity of node feature vectors and the consistency of topological structures. Commonly used graph matching algorithms include graph edit distance, maximum common subgraph, graph kernel, etc. Preferably, in the embodiment of the present application, the server uses a graph kernel algorithm, which calculates the number of matches between two graphs on various substructures to obtain a kernel function value that reflects the overall similarity.

[0037] After calculating the image core similarity between the invoice image and all invoice templates, the server obtains a similarity score list. The server sorts these scores from large to small and selects the target invoice template with the highest similarity as the matching result.

[0038] For example, the server loaded a VAT invoice image to be processed. After segmentation and feature extraction, the invoice image obtained a feature topology map containing 240 nodes. There are 5 invoice templates in the invoice template library, corresponding to VAT general invoice, special invoice, roll invoice, toll invoice and fixed-amount invoice, and the number of nodes in the feature topology map of each invoice template is between 220 and 260. The server uses the graph core algorithm to calculate the similarity between the invoice to be processed and the 5 invoice templates, and the scores obtained are 0.92, 0.34, 0.25, 0.13 and 0.08 respectively. Therefore, the first template with the highest similarity is selected as the matching result, that is, the target invoice template, which corresponds to the VAT general invoice.

[0039] Step S104: Based on the target invoice template, determine the region segmentation parameters for the invoice image, and extract key information from the invoice image according to the region segmentation parameters to obtain multiple key information regions.

[0040] In step S104, the key field boundaries pre-marked in the target invoice template are mapped to the invoice image as the initial segmentation area; based on the initial segmentation area, the region is expanded by a region growing algorithm until it is expanded to the invoice edge of the invoice image to obtain region segmentation parameters; according to the region segmentation parameters, the invoice image is segmented into regions to obtain multiple key information regions.

[0041] Specifically, the server determines the region segmentation parameters for the current invoice image based on the matched target invoice template, and extracts key information from the invoice image according to these parameters to obtain multiple key information regions. The purpose of this step is to accurately locate and segment the region containing key information in the invoice image, such as the invoice number, invoice date, purchaser name, etc., to prepare for subsequent text recognition. Specifically, the server loads the target invoice template that matches the current invoice image from the preset invoice template library. The position and boundaries of each key field are pre-marked in the target invoice template, such as the boundary of "invoice code" is "upper left (50, 50), lower right (200, 80)", etc. These marking information is obtained in advance based on the statistical analysis of a large number of similar invoices, reflecting the typical distribution law of key fields on this type of invoice. The server maps these boundaries to the current invoice image to form a set of initial segmented regions. It should be noted that since the size, rotation, alignment, etc. of each invoice image may be slightly different from the target invoice template, directly applying the target invoice template may cause problems such as offset or omission of the boundary.

[0042] In order to overcome the template matching error, the server uses the region growing algorithm to adaptively adjust the boundaries of the key fields based on the initial segmented regions. Specifically, the server uses the edge pixels of the initial region as seed points, and continuously merges adjacent pixels into the region where the seed points are located based on the similarity of features such as color and texture between pixels, so that the region expands toward the edge of the invoice image. During the region expansion process, the server sets preset growth rules and stop conditions, such as the upper limit of the number of pixels in the seed region, the rate of change of the region area, and the distance between the edge of the region and the edge of the invoice, to balance the integrity and accuracy of field positioning. When all initial regions meet the stop conditions, the expansion process ends, and the precise boundaries of the key fields on the invoice image are obtained.

[0043] After completing the region expansion, the server performs region segmentation on the entire invoice image. The server uses the four vertices of the invoice image as a reference and divides the invoice image into multiple rectangular regions, namely key information regions, according to the expanded field boundaries. Each key information region corresponds to a key information field. Since the parameters such as the position, size, and spacing of the field have been determined through template mapping and region expansion, the segmentation process can directly use the region segmentation parameters without the need for other algorithms.

[0044] For example, the server loaded the template of the VAT general invoice from the preset invoice template library, which recorded the boundary of the "invoice code" field as "upper left (50, 50), lower right (200, 100)" (coordinate units are pixels). The server mapped the boundary to the invoice image to be processed and found that due to the difference in invoice size, the coordinates of the lower right corner of the actual field were offset to (190, 110). So the server used the four sides of the initial area as the starting point, expanded outward using the region growing algorithm, and set the stop condition as "the region area growth rate is less than 1% for three consecutive times". After 60 iterations, the region expanded to the right edge of the invoice image and stopped. Finally, the precise boundary of the "invoice code" field was obtained as "upper left (50, 50), lower right (890, 110)". Similarly, the server also expanded the regions of other key fields on the invoice image, such as "invoice number" and "invoice date", forming 12 rectangular regions on the invoice image. The server divides these areas and removes three areas with an area less than 1,000 square pixels, ultimately obtaining nine key information areas, corresponding to the nine key fields on the invoice image. The images in these areas will be cut out and fed into the subsequent text recognition module.

[0045] Step S105: Perform OCR recognition on each key information area to obtain the invoice information text corresponding to each key information area.

[0046] In step S105, OCR recognition is performed on each key information area to obtain the invoice information text corresponding to each key information area, specifically including: image preprocessing is performed on each key information area, and the image preprocessing includes denoising, binarization and correction; calling the preset OCR model to perform text recognition on each key information area after preprocessing to obtain the invoice information text corresponding to each key information area.

[0047] Specifically, the server performs OCR (optical character recognition) processing on each key information area extracted in the previous step, converts the text content in the image into a text format that can be edited and processed by a computer, and obtains the invoice information text corresponding to each key information area.

[0048] First, the server performs image preprocessing on each key information area to improve the accuracy of OCR recognition. The preprocessing process includes denoising, binarization and correction steps. Among them, denoising is to eliminate noise interference in the image, such as salt and pepper noise, Gaussian noise, etc. The server uses median filtering, wavelet transform and other algorithms for denoising. Binarization is to convert the grayscale image into a black and white binary image to highlight the difference between the text strokes and the background. The server uses the global threshold method and the adaptive threshold method for binary extraction. Correction is to correct the tilt and deformation of the image to make the text arrangement more regular. The server uses Hough transform and block projection for correction.

[0049] After completing the image preprocessing, the server inputs the optimized key information area image into the preset OCR model for text recognition. The preset OCR model is an end-to-end text recognition system based on deep learning. Through training with a large number of annotated samples, it can directly recognize the text content in the image. The preset OCR model includes a convolutional neural network and a recurrent neural network, which are used to extract local features of the image and learn the contextual dependencies of the character sequence, respectively. The server pre-trains the preset OCR model on a large-scale invoice dataset.

[0050] During the recognition process, the server inputs each key information area into the preset OCR model, extracts features through the convolution layer and pooling layer, learns the sequence relationship between characters through the circulation layer, and finally outputs the recognition result, i.e. the invoice information text, through the decoder. At the same time, the server will also post-process the recognition results, such as removing special characters and correcting obvious typos, to further improve the text quality.

[0051] For example, for the key information area of ​​"invoice code", the server first pre-processes the corresponding image. The grid noise of the invoice background is removed by median filtering, and then the grayscale is converted to Figure 2 The image is converted into a black and white image, and finally the image is detected to have a 1.5° tilt through the Hough transform and corrected. The preprocessed "invoice code" image is input into the preset OCR model. After 12 convolutional layers and 2 recurrent layers, the preset OCR model outputs the recognition result with the highest confidence as "031001800104" with a confidence score of 0.95. The confidence threshold set by the server is 0.9, so the recognition result is considered valid and used as the text value of the "invoice code" field. Similarly, the server preprocesses and recognizes the regional images of other key fields in turn, and finally obtains the text values ​​of all fields, such as "invoice number: 12345678", "invoice date: April 1, 2023", "purchaser name: XXX company", etc. These key-value pairs of invoice information texts lay the foundation for subsequent information extraction and structuring.

[0052] Step S106: Determine whether the invoice information text is correct based on preset semantic relevance and preset logical rules.

[0053] In step S106, a semantic network including the invoice information text corresponding to each key information area is constructed; in the semantic network, the semantic similarity between each node is calculated to determine whether the preset semantic relevance is satisfied; each invoice information text is matched with the preset logical rules to determine whether it complies with the preset logical rules; if each node in the semantic network satisfies the preset semantic relevance and each invoice information text complies with the preset logical rules, it is determined that the invoice information text is correct.

[0054] Specifically, the preset semantic relevance includes the format rules of the invoice number, the time sequence of the invoice date, and the information matching degree of the seller and the buyer. The preset logical rules include the calculation relationship between the commodity amount and the total amount, and the corresponding relationship between the tax rate and the tax amount.

[0055] The server organizes the extracted invoice information text into a semantic network according to the field type. In this network, each node represents a key field, such as "invoice number" and "invoice date", and the attributes of the node include the text value and data type of the field. The server determines whether the invoice information text is correct based on the preset semantic relevance and preset logical rules, including: the server extracts key fields such as invoice number, invoice date, seller information, buyer information, product name, product amount, tax rate, tax amount and total amount from the invoice information text; the server determines whether the invoice number conforms to the preset format rules, and the format rules include character length, character type and character combination method; determines whether the invoice date meets the preset time sequence, ensuring that the invoice date is not earlier than the seller's business license registration date and not later than the current system time; calculates the text similarity between the seller information and the buyer information, and compares the text similarity with the preset threshold Perform a comparison to determine whether the matching degree of the seller's information and the buyer's information exceeds the matching threshold; calculate the tax-exclusive amount and tax amount of each product according to the product name and tax rate, and add up the total tax-exclusive amount and total tax amount; determine whether the total tax-exclusive amount is consistent with the total amount extracted from the invoice information text, and determine whether the total tax amount is consistent with the total tax amount extracted from the invoice information text; if the invoice number meets the format rules, and the invoice date meets the time sequence, and the matching degree of the seller's information and the buyer's information exceeds the matching threshold, and the total tax-exclusive amount is consistent with the total amount, and the total tax amount is consistent with the total tax amount, then the invoice information text is determined to be correct, otherwise it is determined that there is an error in the invoice information text.

[0056] Step S107: If it is determined that the invoice information text is correct, key fields are extracted from the invoice information text to obtain invoice keywords, and structured invoice information data is constructed based on the invoice keywords.

[0057] In step S107, the server further extracts key fields and performs structured processing on the invoice information text that has been confirmed to be correct, converting the unstructured text into structured invoice information data. The purpose of this step is to standardize and normalize the invoice information.

[0058] First, the server extracts key fields from the invoice information text. The server identifies key information with specific meanings and functions from the invoice information text, such as "invoice number", "issuing date", "seller name", "buyer name", "commodity name", "amount", "tax rate", "tax amount", etc. The server adopts a strategy that combines rule-based methods and model-based methods. The rule-based method searches for target fields in the text by formulating a series of pattern matching rules, such as keywords and regular expressions; the model-based method constructs a named entity recognition model to automatically learn the context features and extraction rules of the fields. The two methods complement each other to jointly improve the accuracy and recall rate of key field extraction.

[0059] For example, for the text fragment "Issuing date: April 1, 2023", the server first uses the regular expression "\d{4} year \d{1,2} month \d{1,2} day" to match the date string, and then determines the field type corresponding to the date according to the predefined keyword "issuing date". Another example is the fragment "Goods name: Office supplies". Although "Office supplies" does not have obvious pattern features, the named entity recognition model can understand that the content after the keyword "goods name" usually represents a specific commodity by learning a large number of similar text samples, and thus recognizes "Office supplies" as a complete commodity name entity. Through rule matching and model prediction, the server extracts a series of structured key fields and their values from the invoice information text, forming an invoice keyword set.

[0060] Next, the server constructs structured invoice information data based on the invoice keyword set. The server organizes and maps the key fields according to a predefined data model to form an invoice object with a clear structure and explicit semantics. The server has pre-designed an invoice data model that matches the business system. The invoice data model includes metadata information such as the name, data type, value range, and constraint conditions of each field. During the construction process, the server traverses the invoice keyword set, uses the name of each key field as the attribute name of the invoice data model, uses the corresponding field value as the attribute value, and performs conversion and verification according to the type and format defined by the invoice data model, and finally generates structured invoice information data.

[0061] Step S108: Associate the invoice image and the invoice information data, and generate an invoice identification code.

[0062] In step S108, the server associates the identified invoice image with the extracted structured invoice information data, establishes a mapping relationship between the two, and generates a unique invoice identification code.

[0063] First, the server establishes a correspondence between the invoice image and its corresponding structured invoice information data. After confirming the association between the invoice image and the invoice information data, the server generates a unified invoice identification code for them. The invoice identification code is a globally unique, unchangeable string used to identify and track the source, status, and life cycle of each invoice. The generation of the identification code follows certain coding rules and contains key information of the invoice, such as the invoice date, invoice type, serial number, etc., and also adds security mechanisms such as check digits to prevent the identification code from being forged or tampered with. The server generates the invoice identification code according to the coding rules based on the metadata of the current invoice, and stores it in association with the invoice image and invoice information data. At the same time, the server ensures that each invoice identification code is unique.

[0064] In a possible implementation, after step S108, the method further includes: obtaining an invoice query request input by a user, wherein the invoice query request includes a target invoice identification code; determining a corresponding target invoice image and target invoice information data according to the target invoice identification code, and displaying the target invoice image and target invoice information data to the user.

[0065] Specifically, the server provides a user interface for the user to enter the target invoice identification code to be queried. The user interface can be in various forms such as a web page, a mobile app, etc., as long as it can receive user input and send the request to the server. After the user fills in or scans the target invoice identification code of the invoice on the interface, an invoice query request is generated, which includes the target invoice identification code to be queried.

[0066] Next, after receiving the invoice query request from the user, the server parses the target invoice identification code from the request parameters. In the invoice query process, the server searches the associated index of the invoice image and the invoice information data according to the target invoice identification code. Since in step S108, the server has generated a unique invoice identification code for each invoice image and invoice information data, and established a mapping relationship between the invoice identification code and the image and invoice information data, the corresponding invoice information can be quickly located through the target invoice identification code. Finally, the server returns the target invoice image and target invoice information data corresponding to the acquired target invoice identification code to the user interface for display on the interface.

[0067] Reference Figure 2The present application also provides an automated processing device, which is a server, and the server includes an acquisition module 201 and a processing module 202; wherein: the acquisition module 201 is used to acquire an invoice image, which is a digital image containing invoice information; the processing module 202 is used to extract features from the invoice image to obtain layout features corresponding to the invoice image; the processing module 202 is also used to match the corresponding target invoice template in a preset invoice template library according to the layout features; the processing module 202 is also used to determine the region segmentation parameters for the invoice image based on the target invoice template, and perform region segmentation on the invoice image according to the region segmentation parameters. Key information is extracted to obtain multiple key information areas; processing module 202 is also used to perform OCR recognition on each key information area to obtain the invoice information text corresponding to each key information area; processing module 202 is also used to determine whether the invoice information text is correct based on preset semantic relevance and preset logical rules; processing module 202 is also used to extract key fields from the invoice information text if it is determined that the invoice information text is correct, obtain invoice keywords, and construct structured invoice information data based on the invoice keywords; processing module 202 is also used to associate the invoice image with the invoice information data and generate an invoice identification code.

[0068] In a possible implementation, the processing module 202 performs feature extraction on the invoice image to obtain layout features corresponding to the invoice image, specifically including: the processing module 202 divides the invoice image into multiple uniform image blocks; the processing module 202 extracts local texture features, local color features and local edge features from each image block; the processing module 202 constructs a feature topology map of the invoice image as the layout features of the invoice image based on the spatial position relationship, local texture features, local color features and local edge features of each image block.

[0069] In a possible implementation, the processing module 202 matches the corresponding target invoice template in the preset invoice template library according to the layout features, specifically including: the processing module 202 extracts the corresponding template layout features for each invoice template in the preset invoice template library; the processing module 202 calculates the similarity between the layout features of the invoice image and the layout features of each template; the processing module 202 sorts each similarity from large to small to obtain the maximum similarity, and uses the invoice template corresponding to the maximum similarity as the target invoice template.

[0070] In a possible implementation, the processing module 202 determines the regional segmentation parameters for the invoice image based on the target invoice template, and extracts key information from the invoice image according to the regional segmentation parameters to obtain multiple key information regions, specifically including: the processing module 202 maps the key field boundaries pre-marked in the target invoice template to the invoice image as the initial segmentation region; the processing module 202 expands the region through a regional growing algorithm based on the initial segmentation region until it expands to the invoice edge of the invoice image to obtain regional segmentation parameters; and the invoice image is segmented according to the regional segmentation parameters to obtain multiple key information regions.

[0071] In a possible implementation, the processing module 202 performs OCR recognition on each key information area to obtain the invoice information text corresponding to each key information area, specifically including: the processing module 202 performs image preprocessing on each key information area, and the image preprocessing includes denoising, binarization and correction; the processing module 202 calls a preset OCR model to perform text recognition on each key information area after preprocessing, and obtains the invoice information text corresponding to each key information area.

[0072] In a possible implementation, the processing module 202 determines whether the invoice information text is correct based on preset semantic relevance and preset logical rules, specifically including: the processing module 202 constructs a semantic network including the invoice information text corresponding to each key information area; the processing module 202 calculates the semantic similarity between each node in the semantic network to determine whether the preset semantic relevance is satisfied; the processing module 202 matches each invoice information text with the preset logical rules to determine whether it complies with the preset logical rules; the processing module 202 determines that the invoice information text is correct if each node in the semantic network satisfies the preset semantic relevance and each invoice information text complies with the preset logical rules.

[0073] In a possible implementation, after the processing module 202 associates the invoice image and the invoice information data and generates an invoice identification code, the method further includes: the acquisition module 201 acquires the invoice query request input by the user, and the invoice query request includes the target invoice identification code; the processing module 202 determines the corresponding target invoice image and target invoice information data according to the target invoice identification code, and displays the target invoice image and target invoice information data to the user.

[0074] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0075] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0076] The communication bus 302 is used to realize the connection and communication between these components.

[0077] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0078] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0079] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0080] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of an automated processing method.

[0081] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call an application program storing an automated processing method in the memory 305, and when executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0082] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0083] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0088] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0089] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An automated processing method, characterized in that: The method comprises: Acquire an invoice image, wherein the invoice image is a digital image containing invoice information; Extracting features of the invoice image to obtain layout features corresponding to the invoice image; In the preset invoice template library, matching the corresponding target invoice template according to the layout feature; Based on the target invoice template, determining a region segmentation parameter for the invoice image, and extracting key information from the invoice image according to the region segmentation parameter to obtain a plurality of key information regions; Perform OCR recognition on each of the key information areas to obtain the invoice information text corresponding to each of the key information areas; Determine whether the invoice information text is correct based on preset semantic relevance and preset logical rules; If it is determined that the invoice information text is correct, extract key fields from the invoice information text to obtain invoice keywords, and construct structured invoice information data based on the invoice keywords; The invoice image is associated with the invoice information data, and an invoice identification code is generated.

2. The method according to claim 1, characterized in that The extracting features of the invoice image to obtain the layout features corresponding to the invoice image specifically includes: Dividing the invoice image into a plurality of uniform image blocks; Extracting local texture features, local color features and local edge features from each of the image blocks; Based on the spatial position relationship of each of the image blocks, the local texture features, the local color features, and the local edge features, a feature topology map of the invoice image is constructed as a layout feature of the invoice image.

3. The method according to claim 1, characterized in that The matching of the corresponding target invoice template in the preset invoice template library according to the layout feature specifically includes: For each invoice template in the preset invoice template library, extract the corresponding template layout features; Calculating the similarity between the layout features of the invoice image and the layout features of each template; The similarities are sorted from large to small to obtain the maximum similarity, and the invoice template corresponding to the maximum similarity is used as the target invoice template.

4. The method according to claim 1, characterized in that: The method of determining a region segmentation parameter for the invoice image based on the target invoice template, and extracting key information from the invoice image according to the region segmentation parameter to obtain a plurality of key information regions specifically includes: Mapping the key field boundaries pre-marked in the target invoice template onto the invoice image as an initial segmentation area; Based on the initial segmented region, the region is expanded by using a region growing algorithm until it is expanded to the invoice edge of the invoice image, thereby obtaining the region segmentation parameter; The invoice image is segmented into regions according to the region segmentation parameters to obtain a plurality of key information regions.

5. The method according to claim 1, characterized in that The performing OCR recognition on each of the key information areas to obtain the invoice information text corresponding to each of the key information areas specifically includes: Performing image preprocessing on each of the key information areas, wherein the image preprocessing includes denoising, binarization and correction; The preset OCR model is called to perform text recognition on each key information area after preprocessing to obtain the invoice information text corresponding to each key information area.

6. The method according to claim 1, characterized in that The determining whether the invoice information text is correct according to the preset semantic relevance and preset logical rules specifically includes: Constructing a semantic network including the invoice information text corresponding to each of the key information areas; In the semantic network, the semantic similarity between each node is calculated to determine whether the preset semantic relevance is satisfied; Match each invoice information text with the preset logical rules to determine whether it complies with the preset logical rules; If each node in the semantic network satisfies the preset semantic relevance, and each invoice information text complies with the preset logical rules, then it is determined that the invoice information text is correct.

7. The method according to claim 1, characterized in that After associating the invoice image with the invoice information data and generating an invoice identification code, the method further includes: Acquire an invoice query request input by a user, wherein the invoice query request includes a target invoice identification code; According to the target invoice identification code, the corresponding target invoice image and target invoice information data are determined, and the target invoice image and the target invoice information data are displayed to the user.

8. An automated processing device, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire an invoice image, wherein the invoice image is a digital image containing invoice information; The processing module (202) is used to extract features from the invoice image to obtain layout features corresponding to the invoice image; The processing module (202) is further used to match the corresponding target invoice template in the preset invoice template library according to the layout feature; The processing module (202) is further used to determine a region segmentation parameter for the invoice image based on the target invoice template, and extract key information from the invoice image according to the region segmentation parameter to obtain a plurality of key information regions; The processing module (202) is further used to perform OCR recognition on each of the key information areas to obtain the invoice information text corresponding to each of the key information areas; The processing module (202) is further used to determine whether the invoice information text is correct according to preset semantic relevance and preset logical rules; The processing module (202) is further configured to extract key fields from the invoice information text to obtain invoice keywords if it is determined that the invoice information text is correct, and construct structured invoice information data based on the invoice keywords; The processing module (202) is also used to associate the invoice image with the invoice information data and generate an invoice identification code.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images