An invoice information positioning and reading method based on image recognition

Abstract

The application discloses an invoice information positioning reading method based on image recognition and relates to the technical field of machine vision, which comprises the following steps: dividing a structured attribute vector according to the business entity attention preference of each enterprise department known business type, obtaining a business entity and invoice image structured attribute vector interaction correlation network diagram of each business type known by the enterprise department, verifying the invoice image structured attribute vector interaction prior knowledge of the business entity, establishing a cross-modal attention dynamic area positioning mechanism, generating a business entity attention heat map of each business type known by the enterprise department, analyzing the contribution degree of a real-time uploaded invoice image relative to the business entity attention heat map, constructing a business entity adaptive reading decision model, and generating business entity real-time uploaded invoice information positioning reading data known by the enterprise department of each business type. The application has the beneficial effect of significantly improving the processing efficiency and accuracy in a multi-department complex invoice scene.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method for locating and reading invoice information based on image recognition. Background Technology

[0002] Existing invoice information location and reading solutions employ fixed template matching methods, heavily relying on preset coordinate rules. This makes them unable to adapt to differences in invoice formats and the diverse information needs of the same invoice across different departments, resulting in poor generalization ability and high maintenance costs. Secondly, even when deep learning is introduced for full-field recognition, it merely outputs all information indiscriminately, lacking intelligent focusing and filtering of key information in specific business scenarios, leading to redundant and low-practicality output results. The entire recognition process acts as an unexplainable black box, with opaque decision-making logic, making it impossible to verify the rationality of the location results. It also lacks the ability to continuously self-optimize from processing feedback, making it difficult to meet enterprises' needs for accurate, efficient, and auditable intelligent reimbursement. Summary of the Invention

[0003] To address the aforementioned technical problems, an image recognition-based method for locating and reading invoice information is provided. This technical solution resolves the problems described above.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for locating and reading invoice information based on image recognition, comprising:

[0006] S1. Based on the historical invoice image database of each known business type of the enterprise department, quantify the structured attribute vector of the historical invoice image data, divide the structured attribute vector according to the business entity attention preference of each known business type of the enterprise department, and obtain the interaction and association network diagram between the business entity of each known business type of the enterprise department and the structured attribute vector of the invoice image.

[0007] S2. Verify the prior knowledge of the interaction between the structured attribute vectors of the business entities and the invoice images in the network graph of the business entities of each business type known to the enterprise department. Construct the business entity demand encoder and the general feature encoder of the invoice for each business type. Build a cross-modal attention dynamic region localization mechanism and generate the attention heatmap of the business entities of each business type known to the enterprise department.

[0008] S3. Obtain the real-time uploaded invoice image and the attention heatmap of business entities of various business types known to the enterprise department, analyze the contribution of the real-time uploaded invoice image to the attention heatmap of business entities, construct an adaptive reading decision model for business entities, and generate real-time uploaded invoice information location reading data of business entities of various business types known to the enterprise department.

[0009] Preferably, step S1 specifically includes:

[0010] Based on the historical invoice image database of various business types known to the enterprise department, all text fields in the historical invoice image data are extracted using OCR text recognition, and then substituted into the pre-trained BERT natural language model for embedding and transformation to obtain the semantic feature vector of the historical invoice images of various business types known to the enterprise department.

[0011] Based on the historical invoice image database of various business types known to the enterprise department, NER named entity recognition is used to extract all text predefined fields in the historical invoice image data and construct an existence vector of key fields of historical invoice images of various business types known to the enterprise department.

[0012] Based on the historical invoice image database of various business types known to the enterprise department, regular expressions are used to extract all numerical type parameters in the historical invoice image data and construct them into numerical statistical feature vectors of historical invoice images of various business types known to the enterprise department.

[0013] Based on the historical invoice image database of various business types known to the enterprise department, the text block region of each invoice image in the historical invoice image data is extracted using a pre-trained YOLO object detection model. A bounding box is generated for each text block and grayscale binarization is performed. The center coordinates, width and height of each text block bounding box are marked, and the text block bounding box is assigned an ID number. The bounding boxes are combined into a layout feature vector of the historical invoice images of various business types known to the enterprise department.

[0014] Based on the semantic feature vectors, key field existence vectors, numerical statistical feature vectors, and layout feature vectors of historical invoice images for each business type known to the enterprise department, a structured attribute vector of historical invoice image data is constructed by concatenating arrays in the database.

[0015] Based on the structured attribute vectors of historical invoice image data, the existence vectors of key fields belonging to the business entities of historical invoices are marked. Using the K-Means clustering algorithm, each known business type of the enterprise department is used as the initial cluster. The cosine similarity between the existence vector of key fields belonging to the business entities of historical invoices and each initial cluster of each known business type of the enterprise department is calculated. The structured attribute vectors of historical invoice image data are then divided to obtain the structured attribute vectors of invoice images of business entities of each known business type of the enterprise department.

[0016] Preferably, step S1 further includes:

[0017] Based on the structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned about, an undirected graph of structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned about is established, with each business type in the enterprise department as the parent node, each business entity in each business type in the enterprise department as the child node, and the structured attribute vectors of invoice images that business entities are concerned about as invoice attribute nodes.

[0018] Based on the undirected graph of the structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned with, TF-IDF term frequency-inverse document frequency is used to count the frequency of occurrence of attribute nodes under child nodes in parent nodes. According to the frequency of occurrence of attribute nodes relative to the inverse document frequency of parent nodes, the edge weights of attribute nodes under child nodes of business entities in the parent nodes of various business types in the enterprise department are assigned, thus obtaining the interaction and association network graph between business entities of various business types in the enterprise department and the structured attribute vectors of invoice images.

[0019] Preferably, step S2 specifically includes:

[0020] Based on the interaction network graph of the structured attribute vectors of business entities and invoice images of various business types known to the enterprise department, the layout feature vectors of business entities and invoice images are extracted. Using the Gaussian prior distribution, the average coordinates and coordinate variance of the layout feature vectors of the attributes under the child nodes of the business entities in the parent nodes of various business types known to the enterprise department are statistically calculated and weighted by the edge weights of the attribute nodes under the child nodes of the business entities in the parent nodes of various business types known to the enterprise department. This yields the prior attention distribution map of the structured attribute vectors of business entities and invoice images of various business types known to the enterprise department.

[0021] Based on the FNN feedforward neural network, a business entity demand encoder for each business type is established. The structured attribute vectors of the invoice images of the business entities of each business type known to the enterprise department are used as the basis for the interaction and association network. The structured attribute vectors of the invoice images of the business entities of each business type are used as the basis for the vector transformation. The ReLU activation function is then input to output the activation values ​​of the structured attribute vectors of the business entities of each business type and the invoice images. A linear transformation is then performed to obtain the business entity demand query vector for each business type.

[0022] Based on the VGG convolutional neural network, a general feature encoder for invoices is constructed. The structured attribute vectors of invoice images of business entities of various business types known to the enterprise departments are used as input. Several convolutional and pooling layers are used for processing to extract the three-dimensional visual feature map of the structured attribute vectors of invoice images of business entities of various business types, so as to obtain the general feature vectors of invoice images of business entities of various business types.

[0023] The common feature vectors of invoice images that business entities of various business types focus on are projected onto the same dimension space as the query vectors of business entities of various business types through linear mapping.

[0024] Preferably, step S2 further includes:

[0025] Based on the scaling dot product attention mechanism, the business entity demand encoder of each business type is used as the query, and the invoice general feature encoder is used as the key and value. The similarity between the general feature vectors of the invoice images of each business type under the given business entity demand query vector of each business type is measured. The attention score between the business entity demand query vector of each business type and the general feature vector of the invoice images of each business type is obtained. The attention score is then normalized to a probability distribution by the Softmax function to obtain the attention weight between the business entity demand query vector of each business type and the general feature vector of the invoice images of each business type.

[0026] The attention distribution map between the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type is obtained by weighted summing of the attention scores and attention weights between the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type.

[0027] Using KL divergence to establish a loss function, we verify the minimum difference between the attention distribution map between the business entity demand query vector and the general feature vector of the invoice image that the business entity pays attention to for each business type and the prior attention distribution map of the structured attribute vector of the business entity and the invoice image known to the enterprise department for each business type. This generates an attention heatmap of the business entity known to the enterprise department for each business type.

[0028] Preferably, step S3 specifically includes:

[0029] Based on real-time uploaded invoice images, the general feature encoder of invoices is used to obtain the general feature vector of real-time invoice images that business entities of various business types are concerned about.

[0030] Based on the common feature vector of real-time invoice image attention for business entities of various business types, and following the TreeSHAP algorithm, the independent contribution value of the common feature vector of real-time invoice image attention for business entities relative to the business entity demand query vector of each business type in the known business entity attention heatmap of each business type in the enterprise department is independently verified, and the association weight of the independent contribution value of the common feature vector of real-time invoice image attention for each business type relative to the business entity demand query vector of each business type is obtained.

[0031] Preferably, step S3 further includes:

[0032] Based on random forest, the root node is the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type. The branch node is the business entity demand query vector of each business type. The leaf node is the real-time uploaded invoice information of the business entity known to the enterprise department for each business type. The general feature vector of the invoice image that the business entity pays attention to in each business type is used as the input. The threshold for dividing the branch node is based on maximizing information gain. A business entity adaptive reading decision model is established.

[0033] Based on the association weights of the independent contribution values ​​of the general feature vectors of real-time invoice images of business entities of various business types with the query vectors of business entities of various business types, a TreeSHAP interpreter is established. The interpreter takes the data of real-time invoice information location and reading of business entities of various business types known to the enterprise department as input, and generates the association weights of the independent contribution values ​​of real-time invoice information location and reading of business entities of various business types known to the enterprise department with the query vectors of business entities of various business types.

[0034] Based on the independent contribution values ​​of the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department relative to the demand query vectors of business entities of various business types, the association weights are updated to maximize the information gain of the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department relative to the demand query vectors of business entities of various business types in the adaptive retrieval decision model, thereby generating the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention proposes an image recognition-based invoice information location and reading scheme. By constructing a business knowledge-guided cross-modal attention mechanism and interpretable-driven closed-loop optimization, it achieves on-demand accurate location and reading, and can adaptively extract key invoice information according to the business preferences of different departments, completely eliminating the dependence on fixed templates. It establishes a fully interpretable decision-making mechanism, from feature contribution to reading decision path, which is completely transparent and verifiable. Finally, it forms a self-evolving learning capability, continuously optimizing the location accuracy and generalization ability by continuously analyzing the contribution feedback in actual processing, significantly improving the processing efficiency and accuracy in complex invoice scenarios involving multiple departments. Attached Figure Description

[0037] Figure 1 This is a flowchart of an image recognition-based method for locating and reading invoice information. Detailed Implementation

[0038] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0039] Reference Figure 1 As shown, a method for locating and reading invoice information based on image recognition includes:

[0040] S1. Based on the historical invoice image database of each known business type of the enterprise department, quantify the structured attribute vector of the historical invoice image data, divide the structured attribute vector according to the business entity attention preference of each known business type of the enterprise department, and obtain the interaction and association network diagram between the business entity of each known business type of the enterprise department and the structured attribute vector of the invoice image.

[0041] Step S1 specifically includes:

[0042] Based on the historical invoice image database of various business types known to the enterprise department, all text fields in the historical invoice image data are extracted using OCR text recognition, and then substituted into the pre-trained BERT natural language model for embedding and transformation to obtain the semantic feature vector of the historical invoice images of various business types known to the enterprise department.

[0043] Based on the historical invoice image database of various business types known to the enterprise department, NER named entity recognition is used to extract all text predefined fields in the historical invoice image data and construct an existence vector of key fields of historical invoice images of various business types known to the enterprise department.

[0044] Based on the historical invoice image database of various business types known to the enterprise department, regular expressions are used to extract all numerical type parameters in the historical invoice image data and construct them into numerical statistical feature vectors of historical invoice images of various business types known to the enterprise department.

[0045] Based on the historical invoice image database of various business types known to the enterprise department, the text block region of each invoice image in the historical invoice image data is extracted using a pre-trained YOLO object detection model. A bounding box is generated for each text block and grayscale binarization is performed. The center coordinates, width and height of each text block bounding box are marked, and the text block bounding box is assigned an ID number. The bounding boxes are combined into a layout feature vector of the historical invoice images of various business types known to the enterprise department.

[0046] Based on the semantic feature vectors, key field existence vectors, numerical statistical feature vectors, and layout feature vectors of historical invoice images for each business type known to the enterprise department, a structured attribute vector of historical invoice image data is constructed by concatenating arrays in the database.

[0047] Based on the structured attribute vectors of historical invoice image data, the existence vectors of key fields belonging to the business entities of historical invoices are marked. Using the K-Means clustering algorithm, each known business type of the enterprise department is used as the initial cluster. The cosine similarity between the existence vector of key fields belonging to the business entities of historical invoices and each initial cluster of each known business type of the enterprise department is calculated. The structured attribute vectors of historical invoice image data are then divided to obtain the structured attribute vectors of invoice images of business entities of each known business type of the enterprise department.

[0048] Step S1 also includes:

[0049] Based on the structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned about, an undirected graph of structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned about is established, with each business type in the enterprise department as the parent node, each business entity in each business type in the enterprise department as the child node, and the structured attribute vectors of invoice images that business entities are concerned about as invoice attribute nodes.

[0050] Based on the undirected graph of the structured attribute vectors of invoice images that business entities of various business types in the enterprise department are concerned with, TF-IDF term frequency-inverse document frequency is used to count the frequency of occurrence of attribute nodes under child nodes in parent nodes. According to the frequency of occurrence of attribute nodes relative to the inverse document frequency of parent nodes, the edge weights of attribute nodes under child nodes of business entities in the parent nodes of various business types in the enterprise department are assigned, thus obtaining the interaction and association network graph between business entities of various business types in the enterprise department and the structured attribute vectors of invoice images.

[0051] When using it, please refer to the steps outlined above:

[0052] As a further step, unstructured historical invoice images are deeply quantized and constructed into an interactive network through multimodal feature engineering and graph-structured knowledge representation. Semantic features, key field existence, numerical statistics, and layout four-dimensional vectors are extracted from the invoices and concatenated into a complete structured attribute vector. Then, clustering is performed based on business entity preferences, and the TF-IDF mechanism is innovatively applied to calculate edge weights in the department-entity-attribute heterogeneous graph, thereby embedding business knowledge into the network structure. This generates a prior knowledge graph that can accurately quantify the attention preferences of different business entities for invoice attributes. This network graph not only provides interpretable and quantifiable decision-making basis for subsequent adaptive localization and reading but also endows the system with powerful generalization capabilities through embedded business semantics.

[0053] Furthermore, as an embodiment of step S1:

[0054] Large enterprises' innovation project departments build networks that connect their business entities.

[0055] 1. Data preparation and feature extraction:

[0056] Collect images of 5,000 historical invoices reimbursed over the past year.

[0057] Semantic Feature Vector: BERT embedding is performed on the invoice text to obtain a 768-dimensional vector. Example: XX Cloud Platform Service Fee - Project: AIGC2024 - Amount: 50000.

[0058] Existence vector of key fields: Fields (project number, cloud service fee, amount) are identified through NER and a one-hot vector is generated.

[0059] Numerical statistical feature vector: Extract and calculate the logarithm of numerical features (amount) to generate a feature vector.

[0060] Page layout feature vector: Use YOLO to detect text blocks and obtain center coordinates, width and height information.

[0061] 2. Vector concatenation and clustering:

[0062] The four vectors are concatenated into a long vector, which serves as the structured attribute vector for each invoice.

[0063] Using the K-Means algorithm, with each project team under the Innovation Project Department as the initial cluster center, the existence vectors of key fields on invoices were clustered. Ultimately, the cloud service invoices for the AIGC2024 project were accurately categorized into the AIGC project business entity cluster.

[0064] 3. Construct an interactive network diagram:

[0065] Parent node: Innovation Project Department.

[0066] Sub-nodes: AIGC project, blockchain project.

[0067] Attribute node: Extract high-frequency attributes (project number, cloud service fee, external technical service fee, amount) from invoices in the AIGC project.

[0068] Calculate TF-IDF edge weights: Weights are calculated based on the frequency (TF) of the item number within the AIGC project and its scarcity (IDF) in other projects. For example, the item number has a higher TF-IDF weight, while the amount has a lower weight.

[0069] Form a strong connection: Innovation Project Department -> AIGC Project -- (weight 0.95) --> Project Number.

[0070] S2. Verify the prior knowledge of the interaction between the structured attribute vectors of the business entities and the invoice images in the network graph of the business entities of each business type known to the enterprise department. Construct the business entity demand encoder and the general feature encoder of the invoice for each business type. Build a cross-modal attention dynamic region localization mechanism and generate the attention heatmap of the business entities of each business type known to the enterprise department.

[0071] Step S2 specifically includes:

[0072] Based on the interaction network graph of the structured attribute vectors of business entities and invoice images of various business types known to the enterprise department, the layout feature vectors of business entities and invoice images are extracted. Using the Gaussian prior distribution, the average coordinates and coordinate variance of the layout feature vectors of the attributes under the child nodes of the business entities in the parent nodes of various business types known to the enterprise department are statistically calculated and weighted by the edge weights of the attribute nodes under the child nodes of the business entities in the parent nodes of various business types known to the enterprise department. This yields the prior attention distribution map of the structured attribute vectors of business entities and invoice images of various business types known to the enterprise department.

[0073] Based on the FNN feedforward neural network, a business entity demand encoder for each business type is established. The structured attribute vectors of the invoice images of the business entities of each business type known to the enterprise department are used as the basis for the interaction and association network. The structured attribute vectors of the invoice images of the business entities of each business type are used as the basis for the vector transformation. The ReLU activation function is then input to output the activation values ​​of the structured attribute vectors of the business entities of each business type and the invoice images. A linear transformation is then performed to obtain the business entity demand query vector for each business type.

[0074] Based on the VGG convolutional neural network, a general feature encoder for invoices is constructed. The structured attribute vectors of invoice images of business entities of various business types known to the enterprise departments are used as input. Several convolutional and pooling layers are used for processing to extract the three-dimensional visual feature map of the structured attribute vectors of invoice images of business entities of various business types, so as to obtain the general feature vectors of invoice images of business entities of various business types.

[0075] The common feature vectors of invoice images that business entities of various business types focus on are projected onto the same dimension space as the query vectors of business entities of various business types through linear mapping.

[0076] Step S2 also includes:

[0077] Based on the scaling dot product attention mechanism, the business entity demand encoder of each business type is used as the query, and the invoice general feature encoder is used as the key and value. The similarity between the general feature vectors of the invoice images of each business type under the given business entity demand query vector of each business type is measured. The attention score between the business entity demand query vector of each business type and the general feature vector of the invoice images of each business type is obtained. The attention score is then normalized to a probability distribution by the Softmax function to obtain the attention weight between the business entity demand query vector of each business type and the general feature vector of the invoice images of each business type.

[0078] The attention distribution map between the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type is obtained by weighted summing of the attention scores and attention weights between the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type.

[0079] Using KL divergence to establish a loss function, we verify the minimum difference between the attention distribution map between the business entity demand query vector and the general feature vector of the invoice image that the business entity pays attention to for each business type and the prior attention distribution map of the structured attribute vector of the business entity and the invoice image known to the enterprise department for each business type. This generates an attention heatmap of the business entity known to the enterprise department for each business type.

[0080] When using it, please refer to the steps outlined above:

[0081] As a further step, the layout and TF-IDF weights are extracted from the network graph of S1, and a prior attention distribution map is generated through Gaussian distribution modeling. Then, the business entity requirements are vectorized using an FNN encoder, and visual features of invoices are extracted using a VGG encoder. Finally, scaled dot product attention is used to match requirements with features, and KL divergence loss is introduced to force the data-driven attention distribution to converge towards prior knowledge, thereby generating an optimized business entity attention heatmap. By using historical layout priors as supervisory signals, the accuracy and robustness of attention region localization are significantly improved, semantic ambiguity interference is effectively avoided, and the model convergence speed is greatly accelerated. Simultaneously, a quantifiable and verifiable attention generation process is created, making the overall solution both highly accurate and highly stable.

[0082] Furthermore, as an embodiment of step S2:

[0083] Build a heatmap of the business entity attention for the company's finance department.

[0084] 1. Generate a prior attention distribution map:

[0085] From the network diagram of S1, extract the layout vectors of the attributes of total price and tax, tax amount, and invoice date that are of interest to all entities under the Finance Department.

[0086] The average coordinates of the total price including tax on historical invoices are (0.85, 0.90), with a variance of (0.05, 0.03). The TF-IDF edge weights obtained from the network graph are 0.9.

[0087] Other attributes are then processed. Finally, the weighted Gaussian distributions of all attributes are superimposed to generate a prior attention map for the finance department, which is highlighted in the lower right corner of the invoice.

[0088] 2. Encoding and Attention Calculation:

[0089] Demand Encoding: Input the structured attribute vector of the Finance Department into the FNN encoder, and output a 256-dimensional demand query vector of the Finance Department.

[0090] Feature encoding: Input a new travel invoice image into the VGG encoder and output a general visual feature map of [512,14,14].

[0091] Cross-modal attention: Using the finance department's demand query vector as the query and the above feature map as the key / value, a preliminary data attention map is generated by scaling dot product attention calculation. This map has a response in multiple amount field areas of the invoice.

[0092] 3. Optimization and output of knowledge guidance:

[0093] Calculate the KL divergence between the preliminary data attention map and the prior attention map of the finance department.

[0094] By optimizing network parameters through backpropagation, the data attention map is forced to shrink and focus its highlighted area toward the lower right corner of the invoice under the influence of KL divergence loss. This ensures that the final attention heatmap generated by the finance department accurately highlights the lower right corner area where the key fields of total price and tax and tax amount are located, providing reliable guidance for the subsequent accurate positioning and reading by S3.

[0095] S3. Obtain the real-time uploaded invoice image and the attention heatmap of business entities of various business types known to the enterprise department, analyze the contribution of the real-time uploaded invoice image to the attention heatmap of business entities, construct an adaptive reading decision model for business entities, and generate real-time uploaded invoice information location reading data of business entities of various business types known to the enterprise department.

[0096] Step S3 specifically includes:

[0097] Based on real-time uploaded invoice images, the general feature encoder of invoices is used to obtain the general feature vector of real-time invoice images that business entities of various business types are concerned about.

[0098] Based on the common feature vector of real-time invoice image attention for business entities of various business types, and following the TreeSHAP algorithm, the independent contribution value of the common feature vector of real-time invoice image attention for business entities relative to the business entity demand query vector of each business type in the known business entity attention heatmap of each business type in the enterprise department is independently verified, and the association weight of the independent contribution value of the common feature vector of real-time invoice image attention for each business type relative to the business entity demand query vector of each business type is obtained.

[0099] Step S3 also includes:

[0100] Based on random forest, the root node is the business entity demand query vector of each business type and the general feature vector of the invoice image that the business entity pays attention to in each business type. The branch node is the business entity demand query vector of each business type. The leaf node is the real-time uploaded invoice information of the business entity known to the enterprise department for each business type. The general feature vector of the invoice image that the business entity pays attention to in each business type is used as the input. The threshold for dividing the branch node is based on maximizing information gain. A business entity adaptive reading decision model is established.

[0101] Based on the association weights of the independent contribution values ​​of the general feature vectors of real-time invoice images of business entities of various business types with the query vectors of business entities of various business types, a TreeSHAP interpreter is established. The interpreter takes the data of real-time invoice information location and reading of business entities of various business types known to the enterprise department as input, and generates the association weights of the independent contribution values ​​of real-time invoice information location and reading of business entities of various business types known to the enterprise department with the query vectors of business entities of various business types.

[0102] Based on the independent contribution values ​​of the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department relative to the demand query vectors of business entities of various business types, the association weights are updated to maximize the information gain of the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department relative to the demand query vectors of business entities of various business types in the adaptive retrieval decision model, thereby generating the real-time invoice information upload and retrieval data of business entities of various business types known to the enterprise department.

[0103] When using it, please refer to the steps outlined above:

[0104] As a further step, the independent contribution values ​​of real-time invoice feature vectors to the attention heatmap of business entities are quantified using the TreeSHAP algorithm to generate association weights. Then, a random forest decision model is established to map invoice features to location-based data retrieval. Finally, the decision results are innovatively analyzed using the TreeSHAP interpreter and weighted with independent contribution weights to update the information gain criterion for random forest node splitting, achieving model self-optimization. By transforming interpretable contribution analysis into model optimization signals, not only is the process of each retrieval decision completely transparent and verifiable, but the system also achieves continuously improving accuracy and personalized retrieval capabilities during ongoing use, while significantly enhancing the robustness of generalization processing for new invoice styles.

[0105] Furthermore, as an embodiment of step S3:

[0106] Process a new software purchase invoice for the Legal Department.

[0107] 1. Contribution Measurement and Initial Decision:

[0108] The new invoice image is input into the VGG encoder trained by S2 to obtain the general feature vector of the invoice.

[0109] TreeSHAP Independent Analysis: For this invoice, TreeSHAP analysis shows that the contract number dimension in the feature vector contributes the most to the generation of the legal department's attention heatmap (0.50), followed by the software name (0.30), while the amount contributes very little (0.05). This set of data represents the independent contribution value association weight.

[0110] Random Forest Decision: Simultaneously, the invoice feature vector is input into the legal department's random forest model. Based on the current internal information gain rules, the model ultimately outputs three fields: contract number, software name, and supplier.

[0111] 2. Explanation-driven closed-loop optimization:

[0112] TreeSHAP Explains the Decision: Now, let's use the TreeSHAP interpreter to analyze the decision from the random forest. The interpreter finds that in this decision path, the contract number node contributes +0.45 to the final output, the software name contributes +0.25, and the supplier contributes +0.10.

[0113] Contribution weighting: These contributions are weighted and adjusted using the independent weights (0.50, 0.30, 0.05) obtained in the first step, which further strengthens the importance of contract number and software name and weakens the importance of supplier.

[0114] Updating the decision model: The system uses this batch of weighted contribution data to initiate an incremental update of the random forest model. Specifically, the splitting criteria for model nodes are adjusted so that, under similar characteristics, the model assigns higher information gain weights to contract numbers and software names. This means that in the future, when the legal department's model encounters new invoices, it will more effectively make splitting decisions based on features related to contract numbers and software names, outputting more accurate location and retrieval data.

[0115] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. An image recognition-based invoice information positioning reading method, characterized by, Comprise: S1, based on the enterprise department known each business type of historical invoice image database, quantification historical invoice image data structured attribute vector, according to each enterprise department known business type of business entity attention preference for structured attribute vector is divided, get enterprise department known each business type of business entity and invoice image structured attribute vector interaction correlation network diagram; S2, verify the invoice image structured attribute vector interaction priori knowledge of business entity in the business entity and invoice image structured attribute vector interaction correlation network diagram known by the enterprise department of each business type, construct each business type of business entity demand encoder and invoice general feature encoder, form cross-modal attention dynamic region positioning mechanism, generate enterprise department known each business type of business entity attention heat map; S3, get real-time upload invoice image and enterprise department known each business type of business entity attention heat map, analyze the contribution degree of real-time upload invoice image relative to business entity attention heat map, build business entity adaptive reading decision model, generate enterprise department known each business type of business entity real-time upload invoice information positioning reading data, including: Based on random forest, each business type of business entity demand query vector and each business type of business entity attention invoice image general feature vector as root node, each business type of business entity demand query vector as branch and leaf node, enterprise department known each business type of business entity real-time upload invoice information positioning reading data as leaf node, each business type of business entity attention invoice image general feature vector as input, according to the maximum information gain as branch and leaf node division threshold, establish business entity adaptive reading decision model.

2. The invoice information positioning reading method based on image recognition according to claim 1, characterized in that, Step S1 specifically includes: Based on the enterprise department known each business type of historical invoice image database, all text fields in historical invoice image data are extracted by using OCR text recognition, and are substituted into pre-trained BERT natural language model to carry out embedding conversion, to obtain enterprise department known each business type of historical invoice image semantic feature vector; Based on the enterprise department known each business type of historical invoice image database, all text predetermined fields in historical invoice image data are extracted by using NER named entity recognition, and are assembled into enterprise department known each business type of historical invoice image key field existence vector; Based on the enterprise department known each business type of historical invoice image database, all numerical type parameters in historical invoice image data are extracted by using regular expression, and are assembled into enterprise department known each business type of historical invoice image numerical statistical feature vector; Based on the historical invoice image database known by the enterprise department for each business type, the pre-trained YOLO target detection model is used to extract the text block area of each invoice image in the historical invoice image data, generate a boundary box for each text block, and perform gray-scale binary processing, mark the center coordinates, width and height of each text block boundary box, assign a text block boundary box number ID, and combine them into the historical invoice image layout feature vector known by the enterprise department for each business type; According to the historical invoice image semantic feature vector, the key field existence vector, the numerical statistical feature vector and the layout feature vector known by the enterprise department for each business type, the array splicing operation of the database is used to form the structured attribute vector of the historical invoice image data; According to the structured attribute vector of the historical invoice image data, the historical invoice attribution business entity key field existence vector is marked, the cosine similarity between the historical invoice attribution business entity key field existence vector and each initial clustering cluster of the enterprise department known business type is calculated according to the K-Means clustering algorithm, and the structured attribute vector of the business entity attention invoice image of the enterprise department known business type is obtained by dividing the structured attribute vector of the historical invoice image data.

3. The image recognition based invoice information positioning reading method according to claim 2, characterized in that, Step S1 further comprises: Based on the business entity attention invoice image structured attribute vector of the enterprise department known business type, the enterprise department known business type is taken as the parent node, the business entity of the enterprise department known business type is taken as the child node, and the business entity attention invoice image structured attribute vector is taken as the invoice attribute node. The business entity attention invoice image structured attribute vector undirected graph of the enterprise department known business type is established; According to the business entity attention invoice image structured attribute vector undirected graph of the enterprise department known business type, the TF-IDF word frequency-inverse document frequency is used to count the attribute node appearance frequency under the child node in the parent node, and the attribute node edge weight of the business entity child node under the parent node of the enterprise department known business type is assigned according to the inverse document appearance frequency of the attribute node appearance frequency relative to the parent node, to obtain the business entity and invoice image structured attribute vector interaction correlation network graph of the enterprise department known business type.

4. The image recognition based invoice information positioning reading method according to claim 3, characterized in that, Step S2 specifically comprises: Based on the business entity and invoice image structured attribute vector interaction correlation network graph of the enterprise department known business type, the layout feature vector in the business entity and invoice image structured attribute vector is extracted, and the average coordinates and coordinate variance of the layout feature vector in the attribute of the business entity child node in the parent node of the enterprise department known business type are weighted and summed with the attribute node edge weight of the business entity child node in the parent node of the enterprise department known business type by using the Gaussian distribution prior distribution, to obtain the business entity and invoice image structured attribute vector prior attention distribution graph of the enterprise department known business type. Based on the FNN feedforward neural network, the business entity demand encoder of each business type is established. The known business entity of each business type in the enterprise department interacts with the invoice image structured attribute vector in the known business entity attention invoice image structured attribute vector of each business type in the network graph. The known business entity attention invoice image structured attribute vector of each business type is substituted into the nonlinear transformation for vector conversion, and the ReLU activation function is input. The output of each business type business entity and invoice image structured attribute vector activation value is linearly transformed to obtain the business entity demand query vector of each business type. Based on the VGG convolutional neural network, the invoice general feature encoder is constructed, and the known business entity attention invoice image structured attribute vector of each business type in the enterprise department is used as input. The three-dimensional visual feature map of the business entity attention invoice image structured attribute vector is extracted by processing with several convolutional layers and pooling layers to obtain the general feature vector of the business entity attention invoice image of each business type. The general feature vector of the business entity attention invoice image of each business type is projected to the same dimension space of the business entity demand query vector of each business type through linear mapping.

5. The image recognition based invoice information positioning reading method according to claim 4, characterized in that, Step S2 further comprises: According to the scaled dot product attention mechanism, the business entity demand encoder of each business type is used as the query, and the invoice general feature encoder is used as the key and value. The similarity between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type is measured under the given business entity demand query vector of each business type to obtain the attention score between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type. The attention score is normalized to a probability distribution by inputting the Softmax function to obtain the attention weight between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type. Based on the attention score between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type, and the attention weight between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type, the weighted sum is obtained. The attention distribution graph between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type in the enterprise is obtained. The KL divergence is used to establish a loss function to verify the minimum difference between the attention distribution graph between the business entity demand query vector of each business type and the general feature vector of the business entity attention invoice image of each business type and the prior attention distribution graph between the known business entity of each business type in the enterprise department and the invoice image structured attribute vector. The attention heat map of the known business entity of each business type in the enterprise department is generated.

6. The image recognition based invoice information positioning reading method according to claim 5, wherein, Step S3 specifically comprises: Based on the real-time uploaded invoice image, the general feature vector of the business entity attention real-time invoice image of each business type is obtained by substituting the invoice general feature encoder. Based on the business entity attention real-time invoice image general feature vector of each business type, according to the TreeSHAP algorithm, the independent contribution value of the business entity attention real-time invoice image general feature vector of each business type to the business entity demand query vector in the enterprise department known attention heat map of each business type is verified independently, and the independent contribution value correlation weight of the business entity attention real-time invoice image general feature vector of each business type to the business entity demand query vector is obtained.

7. The image recognition based invoice information positioning reading method according to claim 6, characterized in that, Step S3 also includes: Based on the independent contribution value correlation weight of the business entity attention real-time invoice image general feature vector of each business type to the business entity demand query vector, a TreeSHAP interpreter is established, enterprise department known business entity real-time upload invoice information positioning reading data of each business type is taken as input, and the independent contribution value correlation weight of the enterprise department known business entity real-time upload invoice information positioning reading data of each business type to the business entity demand query vector is weighted; Based on the independent contribution value correlation weight of the enterprise department known business entity real-time upload invoice information positioning reading data of each business type to the business entity demand query vector, the maximum information gain of the enterprise department known business entity real-time upload invoice information positioning reading data of each business type to the business entity demand query vector in the business entity adaptive reading decision model is updated, and the enterprise department known business entity real-time upload invoice information positioning reading data of each business type is generated.

Images