Automatic office method and device based on artificial intelligence, terminal equipment and storage medium

Abstract

The invention provides an automatic office method and device based on artificial intelligence, terminal equipment and a storage medium, and the method comprises the steps: inputting an image of a to-be-recognized bill into a pre-constructed neural network model for bill information extraction, and obtaining keyword information; wherein the neural network model is obtained by training based on a CNN neural network and a Transform neural network in combination with a plurality of different types of bill images and keyword information corresponding to each bill image; performing similarity matching in a pre-constructed voucher template database according to the keyword information, calling a corresponding voucher template according to a similarity matching result, and generating an initial voucher based on the voucher template; and verifying the initial voucher based on preset verification, calling a corresponding voucher output rule according to a verification result, and generating a target voucher of the to-be-identified bill based on the voucher output rule and the initial voucher. According to the invention, the bill processing flexibility and intelligence can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an automated office method, apparatus, terminal device, and storage medium based on artificial intelligence. Background Technology

[0002] Invoices are original documents generated in economic activities, carrying key financial information such as the transaction parties, amounts, tax rates, and dates. They are also the direct basis for financial accounting, tax declaration, and cost accounting. The accuracy of their processing directly affects a company's financial and tax compliance, tax risk management, and operational efficiency. Companies handle multiple types of invoices daily, each with different rules for extracting key information. Accounting firms or medium to large enterprises often need to process invoices from multiple clients / branches simultaneously, resulting in massive amounts of data. Invoice classification, information entry, and voucher generation all rely on manual operations by accounting personnel, leading to a large workload of repetitive tasks.

[0003] The current common method for processing original invoices still involves accounting staff manually entering the invoice information into accounting software, and then preparing accounting vouchers based on the entered data. However, manual entry is slow, lacks batch processing capabilities, and struggles to handle the demands of processing massive amounts of data. Furthermore, manual entry is prone to errors due to fatigue, resulting in erroneous vouchers that require significant time for subsequent verification. The current invoice processing method is highly dependent on manual labor; therefore, achieving intelligent invoice processing is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] The present invention aims to provide an automated office method, device, terminal equipment and storage medium based on artificial intelligence to solve the above-mentioned technical problems and improve the flexibility and intelligence of document processing.

[0005] To address the aforementioned technical problems, this invention provides an automated office method based on artificial intelligence, comprising:

[0006] The image of the ticket to be identified is input into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network, combined with multiple different types of ticket images and the keyword information corresponding to each ticket image;

[0007] Based on the keyword information, a similarity match is performed in a pre-built voucher template database. The corresponding voucher template is called based on the similarity match result, and an initial voucher is generated based on the voucher template. The voucher template database contains various types of voucher templates.

[0008] The initial voucher is validated based on a preset validation, and the corresponding voucher output rule is invoked according to the validation result. The target voucher for the document to be identified is generated based on the voucher output rule and the initial voucher.

[0009] The above solution utilizes a neural network model combining CNN and Transformer to quickly process invoice images for identification. CNN excels at capturing local image features, while Transformer handles long-range dependencies. The combination allows the model to efficiently and accurately extract keyword information from images, significantly improving processing speed compared to manually examining and entering information from each invoice. This is particularly suitable for scenarios involving large volumes of invoices, such as accounting firms or large enterprises. Initial invoices are generated by calling voucher templates through similarity matching in a voucher template database, avoiding the tedious process of manual voucher preparation by accountants. This allows for the processing of multiple invoices and the generation of corresponding vouchers in a short time, automating the process from invoice information extraction to voucher generation. The neural network model is trained on multiple different types of invoice images, adapting to various common invoice styles. The voucher template database also contains various types of voucher templates, enabling the generation of corresponding vouchers based on different invoice information, meeting the diverse business needs of enterprises.

[0010] In one implementation, the step of inputting the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information specifically includes:

[0011] The image of the ticket to be identified is input into the CNN neural network, so that the CNN neural network extracts the first ticket information from the image features of the received image and outputs the first spatial features; wherein, the first spatial features are used to define the bounding box where the text lines in the image of the ticket to be identified are located;

[0012] The first spatial feature and the image of the ticket to be identified are input into the OCR engine, so that the OCR engine performs text recognition on the image of the ticket to be identified and outputs the first text feature; wherein, the first text feature is text information carrying spatial location;

[0013] The first text feature is input into a Transformer neural network, which performs semantic analysis on the first text feature and outputs a second text feature; wherein the second text feature is used to characterize the logical relationship between the fields in the first text feature;

[0014] The keyword information is obtained by fusing the first text feature and the second text feature.

[0015] In the above scheme, a CNN neural network is used to extract the first piece of information from the image of the invoice to be recognized, outputting a first spatial feature to define the bounding box of the text lines. This accurately locates the text region, providing a clear target for subsequent processing and improving efficiency and accuracy. The first spatial feature and the invoice image are input into an OCR engine, and combined with the positioning information, text recognition is performed. The output is a first text feature carrying the spatial location, reducing interference and improving recognition accuracy. A Transformer neural network is used to perform semantic analysis on the first text feature, outputting a second text feature that reveals the logical relationship between fields, providing a deeper understanding of the business logic behind the text. Finally, the first and second text features are merged to obtain keyword information, achieving an organic combination of text content and semantic logic, making the information more complete and accurate. Moreover, this scheme is adaptable to various invoice types and has good scalability and versatility.

[0016] In one implementation, the step of enabling the CNN neural network to extract image features from the received image and output a first spatial feature specifically includes:

[0017] The image is convolved with a preset size based on the backbone network, and several first feature maps of different sizes are output and input into the feature pyramid network.

[0018] Based on the feature pyramid network, the first feature map is sampled at multiple scales, and a multi-scale feature pyramid is output and input into the region proposal network.

[0019] Based on the region proposal network, convolution and bounding box prediction are performed at each level of the multi-scale feature pyramid to output a first candidate region; wherein, the first candidate region is a region in the multi-scale feature pyramid that may contain a table or text.

[0020] The multi-scale feature pyramid and the first candidate region are aligned based on linear interpolation to output the first spatial feature.

[0021] In the above scheme, the backbone network is used to perform convolution on the image at a preset size to output first feature maps of different sizes, which can fully extract rich features of the image. The first feature map is sampled at multiple scales by a feature pyramid network to obtain a multi-scale feature pyramid, which can take into account the information of targets of different sizes and enhance the model's ability to recognize tables or text at different scales. The region proposal network outputs the first candidate region through convolution and bounding box prediction at each level of the multi-scale feature pyramid, accurately locating regions that may contain tables or text. Finally, linear interpolation is used to align the multi-scale feature pyramid and the first candidate region to output the first spatial feature, realizing effective matching between features and candidate regions, improving the accuracy and efficiency of text line bounding box localization in the ticket image, and laying a good foundation for subsequent information extraction.

[0022] In the above scheme, keyword information is first segmented and a term matrix is ​​constructed, which can structure the text information and facilitate subsequent processing. The term matrix is ​​transformed into a semantic vector of a preset dimension using a preset semantic model, which can mine the semantic features of keyword information and improve the accuracy of classification. A classifier is constructed with preset multi-level classification items such as bill type, tax type, industry type, and amount type, and the output of the previous classifier is used as the input of the next classifier, realizing progressively refined and hierarchical classification, making the classification more logical and systematic. Finally, multi-level classification results are generated based on the output results of each classifier, which provides a more accurate and detailed classification basis for subsequent similarity matching in the voucher template database, which can effectively improve the efficiency and accuracy of matching, thereby improving the quality of the entire bill information processing process.

[0023] In one implementation, the step of performing similarity matching in a pre-built voucher template database based on the keyword information, calling the corresponding voucher template based on the similarity matching result, and generating an initial voucher based on the voucher template specifically includes:

[0024] Calculate the TF-IDF value of each field in the keyword information, and normalize each TF-ID value to obtain a keyword matrix; wherein each element in the keyword matrix represents the TF-IDF vector value of a field;

[0025] Based on the keyword matrix value, a traversal query is performed in the voucher template database to calculate the cosine similarity between the keyword matrix value and each voucher template in the voucher template database; wherein, each voucher template is composed of several commonly used invoice keywords;

[0026] The voucher template with the highest cosine similarity is selected as the voucher template for the invoice to be identified, and the initial voucher is generated based on the voucher template.

[0027] In the above scheme, the keyword matrix is ​​obtained by calculating and normalizing the TF-IDF value of each field in the keyword information. This effectively measures the importance of fields in the document, highlights key information, and the normalization process makes the data comparable. Based on the keyword matrix, the voucher template database is traversed and the cosine similarity with each voucher template is calculated. Using cosine similarity to measure the degree of similarity between texts, the voucher template that best matches the keyword information of the invoice to be identified can be found quickly and accurately. Finally, the voucher template with the highest cosine similarity is selected to generate the initial voucher, ensuring a high degree of fit between the generated initial voucher and the invoice to be identified, improving the accuracy and efficiency of voucher generation, and laying a good foundation for subsequent financial processing and other work.

[0028] In one implementation, the step of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher specifically includes:

[0029] The initial voucher is first verified based on financial logic rules, and a first verification result is generated based on the verification result; wherein, the financial logic rules include debit and credit balance rules, tax rate logic rules, and amount consistency rules;

[0030] When the first verification result passes, a second verification is performed on the initial voucher based on business logic rules. When both the first and second verification results pass, the direct output rule is invoked to mark the initial voucher as verified and generate the target voucher. The business logic rules include: account matching rules, permission verification rules, and limit verification rules.

[0031] In the above scheme, the initial voucher is first verified based on financial logic rules including debit / credit balance rules, tax rate logic rules, and amount consistency rules. This ensures the accuracy of the voucher in basic financial logic from a financial professional perspective, guaranteeing the compliance and reliability of financial data. After the first verification passes, a second verification is performed based on business logic rules such as account matching rules, permission verification rules, and limit verification rules. This further reviews the voucher from a business perspective, ensuring that the voucher conforms to the actual business situation of the enterprise. Only when both verification results pass is the direct output rule invoked to mark the initial voucher as verified and generate the target voucher. This dual verification mechanism, with its multiple layers of checks, reduces the risk of erroneous vouchers entering the financial system, improves the quality and usability of the target voucher, and provides a solid guarantee for the enterprise's financial management.

[0032] In one implementation, the step of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher further includes:

[0033] A unique identifier is generated for the ticket to be identified by combining the timestamp and preset key elements;

[0034] The unique identification code is associated with and stored with the keyword information;

[0035] When either the first verification result or the second verification result fails, the "mark for review" principle is invoked to trigger a manual review prompt; wherein, the manual review prompt is to send the unique identification code and the reason for verification failure to a preset manual review queue;

[0036] After the reviewer performs manual verification and correction, the target voucher for the document to be identified is generated based on the corrected output principles and the corrected information.

[0037] In the above scheme, a unique identification code is generated for the document to be identified by combining the timestamp and preset key elements, and stored in association with keyword information. This facilitates accurate management and traceability of document information, ensuring data integrity and queryability. When the first or second verification fails, the "mark for review" principle is invoked to trigger a manual review prompt. The unique identification code and the reason for the verification failure are sent to the preset manual review queue, which can promptly and accurately locate the problematic document and notify the reviewer, improving the efficiency of problem handling. After the reviewer manually verifies and corrects the document, the target document is generated based on the corrected output principle and correction information. This ensures the rigor of document generation and allows for manual intervention to guarantee the accuracy of the document when problems occur in automated verification. This achieves an effective combination of automation and manual review, improving the reliability and quality of the entire document processing flow.

[0038] Secondly, this application also provides an automated office device based on artificial intelligence, including: an information extraction module, a voucher generation module, and a voucher verification module;

[0039] The information extraction module is used to input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network, combined with multiple different types of ticket images and the keyword information corresponding to each ticket image;

[0040] The voucher generation module is used to perform similarity matching in a pre-built voucher template database based on the keyword information, call the corresponding voucher template based on the similarity matching result, and generate an initial voucher based on the voucher template; wherein, the voucher template database contains multiple different types of voucher templates;

[0041] The voucher verification module is used to verify the initial voucher based on a preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher.

[0042] The above solution utilizes a neural network model combining CNN and Transformer to quickly process invoice images for identification. CNN excels at capturing local image features, while Transformer handles long-range dependencies. The combination allows the model to efficiently and accurately extract keyword information from images, significantly improving processing speed compared to manually examining and entering information from each invoice. This is particularly suitable for scenarios involving large volumes of invoices, such as accounting firms or large enterprises. Initial invoices are generated by calling voucher templates through similarity matching in a voucher template database, avoiding the tedious process of manual voucher preparation by accountants. This allows for the processing of multiple invoices and the generation of corresponding vouchers in a short time, automating the process from invoice information extraction to voucher generation. The neural network model is trained on multiple different types of invoice images, adapting to various common invoice styles. The voucher template database also contains various types of voucher templates, enabling the generation of corresponding vouchers based on different invoice information, meeting the diverse business needs of enterprises.

[0043] In one implementation, the information extraction module is used to input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information, specifically including:

[0044] The image of the ticket to be identified is input into the CNN neural network, so that the CNN neural network extracts the first ticket information from the image features of the received image and outputs the first spatial features; wherein, the first spatial features are used to define the bounding box where the text lines in the image of the ticket to be identified are located;

[0045] The first spatial feature and the image of the ticket to be identified are input into the OCR engine, so that the OCR engine performs text recognition on the image of the ticket to be identified and outputs the first text feature; wherein, the first text feature is text information carrying spatial location;

[0046] The first text feature is input into a Transformer neural network, which performs semantic analysis on the first text feature and outputs a second text feature; wherein the second text feature is used to characterize the logical relationship between the fields in the first text feature;

[0047] The keyword information is obtained by fusing the first text feature and the second text feature.

[0048] In the above scheme, a CNN neural network is used to extract the first piece of information from the image of the invoice to be recognized, outputting a first spatial feature to define the bounding box of the text lines. This accurately locates the text region, providing a clear target for subsequent processing and improving efficiency and accuracy. The first spatial feature and the invoice image are input into an OCR engine, and combined with the positioning information, text recognition is performed. The output is a first text feature carrying the spatial location, reducing interference and improving recognition accuracy. A Transformer neural network is used to perform semantic analysis on the first text feature, outputting a second text feature that reveals the logical relationship between fields, providing a deeper understanding of the business logic behind the text. Finally, the first and second text features are merged to obtain keyword information, achieving an organic combination of text content and semantic logic, making the information more complete and accurate. Moreover, this scheme is adaptable to various invoice types and has good scalability and versatility.

[0049] In one implementation, the CNN neural network extracts first ticket information from the received image features and outputs first spatial features, specifically including:

[0050] The image is convolved with a preset size based on the backbone network, and several first feature maps of different sizes are output and input into the feature pyramid network.

[0051] Based on the feature pyramid network, the first feature map is sampled at multiple scales, and a multi-scale feature pyramid is output and input into the region proposal network.

[0052] Based on the region proposal network, convolution and bounding box prediction are performed at each level of the multi-scale feature pyramid to output a first candidate region; wherein, the first candidate region is a region in the multi-scale feature pyramid that may contain a table or text.

[0053] The multi-scale feature pyramid and the first candidate region are aligned based on linear interpolation to output the first spatial feature.

[0054] In the above scheme, the backbone network is used to perform convolution on the image at a preset size to output first feature maps of different sizes, which can fully extract rich features of the image. The first feature map is sampled at multiple scales by a feature pyramid network to obtain a multi-scale feature pyramid, which can take into account the information of targets of different sizes and enhance the model's ability to recognize tables or text at different scales. The region proposal network outputs the first candidate region through convolution and bounding box prediction at each level of the multi-scale feature pyramid, accurately locating regions that may contain tables or text. Finally, linear interpolation is used to align the multi-scale feature pyramid and the first candidate region to output the first spatial feature, realizing effective matching between features and candidate regions, improving the accuracy and efficiency of text line bounding box localization in the ticket image, and laying a good foundation for subsequent information extraction.

[0055] In one implementation, before performing similarity matching based on the keyword information within a pre-built voucher template database, the method further includes multi-level classification of the keyword information and outputting multi-level classification results. Specifically:

[0056] The keyword information is segmented into words, and a term matrix is ​​constructed based on the segmentation results;

[0057] The term matrix is ​​input into a preset semantic model so that the semantic model outputs a semantic vector of a preset dimension;

[0058] Using a preset multi-level classification item as the classification task, a classifier is constructed based on each of the classification items. The semantic vector is input into each of the classifiers one by one, and the classification result output by the previous classifier is synchronously used as the input of the next classifier.

[0059] The multi-level classification results are generated based on the classification results output by each classifier; wherein the multi-level classification items are, in order, invoice type, tax type, industry type and amount type.

[0060] In the above scheme, keyword information is first segmented and a term matrix is ​​constructed, which can structure the text information and facilitate subsequent processing. The term matrix is ​​transformed into a semantic vector of a preset dimension using a preset semantic model, which can mine the semantic features of keyword information and improve the accuracy of classification. A classifier is constructed with preset multi-level classification items such as bill type, tax type, industry type, and amount type, and the output of the previous classifier is used as the input of the next classifier, realizing progressively refined and hierarchical classification, making the classification more logical and systematic. Finally, multi-level classification results are generated based on the output results of each classifier, which provides a more accurate and detailed classification basis for subsequent similarity matching in the voucher template database, which can effectively improve the efficiency and accuracy of matching, thereby improving the quality of the entire bill information processing process.

[0061] In one implementation, the voucher generation module is used to perform similarity matching in a pre-built voucher template database based on the keyword information, call the corresponding voucher template based on the similarity matching result, and generate an initial voucher based on the voucher template, specifically including:

[0062] Calculate the TF-IDF value of each field in the keyword information, and normalize each TF-ID value to obtain a keyword matrix; wherein each element in the keyword matrix represents the TF-IDF vector value of a field;

[0063] Based on the keyword matrix value, a traversal query is performed in the voucher template database to calculate the cosine similarity between the keyword matrix value and each voucher template in the voucher template database; wherein, each voucher template is composed of several commonly used invoice keywords;

[0064] The voucher template with the highest cosine similarity is selected as the voucher template for the invoice to be identified, and the initial voucher is generated based on the voucher template.

[0065] In the above scheme, the keyword matrix is ​​obtained by calculating and normalizing the TF-IDF value of each field in the keyword information. This effectively measures the importance of fields in the document, highlights key information, and the normalization process makes the data comparable. Based on the keyword matrix, the voucher template database is traversed and the cosine similarity with each voucher template is calculated. Using cosine similarity to measure the degree of similarity between texts, the voucher template that best matches the keyword information of the invoice to be identified can be found quickly and accurately. Finally, the voucher template with the highest cosine similarity is selected to generate the initial voucher, ensuring a high degree of fit between the generated initial voucher and the invoice to be identified, improving the accuracy and efficiency of voucher generation, and laying a good foundation for subsequent financial processing and other work.

[0066] In one implementation, the voucher verification module is used to verify the initial voucher based on a preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher, specifically including:

[0067] The initial voucher is first verified based on financial logic rules, and a first verification result is generated based on the verification result; wherein, the financial logic rules include debit and credit balance rules, tax rate logic rules, and amount consistency rules;

[0068] When the first verification result passes, a second verification is performed on the initial voucher based on business logic rules. When both the first and second verification results pass, the direct output rule is invoked to mark the initial voucher as verified and generate the target voucher. The business logic rules include: account matching rules, permission verification rules, and limit verification rules.

[0069] In the above scheme, the initial voucher is first verified based on financial logic rules including debit / credit balance rules, tax rate logic rules, and amount consistency rules. This ensures the accuracy of the voucher in basic financial logic from a financial professional perspective, guaranteeing the compliance and reliability of financial data. After the first verification passes, a second verification is performed based on business logic rules such as account matching rules, permission verification rules, and limit verification rules. This further examines the voucher from a business perspective, ensuring that the voucher conforms to the company's actual business situation. Only when both verification results pass can the direct output rules be invoked to mark the initial voucher as verified and generate the target voucher. This dual verification mechanism, with its multiple layers of checks, reduces the risk of erroneous vouchers entering the financial system, improves the quality and usability of the target voucher, and provides a solid guarantee for the company's financial management.

[0070] In one implementation, the step of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher further includes:

[0071] A unique identifier is generated for the ticket to be identified by combining the timestamp and preset key elements;

[0072] The unique identification code is associated with and stored with the keyword information;

[0073] When either the first verification result or the second verification result fails, the "mark for review" principle is invoked to trigger a manual review prompt; wherein, the manual review prompt is to send the unique identification code and the reason for verification failure to a preset manual review queue;

[0074] After the reviewer performs manual verification and correction, the target voucher for the document to be identified is generated based on the corrected output principles and the corrected information.

[0075] In the above scheme, a unique identification code is generated for the document to be identified by combining the timestamp and preset key elements, and stored in association with keyword information. This facilitates accurate management and traceability of document information, ensuring data integrity and queryability. When the first or second verification fails, the "mark for review" principle is invoked to trigger a manual review prompt. The unique identification code and the reason for the verification failure are sent to the preset manual review queue, which can promptly and accurately locate the problematic document and notify the reviewer, improving the efficiency of problem handling. After the reviewer manually verifies and corrects the document, the target document is generated based on the corrected output principle and correction information. This ensures the rigor of document generation and allows for manual intervention to guarantee the accuracy of the document when problems occur in automated verification. This achieves an effective combination of automation and manual review, improving the reliability and quality of the entire document processing flow.

[0076] Thirdly, this application also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the artificial intelligence-based automated office method described above.

[0077] Fourthly, this application also provides a computer-readable storage medium, which includes a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to perform the artificial intelligence-based automated office method described above. Attached Figure Description

[0078] Figure 1 This is a flowchart illustrating an automated office method based on artificial intelligence provided in one embodiment of the present invention;

[0079] Figure 2 This is a schematic diagram of a module of an automated office device based on artificial intelligence provided in one embodiment of the present invention. Detailed Implementation

[0080] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0081] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0082] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0083] Example 1

[0084] See Figure 1 , Figure 1This is a flowchart illustrating an automated office method based on artificial intelligence according to an embodiment of the present invention. The embodiment of the present invention provides an automated office method based on artificial intelligence, including steps 101 to 103, each step of which is detailed below:

[0085] Step 101: Input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network.

[0086] In this embodiment of the invention, a digital image of a document to be processed (such as an invoice, receipt, or check) is input into a pre-built neural network model. It should be noted that the input document image must be clear and unobstructed. The CNN neural network can extract the basic visual features of the document, and combined with the Transformer neural network for collaborative processing, outputting directly usable key field information.

[0087] In one embodiment, the step of inputting the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information specifically includes: inputting the image of the ticket to be identified into the CNN neural network, so that the CNN neural network performs image feature extraction on the received image to extract first ticket information and outputs first spatial features; wherein, the first spatial features are used to define the bounding boxes where the text lines in the image of the ticket to be identified are located; inputting the first spatial features and the image of the ticket to be identified into an OCR engine, so that the OCR engine performs text recognition on the image of the ticket to be identified and outputs first text features; wherein, the first text features are text information carrying spatial location; inputting the first text features into a Transformer neural network, so that the Transformer neural network performs semantic analysis on the first text features and outputs second text features; wherein, the second text features are used to characterize the logical relationship between the fields in the first text features; and fusing the first text features and the second text features to obtain the keyword information.

[0088] In this embodiment of the invention, an invoice to be identified (e.g., a 1024×768 pixel RGB image containing invoice code, amount, buyer and seller information, etc.) is input into a CNN neural network. Low-level features (edges, textures) are extracted through convolutional layers, and then semantic information (e.g., text region features) is fused through a deep network. Finally, the precise location of the text line is predicted through a regression layer, outputting the first spatial feature. The first spatial feature is the first normalized bounding box coordinates (format: [top left x, top left y, bottom right x, bottom right y]), with each box corresponding to a suspected text line region. For example,

[0089] Box 1: [0.12, 0.23, 0.28, 0.27]: Override the "Invoice Code" label;

[0090] Box 2: [0.32, 0.23, 0.45, 0.27]: Covers the invoice code number "044001800111";

[0091] Box 3: [0.10, 0.68, 0.45, 0.72]: Overrides the "Amount (in uppercase)" label;

[0092] Box 4: [0.60, 0.75, 0.85, 0.79]: Covers the amount "￥5,300.00";

[0093] Then, an OCR engine is used to crop the image region based on the bounding box coordinates, and each region is independently identified, outputting structured text information containing spatial location and confidence level, i.e., the first text feature. For example,

[0094]

[0095] Then, the first text information is input into a Transformer neural network (such as the BERT architecture). The Transformer neural network transforms the physical coordinates into vectors and incorporates them into the text embedding. It analyzes the logical relationships between fields (such as the association between "amount (uppercase)" and "five thousand three hundred yuan") and identifies key fields (such as locating company information through "buyer's name"). The output is an enhanced representation containing semantic relationships and field types, i.e., the second text feature. For example, "044001800111" is classified as an invoice code; "five thousand three hundred yuan" is established as an equivalent association with "¥5,300.00"; "XX Technology Co., Ltd." is labeled as the seller's name; and "2025-07-20" is identified as the invoice date. Finally, the two generated text features are fused, including alignment matching and confidence weighting. Specifically, the two types of features are associated according to spatial location (such as binding text with overlapping coordinates to semantic labels) and the final weight, i.e., the confidence score, is used as the OCR confidence score × semantic confidence score. Output the final keyword information, for example,

[0096]

[0097] In one embodiment, the step of using a CNN neural network to extract first ticket information from the received image features and output first spatial features specifically includes: performing convolution of the image at a preset size based on the backbone network to output several first feature maps of different sizes and inputting them into a feature pyramid network; performing multi-scale sampling of the first feature maps based on the feature pyramid network to output a multi-scale feature pyramid and inputting it into a region proposal network; performing convolution and bounding box prediction at each level of the multi-scale feature pyramid based on the region proposal network to output a first candidate region; wherein the first candidate region is a region in the multi-scale feature pyramid that may contain tables or text; aligning the multi-scale feature pyramid and the first candidate region based on linear interpolation to output the first spatial features.

[0098] In this embodiment of the invention, a ResNet-50 backbone network is used, and downsampling is performed through five convolutional stages (conv1 to conv5). Specifically, the initial convolutional layer (conv1) outputs a 1 / 2-size feature map, and then the residual block groups (conv2 to conv5) output feature maps of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 sizes respectively. Among them, size 1: 512×384 (conv2 output) is used to capture thick lines and large text in tables; size 2: 256×192 (conv3 output) is used to locate text blocks; size 3: 128×96 (conv4 output) is used to identify small-sized text lines; and size 4: 64×48 (conv5 output) is used to extract local character features. The four sets of first feature maps output by the backbone network (sizes of 512×384, 256×192, 128×96, and 64×48 respectively) are sampled from top to bottom by the feature pyramid network. That is, upsampling starts from the smallest feature map (64×48), and the maps are superimposed with feature maps from adjacent layers. After each layer is superimposed, the number of channels is adjusted through a 1×1 convolution, ultimately generating a four-layer unified 256-channel feature pyramid, which is then input into the region proposal network. For example,

[0099]

[0100] The region proposal network scans the feature maps at each layer using 3×3 convolutional kernels. Each window simultaneously outputs: a region classification score (table / text vs. background) and bounding box offsets (Δx, Δy, Δw, Δh). Low-scoring candidate boxes with an overlap rate >70% are filtered out, and then the first candidate region is output. For example,

[0101] Region 1: P2 layer prediction, covering the "Buyer Information" table (confidence level 0.98);

[0102] Region 2: P3 layer prediction, covering "Cargo Details" (confidence level 0.95);

[0103] Region 3: P4 layer prediction, covering the "total price and tax" text line (confidence level 0.97);

[0104] Region 4: P5 layer prediction, covering the "invoice code" numeric area (confidence level 0.92).

[0105] The multi-scale feature pyramid and the first candidate region obtained earlier are aligned, that is, each candidate box is mapped to the corresponding pyramid level (small box → high level P5, large box → low level P2). Then, bilinear interpolation is used to calculate the feature value by weighting the non-integer coordinate positions. The output is the first spatial feature with a dimension of 200×256×7×7 (200 regions, 256 channels, 7×7 grid). That is, each 7×7 grid encodes the local features of a candidate region (such as the grid center point to enhance the table line features, and the edge points to capture the strokes of the text).

[0106] Step 102: Perform similarity matching in the pre-built voucher template database based on the keyword information, call the corresponding voucher template based on the similarity matching result, and generate an initial voucher based on the voucher template; wherein, the voucher template database contains multiple different types of voucher templates.

[0107] In this embodiment of the invention, the structured keyword information output by the invoice recognition is matched with the voucher template database for similarity, thereby directly calling the voucher template with high similarity to generate vouchers, without the need for manual voucher filling.

[0108] In one embodiment, before performing similarity matching based on the keyword information within a pre-built voucher template database, the method further includes performing multi-level classification on the keyword information and outputting multi-level classification results. Specifically: the keyword information is segmented into words, and a term matrix is ​​constructed based on the segmentation results; the term matrix is ​​input into a preset semantic model, using preset multi-level classification items as the classification task, and a classifier is constructed based on each of the classification items; the semantic vector is input into each of the classifiers one by one, and the classification result output by the previous classifier is synchronously used as the input of the next classifier; the multi-level classification results are generated based on the classification results output by each classifier; wherein, the multi-level classification items are, in order, invoice type, tax type, industry type, and amount type.

[0109] In this embodiment of the invention, keyword information is concatenated into a string and then segmented. For example, the concatenated string is "Value-added tax special invoice, invoice copy, invoice date: 2023-07-20, buyer: ABC Technology, tax number: 913205XX, catering service * meal fee, amount: ¥1,280.00, tax amount: ¥64.00". Segmentation is performed based on common financial terminology, product name splitting, numerical tagging, and stop word filtering. For example, "value-added tax special invoice" is retained as a whole as an indivisible word; "catering service * meal fee" is split into ["catering service", "meal fee"]; and the amount ¥1,280.00 (marked as...) is further segmented. <amt>) and the date 2023-07-20 (marked as <date>Mark the terms; remove the word "coupling" from "invoice copy" to obtain the final word segmentation result, i.e., the term matrix is ​​["Value-added tax special invoice", "invoice date", " <date>","Purchaser","ABC Technology","Tax ID","913205XX","Catering Services","Meal Expenses","Amount", <amt>","tax"," <amt>The term matrix is ​​input into the Sentence-BERT model for context awareness, which strengthens the association between terms. For example, detecting the "tax amount" field activates the tax feature dimension; detecting the keyword "catering" activates the industry feature dimension; "catering service" + "meal cost" strengthens the catering feature; "VAT invoice" + "tax amount" strengthens the input tax feature. The final output is a 128-dimensional comprehensive semantic vector, where each dimension represents a specific semantic feature learned by the model, with values ​​ranging from -1 to 1. For example, [0.24, -0.18, 0.76, 0.05, 0.91, -0.32, 0.67, ..., 0.12] represent 128 dimensions. Semantic features are automatically learned through the Transformer self-attention layer, with each dimension corresponding to a... Latent semantic concepts (such as industry, region, consumption attributes). It should be noted that this Transformer self-attention layer is trained on a large number of control group invoice samples of different types (restaurant invoices - non-restaurant invoices; international invoices - domestic invoices, etc.). After training, thresholds for each dimension are defined. For example, dimension 32 with a threshold > 0.7 indicates the consumption type is catering, while < 0 indicates goods (learned from different invoice types such as catering services or office supplies); dimension 67 with a threshold > 0.5 indicates domestic consumption, while < 0.3 indicates domestic consumption (learned from the currency symbol or tax number format of the invoice); dimension 112 with a threshold > 0.8 indicates corporate consumption, while < 0.2 indicates personal consumption (learned from the presence or absence of a tax number). Examples of several key dimensions in the 128-dimensional semantic vector are as follows:

[0110]

[0111] The semantic vector is then input into a classifier. For example, this embodiment of the invention uses an XGBoost classifier. Four independent XGBoost classifiers (decision trees) are used to handle four classification tasks (invoice type, tax type, industry type, and amount type). Each classifier receives the same 128-dimensional semantic vector as input and makes classification decisions according to preset decision rules. Specifically, the XGBoost classifier for invoice type classification uses the thresholds of each dimension defined by the Transformer self-attention layer as path judgments for the root node, left child node, and right child node in the decision tree. For example, if node dimension 112 is determined to be "enterprise consumption" based on dimension > 0.8, the next level left child node (dimension 45 > 0.7) is determined to be "office supplies," and the right child node (dimension 32 > 0.6) is determined to be "catering." The output classification result can be exemplified as follows:

[0112]

[0113] The XGBoost classifier, which classifies tax types, uses a pre-defined decision rule: when it detects "corporate consumption (determined by the threshold of dimension 112) + VAT invoice (invoice type determined by the classification result of the previous classifier)," it classifies the tax type as input tax. An example of its output classification result is shown below:

[0114]

[0115] The XGBoost classifier, which classifies by industry type, also uses pre-defined decision rules. These rules are based on industry-wide rules and are used to determine the industry type of the ticket to be identified (office supplies retail, food service, transportation, and information technology services), including:

[0116]

[0117]

[0118] Based on the above combination rules, the industry type of the invoice can be identified and output. An example of the output classification result is as follows:

[0119]

[0120] It should be noted that the above thresholds are set based on the user's usage scenario needs and experience, and can be customized and adjusted, without limitation here.

[0121] Preferably, once the industry type is determined, the deductible tax rate can be further determined based on the industry type. The deductible tax rate for invoices in the office supplies retail industry is 13%; for invoices in the catering service industry, it is 6%; and for invoices in the transportation industry, it is 9%. The XGBoost classifier, which classifies by industry type, determines whether the actual amount in the semantic vector is eligible for tax deduction based on the classification result of the industry type. Based on the classification results of the above classifiers, the final multi-level classification result can be obtained, which can be exemplified as follows:

[0122] 1. Invoice type: Value-added tax special invoice (confidence level 0.97);

[0123] 2. Tax type: Input tax (confidence level 0.98);

[0124] 3. Industry type: Office supplies retail (confidence level 0.96);

[0125] 4. Amount type: Deductible (confidence level 0.92).

[0126] In one embodiment, the step of performing similarity matching in a pre-built voucher template database based on the keyword information, calling the corresponding voucher template based on the similarity matching result, and generating an initial voucher based on the voucher template specifically includes: calculating the TF-IDF value of each field in the keyword information, and normalizing each TF-ID value to obtain a keyword matrix; wherein each element in the keyword matrix represents the TF-IDF vector value of a field; performing a traversal query in the voucher template database based on the keyword matrix value, and calculating the cosine similarity between the keyword matrix value and each voucher template in the voucher template database; wherein each voucher template consists of several commonly used invoice keywords; selecting the voucher template with the highest cosine similarity as the voucher template for the invoice to be identified, and generating the initial voucher based on the voucher template.

[0127] In this embodiment of the invention, a historical invoice dataset is constructed based on multiple invoice types, and a database of commonly used invoice keywords defined by domain experts is used for feature word selection to form different types of initial voucher templates. The TF-IDF value of each feature word in the initial voucher template is calculated and normalized to obtain different types of voucher templates. TF (Term Frequency) refers to the frequency of a word appearing in a single template document. For example, there are three different types of voucher templates T001, T002, and T003. For instance, for template T001 and the feature word "value-added tax," if "value-added tax" appears 95 times in document T001, and the total number of words in T001 is 1000, then TF = 95 / 1000 = 0.095. IDF (Inverse Document Frequency) measures the rarity of a word in the entire corpus, and its calculation is based on all template documents. For example, to calculate the IDF value of "Value Added Tax" (VAT): Since "VAT" appears in both T001 and T002, then IDF("VAT") = log(3 / 2) ≈ log(1.5) ≈ 0.176 (using the natural logarithm). The TF-IDF value is the product of TF and IDF. For example, for T001 and "VAT": TF-IDF = 0.095 × 0.176 ≈ 0.0167.

[0128] For example, an example of a voucher template database is as follows:

[0129]

[0130]

[0131] It should be noted that the keyword features and vector values ​​mentioned above are calculated based on user-collected data and are subject to updates and adjustments. The database mentioned above is for illustrative purposes only and is not intended to limit the scope. Assuming the keyword information of the invoice to be identified is: ["VAT invoice", "input tax", "office supplies", "deductible", "printer paper"], the TF-IDF value of each field in the keyword information is calculated. The calculation process of the TF-IDF value has been explained in detail above and will not be repeated here. Then, the calculated TF-IDF values ​​are normalized to obtain the normalized invoice TF-IDF vector, i.e., the keyword matrix [0.97, 0.93, 0.01, 0.89]. Based on this keyword matrix, the similarity is calculated for each voucher template in the voucher template database. The similarity ranking results between the invoice to be identified and each voucher template are output. The voucher template with the highest similarity result is selected as the voucher template for the invoice to be identified. Then, the complete structure of this voucher template is called, and the keyword information of the invoice is filled into the voucher template to generate the initial voucher.

[0132] Step 103: Verify the initial voucher based on the preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher.

[0133] In one embodiment, the step of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher specifically includes: performing a first verification on the initial voucher based on financial logic rules, and generating a first verification result based on the verification result; wherein, the financial logic rules include debit / credit balance rules, tax rate logic rules, and amount consistency rules; when the first verification result passes, performing a second verification on the initial voucher based on business logic rules; when both the first verification result and the second verification result pass, invoking the direct output rule to mark the initial voucher as verified and generating the target voucher; wherein, the business logic rules include: account matching rules, permission verification rules, and limit verification rules.

[0134] In this embodiment of the invention, initial vouchers are preliminarily verified using common financial logic rules. These rules include debit / credit balance rules, tax rate logic rules, and amount consistency rules. Specifically, debit / credit balance checks whether the total debit amount equals the total credit amount of the voucher; the tax rate logic rule verifies whether the applied tax rate complies with national regulations and corporate policies based on the "voucher type" and "goods / service type" in the keyword information (e.g., whether the VAT rate for special VAT invoices is 3%, 6%, 9%, or 13%, and whether ordinary invoices should be tax-free or have a zero tax rate); the amount consistency rule compares the logical relationships between key amounts identified from the voucher image (e.g., total price including tax, amount excluding tax, and tax amount) to ensure they are valid (e.g., total price including tax = amount excluding tax * (1 + tax rate)). Only after the above basic financial logic rules are passed will a second layer of verification be triggered: business logic rule verification, including account matching rules, permission verification rules, and limit verification rules. Specifically, the account matching rule verifies whether the assigned accounting account conforms to the enterprise's accounting system or preset mapping rules based on keywords in the voucher template (such as "transportation expenses", "accommodation expenses", "office supplies"). The permission verification rule determines whether the voucher requires specific approver permissions. The limit verification rule checks whether specific types of expenses (such as "business entertainment expenses" and "travel expenses") exceed preset single or monthly limits. Furthermore, certain business list verification rules can be set, such as checking whether the counterparty's name on the invoice exists in a preset blacklist database. If all verifications pass, the direct output rule is invoked, the initial voucher is marked as "verified," and the final target voucher is automatically generated (usually in a standard electronic voucher format, such as XML, JSON, or directly entered into the financial system).

[0135] In one embodiment, the step of verifying the initial voucher based on a preset verification, calling the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher further includes: generating a unique identification code for the document to be identified by combining the timestamp and preset key elements; associating and storing the unique identification code with the keyword information; when the first verification result or the second verification result is unsuccessful, triggering a manual review prompt by calling the marking-for-review principle; wherein, the manual review prompt is to send the unique identification code and the reason for verification failure to a preset manual review queue; after the reviewer performs manual verification correction, generating the target voucher for the document to be identified based on the corrected output principle and the corrected information.

[0136] In this embodiment of the invention, a unique identifier for the invoice to be identified is generated by combining a timestamp with key elements such as the user ID (uploader), invoice type, and hash value of invoice information elements (invoice code + invoice number + amount). Then, key-value pairs are used for associated storage, with the unique identifier as the key. Stored data includes: the original invoice image storage path (or binary data thumbnail), extracted keyword information (structured data, such as invoice code, invoice number, amount, tax, etc.), and other metadata (upload time, user information, etc.). When either the first or second verification result fails, a manual review prompt is triggered, and the invoice corresponding to the unique identifier is marked as requiring manual review. The unique identifier and verification identifier are then sent to the finance staff's workbench or other manual review channels. During manual review, finance staff retrieve all associated information from storage based on the unique identifier, check the information in the associated storage according to the reason for verification failure, confirm and correct the correct content, and generate the final target voucher after correction. For example, the finance staff discovered that the OCR engine incorrectly identified the tax rate of 6% as 5%, resulting in an incorrect tax calculation (the actual tax amount was 60 yuan, but it was identified as 50 yuan). The finance staff corrected the tax amount in the keyword information to 60 yuan and confirmed that the tax rate field was 6%. After the correction, the corrected output principle automatically regenerated the voucher.

[0137] In this embodiment of the invention, an automated office device based on artificial intelligence is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-mentioned automated office method based on artificial intelligence.

[0138] In this embodiment of the invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the above-described artificial intelligence-based automated office method when it is running.

[0139] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to perform the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in an AI-based automated office device.

[0140] The AI-based automated office equipment can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. AI-based automated office equipment may include, but is not limited to, processors, memory, and displays. Those skilled in the art will understand that the above components are merely examples of AI-based automated office equipment and do not constitute a limitation on AI-based automated office equipment. It may include more or fewer components, combinations of certain components, or different components. For example, AI-based automated office equipment may also include input / output devices, network access devices, buses, etc.

[0141] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of AI-based automated office equipment, connecting all parts of the AI-based automated office processing equipment through various interfaces and lines.

[0142] The memory can be used to store computer programs and / or modules. The processor implements various functions of AI-based automated office equipment by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback, text conversion, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0143] In this invention, if the AI-based automated office module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Those skilled in the art can understand and implement this invention without any inventive effort.

[0144] This invention provides an automated office method based on artificial intelligence. Utilizing a neural network model combining CNN and Transformer, it can quickly process images of invoices to be identified. CNN excels at capturing local image features, while Transformer handles long-distance dependencies. The combination of these two allows the model to efficiently and accurately extract keyword information from images. Compared to manually examining and entering information from each invoice, this significantly improves processing speed, making it particularly suitable for scenarios involving large volumes of invoices, such as accounting firms or large enterprises. Initial invoices are generated by calling voucher templates through similarity matching in a voucher template database, avoiding the tedious process of manual voucher preparation by accountants. This allows for the processing of multiple invoices and the generation of corresponding vouchers in a short time, achieving automation from invoice information extraction to voucher generation. The neural network model is trained on multiple different types of invoice images, enabling it to adapt to various common invoice styles. The voucher template database also contains various types of voucher templates, allowing for the generation of corresponding vouchers based on different invoice information, meeting the diverse business needs of enterprises.

[0145] Example 2

[0146] See Figure 2 , Figure 2 This is a schematic diagram of a module of an automated office device based on artificial intelligence provided in one embodiment of the present invention. The embodiment of the present invention provides an automated office device based on artificial intelligence, including: an information extraction module 201, a voucher generation module 202, and a voucher verification module 203;

[0147] The information extraction module 201 is used to input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network, combined with multiple different types of ticket images and the keyword information corresponding to each ticket image;

[0148] The voucher generation module 202 is used to perform similarity matching in a pre-built voucher template database based on the keyword information, call the corresponding voucher template according to the similarity matching result, and generate an initial voucher based on the voucher template; wherein, the voucher template database contains a variety of different types of voucher templates;

[0149] The voucher verification module 203 is used to verify the initial voucher based on a preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher.

[0150] In one embodiment, the information extraction module is used to input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information, specifically including:

[0151] The image of the ticket to be identified is input into the CNN neural network, so that the CNN neural network extracts the first ticket information from the image features of the received image and outputs the first spatial features; wherein, the first spatial features are used to define the bounding box where the text lines in the image of the ticket to be identified are located;

[0152] The first spatial feature and the image of the ticket to be identified are input into the OCR engine, so that the OCR engine performs text recognition on the image of the ticket to be identified and outputs the first text feature; wherein, the first text feature is text information carrying spatial location;

[0153] The first text feature is input into a Transformer neural network, which performs semantic analysis on the first text feature and outputs a second text feature; wherein the second text feature is used to characterize the logical relationship between the fields in the first text feature;

[0154] The keyword information is obtained by fusing the first text feature and the second text feature.

[0155] In one embodiment, the CNN neural network extracts first ticket information from the received image features and outputs first spatial features, specifically including:

[0156] The image is convolved with a preset size based on the backbone network, and several first feature maps of different sizes are output and input into the feature pyramid network.

[0157] Based on the feature pyramid network, the first feature map is sampled at multiple scales, and a multi-scale feature pyramid is output and input into the region proposal network.

[0158] Based on the region proposal network, convolution and bounding box prediction are performed at each level of the multi-scale feature pyramid to output a first candidate region; wherein, the first candidate region is a region in the multi-scale feature pyramid that may contain a table or text.

[0159] The multi-scale feature pyramid and the first candidate region are aligned based on linear interpolation to output the first spatial feature.

[0160] In one embodiment, before performing similarity matching within a pre-built voucher template database based on the keyword information, the method further includes multi-level classification of the keyword information and outputting multi-level classification results. Specifically:

[0161] The keyword information is segmented into words, and a term matrix is ​​constructed based on the segmentation results;

[0162] The term matrix is ​​input into a preset semantic model so that the semantic model outputs a semantic vector of a preset dimension;

[0163] Using a preset multi-level classification item as the classification task, a classifier is constructed based on each of the classification items. The semantic vector is input into each of the classifiers one by one, and the classification result output by the previous classifier is synchronously used as the input of the next classifier.

[0164] The multi-level classification results are generated based on the classification results output by each classifier; wherein the multi-level classification items are, in order, invoice type, tax type, industry type and amount type.

[0165] In one embodiment, the voucher generation module is used to perform similarity matching in a pre-built voucher template database based on the keyword information, call the corresponding voucher template according to the similarity matching result, and generate an initial voucher based on the voucher template, specifically including:

[0166] Calculate the TF-IDF value of each field in the keyword information, and normalize each TF-ID value to obtain a keyword matrix; wherein each element in the keyword matrix represents the TF-IDF vector value of a field;

[0167] Based on the keyword matrix value, a traversal query is performed in the voucher template database to calculate the cosine similarity between the keyword matrix value and each voucher template in the voucher template database; wherein, each voucher template is composed of several commonly used invoice keywords;

[0168] The voucher template with the highest cosine similarity is selected as the voucher template for the invoice to be identified, and the initial voucher is generated based on the voucher template.

[0169] In one embodiment, the voucher verification module is used to verify the initial voucher based on a preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher, specifically including:

[0170] The initial voucher is first verified based on financial logic rules, and a first verification result is generated based on the verification result; wherein, the financial logic rules include debit and credit balance rules, tax rate logic rules, and amount consistency rules;

[0171] When the first verification result passes, a second verification is performed on the initial voucher based on business logic rules. When both the first and second verification results pass, the direct output rule is invoked to mark the initial voucher as verified and generate the target voucher. The business logic rules include: account matching rules, permission verification rules, and limit verification rules.

[0172] In one embodiment, the step of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher further includes:

[0173] A unique identifier is generated for the ticket to be identified by combining the timestamp and preset key elements;

[0174] The unique identification code is associated with and stored with the keyword information;

[0175] When either the first verification result or the second verification result fails, the "mark for review" principle is invoked to trigger a manual review prompt; wherein, the manual review prompt is to send the unique identification code and the reason for verification failure to a preset manual review queue;

[0176] After the reviewer performs manual verification and correction, the target voucher for the document to be identified is generated based on the corrected output principles and the corrected information.

[0177] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0178] This invention provides an automated office device based on artificial intelligence. Utilizing a neural network model combining CNN and Transformer, it can quickly process images of invoices to be identified. CNN excels at capturing local image features, while Transformer handles long-distance dependencies. The combination of these two technologies allows the model to efficiently and accurately extract keyword information from images. Compared to manually examining and entering information from each invoice, this significantly improves processing speed, making it particularly suitable for scenarios involving large volumes of invoices, such as accounting firms or large enterprises. Initial invoices are generated by calling voucher templates through similarity matching in a voucher template database, avoiding the tedious process of manual voucher preparation by accountants. This allows for the processing of multiple invoices and the generation of corresponding vouchers in a short time, achieving automation from invoice information extraction to voucher generation. The neural network model is trained on multiple different types of invoice images, enabling it to adapt to various common invoice styles. The voucher template database also contains various types of voucher templates, allowing for the generation of corresponding vouchers based on different invoice information, meeting the diverse business needs of enterprises.

[0179] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.< / amt> < / amt> < / date> < / date> < / amt>

Claims

1. An automated office method based on artificial intelligence, characterized in that, include: The image of the ticket to be identified is input into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network, combined with multiple different types of ticket images and the keyword information corresponding to each ticket image; Based on the keyword information, a similarity match is performed in a pre-built voucher template database. The corresponding voucher template is called based on the similarity match result, and an initial voucher is generated based on the voucher template. The voucher template database contains various types of voucher templates. The initial voucher is validated based on a preset validation, and the corresponding voucher output rule is invoked according to the validation result. The target voucher for the document to be identified is generated based on the voucher output rule and the initial voucher.

2. The automated office method based on artificial intelligence as described in claim 1, characterized in that, The step of inputting the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information specifically includes: The image of the ticket to be identified is input into the CNN neural network, so that the CNN neural network extracts the first ticket information from the image features of the received image and outputs the first spatial features; wherein, the first spatial features are used to define the bounding box where the text lines in the image of the ticket to be identified are located; The first spatial feature and the image of the ticket to be identified are input into the OCR engine, so that the OCR engine performs text recognition on the image of the ticket to be identified and outputs the first text feature; wherein, the first text feature is text information carrying spatial location; The first text feature is input into a Transformer neural network, which performs semantic analysis on the first text feature and outputs a second text feature; wherein the second text feature is used to characterize the logical relationship between the fields in the first text feature; The keyword information is obtained by fusing the first text feature and the second text feature.

3. The automated office method based on artificial intelligence as described in claim 2, characterized in that, The CNN neural network extracts first ticket information from the received image features and outputs first spatial features, specifically including: The image is convolved with a preset size based on the backbone network, and several first feature maps of different sizes are output and input into the feature pyramid network. Based on the feature pyramid network, the first feature map is sampled at multiple scales, and a multi-scale feature pyramid is output and input into the region proposal network. Based on the region proposal network, convolution and bounding box prediction are performed at each level of the multi-scale feature pyramid to output a first candidate region; wherein, the first candidate region is a region in the multi-scale feature pyramid that may contain a table or text. The multi-scale feature pyramid and the first candidate region are aligned based on linear interpolation to output the first spatial feature.

4. The automated office method based on artificial intelligence as described in claim 1, characterized in that, Before performing similarity matching within a pre-built voucher template database based on the keyword information, the method further includes multi-level classification of the keyword information and outputting multi-level classification results. Specifically: The keyword information is segmented into words, and a term matrix is ​​constructed based on the segmentation results; The term matrix is ​​input into a preset semantic model so that the semantic model outputs a semantic vector of a preset dimension; Using a preset multi-level classification item as the classification task, a classifier is constructed based on each of the classification items. The semantic vector is input into each of the classifiers one by one, and the classification result output by the previous classifier is synchronously used as the input of the next classifier. The multi-level classification results are generated based on the classification results output by each classifier; wherein the multi-level classification items are, in order, invoice type, tax type, industry type and amount type.

5. The automated office method based on artificial intelligence as described in claim 4, characterized in that, The step of performing similarity matching in a pre-built voucher template database based on the keyword information, calling the corresponding voucher template based on the similarity matching result, and generating an initial voucher based on the voucher template specifically includes: Calculate the TF-IDF value of each field in the keyword information, and normalize each TF-ID value to obtain a keyword matrix; wherein each element in the keyword matrix represents the TF-IDF vector value of a field; Based on the keyword matrix value, a traversal query is performed in the voucher template database to calculate the cosine similarity between the keyword matrix value and each voucher template in the voucher template database; wherein, each voucher template is composed of several commonly used invoice keywords; The voucher template with the highest cosine similarity is selected as the voucher template for the invoice to be identified, and the initial voucher is generated based on the voucher template.

6. The automated office method based on artificial intelligence as described in claim 1, characterized in that, The process of verifying the initial voucher based on a preset verification, invoking the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher specifically includes: The initial voucher is first verified based on financial logic rules, and a first verification result is generated based on the verification result; wherein, the financial logic rules include debit and credit balance rules, tax rate logic rules, and amount consistency rules; When the first verification result passes, a second verification is performed on the initial voucher based on business logic rules. When both the first and second verification results pass, the direct output rule is invoked to mark the initial voucher as verified and generate the target voucher. The business logic rules include: account matching rules, permission verification rules, and limit verification rules.

7. The automated office method based on artificial intelligence as described in claim 6, characterized in that, The step of verifying the initial voucher based on a preset verification, calling the corresponding voucher output rule according to the verification result, and generating the target voucher for the document to be identified based on the voucher output rule and the initial voucher further includes: A unique identifier is generated for the ticket to be identified by combining the timestamp and preset key elements; The unique identification code is associated with and stored with the keyword information; When either the first verification result or the second verification result fails, the "mark for review" principle is invoked to trigger a manual review prompt; wherein, the manual review prompt is to send the unique identification code and the reason for verification failure to a preset manual review queue; After the reviewer performs manual verification and correction, the target voucher for the document to be identified is generated based on the corrected output principles and the corrected information.

8. An automated office device based on artificial intelligence, characterized in that, include: Information extraction module, voucher generation module, and voucher verification module; The information extraction module is used to input the image of the ticket to be identified into a pre-built neural network model to extract ticket information and obtain keyword information; wherein, the neural network model is trained based on CNN neural network and Transformer neural network, combined with multiple different types of ticket images and the keyword information corresponding to each ticket image; The voucher generation module is used to perform similarity matching in a pre-built voucher template database based on the keyword information, call the corresponding voucher template based on the similarity matching result, and generate an initial voucher based on the voucher template; wherein, the voucher template database contains multiple different types of voucher templates; The voucher verification module is used to verify the initial voucher based on a preset verification, call the corresponding voucher output rule according to the verification result, and generate the target voucher of the document to be identified based on the voucher output rule and the initial voucher.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the AI-based automated office method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the automated office method based on artificial intelligence as described in any one of claims 1 to 7.

Images