A method and system for digitizing audit vouchers

Abstract

The present invention discloses a method and system for digitizing audit vouchers, including: constructing an image resource library of audit documents; using an OCA multi-stage optical character recognition model to perform text recognition on images in the image resource library to obtain digitized audit documents; using an ACPmarked architecture to perform text classification on digitized audit documents to remove audit-irrelevant text content. The present invention can effectively realize the digitization of audit vouchers, reduce the labor intensity of audit professionals, and is of great significance to improving audit efficiency and quality.

Description

Technical Field

[0001] The present invention relates to the technical field of document text detection and recognition, and in particular to a method and system for digitizing audit vouchers. Background Art

[0002] At present, the main application scenarios of OCR include natural scene text detection and recognition, handwritten text detection and recognition, document text detection and recognition, and audit vouchers belong to document text detection and recognition. The processing methods of document text are divided into single-stage and multi-stage processing. Traditional single-stage text recognition has low accuracy and there is a phenomenon of losing text content. Traditional multi-stage text recognition has good accuracy, but the detection time is too long, and the recognition effect is not good for audit texts with long lengths. Summary of the invention

[0003] The purpose of the present invention is to provide an audit voucher digitization method and system based on the OAT architecture, which method and system can also have excellent recognition effects on audit texts with longer lengths.

[0004] The audit voucher digitization method based on the OAT architecture provided by the present invention includes:

[0005] Build an image repository for audit documents;

[0006] Use the OCA multi-stage optical character recognition model to perform text recognition on images in the image resource library to obtain digital audit documents;

[0007] Use the ACPmarked architecture to perform text classification on digitized audit documents and remove text content irrelevant to the audit.

[0008] In some specific implementations, the method for constructing an image resource library for audit documents is:

[0009] Use crawler technology to crawl audit documents on the Internet; print audit documents; scan paper audit documents to obtain images of audit documents.

[0010] In some specific implementations, the OCA multi-stage optical character recognition model is used to first enhance the image, including correcting the problem image, highlighting the text features, and sharpening the text; then the Teacher text detection model and the Teacher text recognition model are used to train the Student text detection model and the Student text recognition model; finally, the Student text detection model and the Student text recognition model are used to detect and recognize text on the image.

[0011] In some specific implementations, knowledge distillation is performed on the text detection model, including: the Teacher text detection model uses VGG16 and ResNet50 network structures in parallel in the convolution layer, and splices the operation results of the two; in the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used to extract serialization features; the operation results are obtained through the fully connected layer and the RPN network, and the knowledge is distilled to obtain soft knowledge;

[0012] The Student text detection model uses the VGG16 network and adds a dropout mechanism. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used to obtain the calculation results through a single fully connected layer and the RPN network.

[0013] In some specific implementations, knowledge distillation is performed on the text recognition model, including:

[0014] The Teacher text recognition model uses the VGG16 convolutional neural network to extract image features in the convolutional layer. It then uses a bidirectional long short-term memory network based on the attention mechanism to extract text serialization features. Finally, the vector is passed to the transcription layer with a CTC loss function to obtain the result, and the result is distilled to obtain soft knowledge.

[0015] The Student text recognition model uses a three-layer CNN convolutional unit to extract image features, then passes the resulting vector through a bidirectional long short-term memory neural network based on an attention mechanism, and finally uses a transcription layer with a CTC loss function to obtain the vector result.

[0016] In some specific implementations, the ACPmarked framework is used to perform text classification on digitized audit documents, including:

[0017] Crawl text from the Internet as pre-training corpus for the Transformer module and use the word2vec model for training; introduce the position encoding matrix of the recognized text to obtain word vectors with position encoding;

[0018] The attention-based bidirectional long short-term memory module uses pre-trained word vectors to encode the input text vectors and obtain word vector representations;

[0019] Convolution and pooling modules are used to extract text features and classify text.

[0020] The audit voucher digitization system based on the OAT architecture provided by the present invention includes:

[0021] The first module is used to build an image resource library for audit documents;

[0022] The second module is used to use the OCA multi-stage optical character recognition model to perform text recognition on images in the image resource library to obtain digital audit documents;

[0023] The third module is used to use the ACPmarked architecture to perform text classification on digitized audit documents and remove text content that is not relevant to the audit.

[0024] The present invention integrates the traditional audit process with information technology, and proposes an OAT-based audit voucher digitization architecture to realize the digitization of audit vouchers. The OAT architecture is mainly composed of the OCA multi-stage optical character recognition model and the ACPmarked multi-semantic fusion architecture. Different from the traditional OCR recognition method, the OCA multi-stage optical character recognition technology integrates image enhancement technology and attention mechanism, and uses knowledge distillation technology to improve the model processing speed while ensuring accuracy. Subsequently, the ACPmarked architecture is used to extract sentences related to the audit task to form a digital high-quality audit text. Finally, the encrypted graph transmission method is used to protect the data security in the audit process. The OAT architecture can effectively realize the digitization of audit vouchers, reduce the labor intensity of audit professionals, and is of great significance to improving audit efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the flowchart of CRNN model;

[0026] Figure 2 This is the Encoder-Decoder framework diagram without attention mechanism. In the figure, Encoder is the encoding end and Decoder is the decoding end.

[0027] Figure 3 This is the Encoder-Decoder framework diagram with an attention mechanism. In the figure, x1, x2, x3, x4 are input data, y1, y2, y3, y4 are output data, Encoder is the encoding end, and Decoder is the decoding end;

[0028] Figure 4 This is a flow chart of the knowledge distillation model;

[0029] Figure 5 A flowchart of the OCA multi-stage optical character recognition model;

[0030] Figure 6 Schematic diagram of the image correction process, where (a) is the image after binarization, (b) is the image after pixel inversion, and (c) is the image after affine transformation;

[0031] Figure 7Schematic diagram of the comparison before and after image enhancement, wherein (a) is the image before enhancement, and (b) is the image after enhancement;

[0032] Figure 8 This is the ACPmarked multi-semantic fusion architecture diagram;

[0033] Fig. 9 1 are problematic images that may appear when constructing an image data resource library in the embodiment, wherein (a) is an overexposed image, (b) is a blurred image, and (c) is an image with an excessively large rotation angle;

[0034] Fig.10 is the accuracy of different models in the embodiment;

[0035] Fig.11 2 are the visualization results of the OCA-student model and the PaddleOCR model in the embodiment, wherein Figure (a) is the visualization result of the OCA-student model, and Figure (b) is the visualization result of the PaddleOCR model. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the specific embodiments described are only used to explain but not to limit the scope of protection of the present invention, and all other specific embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] First, the technology involved in the present invention will be described in detail.

[0038] 1. Multi-stage optical character recognition model based on OCA.

[0039] 1. Position detection model.

[0040] The primary task of paper text digitization is to locate the position of text in complex scenes, that is, text detection. The Connectionist Text Proposal Network (CTPN) method uses an improved vertical anchor method (vertical grid method) to only predict the vertical position of the text, fix the width of the grid, and do not predict the horizontal direction. The horizontal length of the grid is 16 pixels. The vertical length is from 11 pixels to 283 pixels. This interval is divided by 0.7 each time to get 10 anchors, with specific values: 11, 16, 23, 33, 48, 68, 97, 139, 198, 283. Then the recurrent network is used to connect the detected small-scale text to obtain text lines. Among them, the convolutional neural network uses VGG to extract image features; the recurrent neural network uses a bidirectional long short-term memory network to splice the features; and finally the fully connected neural network and RPN network are used to obtain the CTPN text detection results.

[0041] 2. Text Recognition Model

[0042] The position detection model can be used to locate the specific position of the text in the image, and then the text recognition model can be used to recognize the text. In this specific implementation, the Convolutional Recurrent Neural Network (CRNN) method is used to recognize text. The CRNN model is mainly divided into three parts: convolutional neural network layer, recurrent neural network layer, and transcription layer. The convolutional neural network layer extracts image features through a convolutional neural network (CNN); the recurrent neural network layer uses a bidirectional long short-term memory network to extract serialized features of the text; the transcription layer introduces the Connectionist temporal classification (CTC) Loss calculation method to calculate the loss value of the recognition content without alignment; and finally the character output is obtained through the softmax layer. See the process of the CRNN model. Figure 1 Shown

[0043] 3. OCA multi-stage optical character recognition model.

[0044] 3.1 Introducing the Attention Mechanism

[0045] The attention mechanism was first widely used in the field of computer vision. The traditional Encoder-Decoder framework without the attention mechanism processes implicit information uniformly, thus losing key information and reducing the overall level of the model. Figure 2As shown in Figure 1. Different from the model without attention mechanism, the model with attention mechanism assigns different attention probabilities to each word in the input sentence according to its semantic information. Figure 3 shown.

[0046] The present invention uses the SoftAttention mechanism to construct an environment vector and embed it into the model for joint training with the model. The principle is shown in formulas (1) to (2):

[0047]

[0048] In formulas (1) to (2), Q is the query vector, K and V are usually the sequence itself, and d k represents the dimension of vector K, x represents any input data, and the summation range of the summation symbol in formula (2) is all input data.

[0049] When performing attention operations, we first calculate the similarity between vector V and each vector K, and normalize the results through the softmax function. Then, we use the obtained weights and the corresponding vector V for weighted summation to obtain the final Attention(Q,K,V).

[0050] 3.2 Knowledge Distillation.

[0051] Knowledge distillation is usually used for model compression to improve model operation efficiency. The flow chart is as follows: Figure 4 shown. Figure 4 The left side of the flowchart is the teacher network model TeacherModel(s), and the right side is the student network model StudentModel(s). The process is as follows:

[0052] First train TeacherModel(s); then use TeacherModel(s) to construct SoftTarget using the temperature parameter, that is, divide the predicted output of TeacherModel(s) by the temperature parameter T, and then perform a softmax operation to obtain a softened probability distribution SoftTarget with a value between 0 and 1; then use Soft Target and HardTarget to train the student model. This method transfers the model knowledge with large parameters to a model with small parameters, greatly improving the model operation efficiency. Therefore, in order to solve the problem of long operation time of multi-stage OCR models, the present invention uses knowledge distillation technology to improve the overall efficiency of the model while ensuring accuracy. Knowledge distillation can be considered as a generalized Softmax function, as shown in formula (3), and the cross entropy loss formula is shown in formula (4).

[0053]

[0054] In formulas (3) and (4), T is a tuning parameter. When T approaches 0 infinitely, the result is close to a one-hot vector (unique hot encoding); when T approaches infinity, the model can learn more possibilities.

[0055] In formula (3), ∑ j exp(z j / T) represents the exp(z j / T) values ​​are summed.

[0056] N is the number of input data sets, i and j are the numbers of input data sets; i Logits (vector) generated for the new model, q i represents the true probability, p i Represents the predicted probability.

[0057] 3.3OCA Multi-stage Optical Character Recognition Model

[0058] In order to solve the problem of model operation efficiency while ensuring model accuracy, the present invention constructs an OCA multi-stage optical character recognition model. The model flow chart is shown in Figure 5 The image enhancement step mainly includes three sub-steps: (1) image correction; (2) highlighting text features; (3) text sharpening. In the image correction stage, the image is first binarized, the text part is converted to white, and the image background is converted to black. Secondly, the minimum rotated text bounding box containing all the text is calculated, and the rotation angle of this bounding box is consistent with the rotation angle of the text. Finally, the image angle is adjusted to perform an affine transformation. The process is as follows Figure 6 shown.

[0059] In the text feature highlighting stage, the image contrast is first enhanced, and then the histogram normalization technology is used to process the image details. The histogram normalization formula is as follows: (5) to (7).

[0060]

[0061]

[0062] Histogram normalization is a linear transformation method. The input image is I, I(r,c) represents the grayscale value of the rth row and cth column of I, and the minimum grayscale level in I is recorded as I min , the maximum gray level is recorded as I max , that is, I(r,c)∈[I min ,I max ],O(r,c)∈[O min ,O max] is the grayscale value range of the output image. In the text sharpening stage, the sharpening operation on the text in the image is helpful to improve the final recognition accuracy of the model. Figure 7 Comparison of the effects before and after image enhancement.

[0063] In the text detection and text recognition stages, the SoftAttention mechanism is used to construct the environment vector and embed it into the model for joint training. This allows the words currently output by the model to receive different attention probability allocations, which helps improve the model recognition accuracy. In the knowledge distillation stage of text detection, the Teacher model uses the VGG16 and ResNet50 network structures in parallel in the convolutional layer, and concatenates the calculation results of the two. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used to extract serialized features. Finally, the calculation results are obtained through the fully connected layer and the RPN network, and the knowledge is distilled to obtain soft knowledge. The Student model uses the VGG16 network and adds a dropout mechanism. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used, and the calculation results are obtained through a single fully connected layer and the RPN network. When designing the loss function, the Student model simultaneously learns hard knowledge (hardtag) and soft knowledge (soft tag) such as Figure 4 shown.

[0064] In the knowledge distillation stage of text recognition, the Teacher model uses the VGG16 convolutional neural network to extract image features in the convolutional layer. Then, the bidirectional long short-term memory network based on the attention mechanism is used to extract text serialization features. Finally, the vector is passed to the transcription layer with the CTC loss function to obtain the result, and the result is subjected to knowledge distillation to obtain soft knowledge. The Student model uses a 3-layer CNN convolutional unit to extract image features. Then, the obtained vector is passed through the bidirectional long short-term memory neural network based on the attention mechanism. Finally, the transcription layer with the CTC loss function is used to obtain the vector result. When designing the loss function, the Student model learns both hard knowledge (hard tag) and soft knowledge (softtag).

[0065] In summary, the OCA multi-stage optical character recognition model first enhances the input image, and then uses the Teacher text detection model and the Teacher text recognition model to train the Student text detection model and the Student text recognition model. Finally, the image enhancement, the Student text detection model and the Student text recognition model are connected in series to obtain the OCA multi-stage optical character recognition model.

[0066] In the knowledge distillation stage of text detection, the Teacher model uses VGG16 and ResNet50 network structures in parallel in the convolutional layer, and concatenates the results of the two operations. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used to extract serialized features. Finally, the operation results are obtained through the fully connected layer and the RPN network, and the knowledge is distilled to obtain soft knowledge. The Student model uses the VGG16 network and adds a dropout mechanism. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used, and the operation results are obtained through a single fully connected layer and the RPN network. When designing the loss function, the Student model simultaneously learns hard knowledge (hard tag) and soft knowledge (soft tag) such as Figure 4 shown.

[0067] In the knowledge distillation stage of text recognition, the Teacher model uses the VGG16 convolutional neural network to extract image features in the convolutional layer. Then, the bidirectional long short-term memory network based on the attention mechanism is used to extract text serialization features. Finally, the vector is passed to the transcription layer with the CTC loss function to obtain the result, and the result is subjected to knowledge distillation to obtain soft knowledge. The Student model uses a 3-layer CNN convolutional unit to extract image features. Then, the obtained vector is passed through the bidirectional long short-term memory neural network based on the attention mechanism. Finally, the transcription layer with the CTC loss function is used to obtain the vector result. When designing the loss function, the Student model learns both hard knowledge (hard tag) and soft knowledge (softtag).

[0068] 2. Semantic classification architecture based on ACPmarked.

[0069] At present, audit reports obtained through crawler technology on the network platform contain a lot of information irrelevant to the audit business itself, such as company introduction, channel advertisements, etc. The present invention uses the ACPmarked multi-semantic fusion architecture to classify the digitized text and effectively remove information irrelevant to the audit business.

[0070] The CPmarken architecture mainly consists of three modules: Transformer module, attention-based bidirectional long short-term memory module, and convolution and pooling module. First, the Transformer module and the attention-based bidirectional long short-term memory module are arranged in parallel to enrich the word vector expression of the audit report, and then the convolution and pooling modules are used to further extract text features. Finally, the prediction results are output through the fully connected layer.

[0071] First, about 28G of text from Baidu Encyclopedia and headlines is crawled as the pre-training corpus for the Transformer module, and the word2vec model is used for training. Subsequently, the position encoding matrix is ​​introduced to obtain the word vector with position encoding. Then, the multi-head self-attention mechanism is used to encode the incoming text vector to obtain the word vector representation. The attention-based bidirectional long short-term memory module uses the wiki50-dimensional pre-trained word vector to vectorize the text, and then the attention mechanism is introduced to strengthen the attention of the output results of the bidirectional long short-term memory network to improve the classification accuracy of the model. The convolution and pooling module first splices the results output by the Transformer module and the attention-based bidirectional long short-term memory module, and then uses convolution kernels of different sizes to extract the relationship between the text. By setting the convolution kernel size to kernel=2, kernel=3, and kernel=4, the relationship between two words, three words, and four words is extracted to improve the overall effect of model recognition. ACPmarked multi-semantic fusion architecture such as Figure 8 shown.

[0072] Figure 8 The flowchart shown is described as follows:

[0073] First, the Input Embeddings module is used to vectorize the input text Input. Then the vectorized data is passed in parallel through the Transform module on the left and the long short-term memory module Lstm+Attention based on the attention mechanism on the right. Then the TextConv_unit module is used to merge the outputs of the Transform module and the Lstm+Attention module. Finally, the Max-Poolong module of the convolutional neural network is used to extract the spatial features of the vector, and finally the fully connected neural network Linear is used to complete the downstream task.

[0074] Example

[0075] In order to obtain real experimental data, this embodiment obtains high-quality audit documents through relevant audit departments, and crawls relevant audit work reports from Baidu Library and Baidu Tieba using crawler technology. In order to simulate the traditional audit process, the present invention prints audit documents and work reports, and uses a scanner to scan paper audit documents to build an image data resource library. The OCA model is used to identify the data in the image data resource library to obtain a digitally editable audit work report. The ACPmarked architecture is used to classify the audit work report and remove content in the text that is not related to the audit.

[0076] The first step is to build an image data resource library

[0077] The traditional audit process relies on manual work by staff to audit paper documents. This embodiment uses a scanner to scan the audited document to obtain an image. When scanning paper documents, problems such as (1) overexposure (2) image blur (3) excessive image rotation angle will inevitably occur. Fig. 9 As shown in the figure. Among them, overexposure and image blur will greatly affect the effect of subsequent model recognition, and the image needs to be re-scanned. If the image is rotated too much, it does not need to be re-scanned and can be used as an image source for subsequent work.

[0078] Step 2: OCA multi-stage optical character recognition model parameter selection

[0079] 2.1 Method selection in the OCA image enhancement stage

[0080] The images in the image resource library may have problems such as too large image rotation angle and unclear text in the images. This embodiment conducts experiments on 5,000 problem images and uses the OCA model without introducing the complete image enhancement stage to detect and recognize the text in the image. The accuracy is shown in formula (8), where TP is the number of positive examples that are also positive examples, TN is the number of negative examples that are also negative examples, FP is the number of positive examples that are predicted as negative examples, FN is the number of negative examples that are predicted as positive examples, the numerator is the number of all correct predictions, and the denominator is the total number of all data. The experimental results are shown in Tables (1) to (5).

[0081]

[0082] Table 1 Method selection in OCA image enhancement stage - image correction

[0083]

[0084]

[0085] Table 2 Method selection in the OCA image enhancement stage - highlighting text features

[0086] Image Problem Type Introducing prominent text features Average recognition accuracy of OCA model The rotation angle is too large yes 9.63％ The rotation angle is too large no 8.03％ Unclear text yes 85.55％ Unclear text no 82.75％ Poor overall image quality yes 9.73％ Poor overall image quality no 3.59％

[0087] Table 3 Method selection in OCA image enhancement stage - text sharpening

[0088] Image Problem Type Introducing text sharpening Average recognition accuracy of OCA model The rotation angle is too large yes 8.63％ The rotation angle is too large no 8.03％ Unclear text yes 83.25％ Unclear text no 82.75％ Poor overall image quality yes 8.71％ Poor overall image quality no 3.59％

[0089] Table 4 Method selection in OCA image enhancement stage - Fourier transform

[0090]

[0091]

[0092] Table 5 Method selection in OCA image enhancement stage - Laplacian operator

[0093] Image Problem Type Introducing the Laplace operator Average recognition accuracy of OCA model The rotation angle is too large yes 8.13％ The rotation angle is too large no 8.03％ Unclear text yes 82.82％ Unclear text no 82.75％ Poor overall image quality yes 3.92％ Poor overall image quality no 3.59％

[0094] In summary, this embodiment uses (1) image correction (2) text feature highlighting (3) text sharpening in the image enhancement stage, wherein the image correction stage uses image binarization, calculation of the minimum rotated text border, affine transformation and other operations to process image data. In the text feature highlighting stage, the image is subjected to contrast enhancement and histogram normalization. In the text sharpening stage, the image is sharpened using a sharpening algorithm. The final results are shown in Table 6.

[0095] Table 6 OCA image enhancement recognition results

[0096] Image Problem Type OCA Image Enhancement Average recognition accuracy of OCA model The rotation angle is too large yes 99.18％ The rotation angle is too large no 8.03％ Unclear text yes 99.29％ Unclear text no 82.75％ Poor overall image quality yes 99.15％ Poor overall image quality no 3.59％

[0097] The above results show that the use of OCA image enhancement technology has achieved good results in image enhancement of audit texts and has made significant contributions to subsequent OCA model recognition. Therefore, this model uses OCA image enhancement technology to process the image to be detected.

[0098] 2.2 OCA Knowledge Distillation Parameter Selection

[0099] The present invention can improve the model's operational efficiency while ensuring the model's accuracy through the knowledge distillation method. According to the difference between the original text and the recognized text (i.e. the text recognized by the model), the OCA multi-stage optical character recognition model operation results are calculated by the average precision P, average recall R and average F1 value (see formulas (9) to (11) respectively).

[0100]

[0101] Among them, x i Indicates the number of manually annotated labels, y i It represents the number of labels obtained by the algorithm, and N is the total number of audit texts.

[0102] The experimental results of distilling model knowledge using different temperatures T are shown in Table 7.

[0103] Table 7 Experimental results corresponding to temperature parameters

[0104] Temperature T Average accuracy of the teancher model Student model average accuracy 0.1 99.63％ 78.61％ 0.5 99.63％ 78.61％ 1 99.63％ 80.20％ 3 99.63％ 83.68％ 5 99.63％ 86.93％ 8 99.63％ 91.13％ 15 99.63％ 86.54％ 30 99.63％ 82.84％

[0105] From Table 7, we can see that when the temperature T = 8, the average accuracy of the student model is the highest. By reading the parameter adjustment techniques in related papers, the relu activation function is introduced into the fully connected layer of the student model. The experimental results corresponding to the temperature parameter T and the activation function are shown in Table 8, and the accuracy change during the experiment is shown in the figure Fig.10 shown.

[0106] Table 8 Experimental results corresponding to temperature parameters and activation functions

[0107] temperature TEANCHER model average accuracy Average accuracy of the STUDENT model 8 99.63％ 99.15％

[0108] Combine Table 8 with Fig.10 As shown, this embodiment finally selects the knowledge distillation temperature T as 8, and uses the relu activation function for the fully connected layer.

[0109] The third step is the OCA multi-stage optical character recognition model algorithm evaluation.

[0110] In the image data resource library, 20,000 manually scanned audit work reports were randomly selected, and experiments were conducted using PaddleOCR, tesseract, and EasyOCR as comparison models. The experimental results are shown in Table 9.

[0111] Table 9 Average F1 value calculation results

[0112] algorithm Average F1 value CPU-average computation time EASYOCR 0.604 72.3S TESSERACT 0.685 1.47S PADDLEOCR 0.781 5.78S OCA-TEACHER Model 0.996 9.33S OCA-STUDENT Model 0.991 4.31S

[0113] The data shows that the OCA-student model has achieved the best results in comprehensive processing efficiency compared to other models. The fundamental reason for this result is that the manually scanned audit work report is not completely horizontal, and there is an inevitable rotation angle in the manual scanning. Taking PaddleOCR as an example, it has a good effect in recognizing horizontal text, but it does not get good results for manually scanned audit work reports. The model results are visualized as follows: Fig.11 As shown. Therefore, the OCA-student model is defined as the final OCA multi-stage optical character recognition model.

[0114] Step 4: ACPmarked multi-semantic fusion architecture algorithm evaluation

[0115] The ACPmarken architecture is mainly composed of a Transformer module, an attention-based bidirectional long short-term memory module, and a convolution and pooling module. Its purpose is to extract semantic information related to the audit task from the digitized audit text. By summarizing the keywords involved in advertising slogans and company introductions, this paper further constructs a stop word dictionary based on the audit work report. Finally, based on the joint processing of the ACPmarken architecture and the stop word dictionary, a digitized high-quality audit text is formed. Due to the timeliness requirements of the audit work, the audit results need to be given in a short time. Therefore, the audit voucher digitization process involved in the present invention pays great attention to the efficiency of the model. The time efficiency ratio formula is introduced here to measure the efficiency ratio of the model accuracy and the time required, as shown in formula (12). Where T p is the time efficiency ratio, and t is the average time for cup to calculate each piece of data. This embodiment uses Bag-of-word, LSTM, AttentionbasedBi-LSTM, and BERT-Attention as comparison models for experiments, and the results are shown in Table 10.

[0116]

[0117] Table 10 Average F1 value calculation results

[0118] algorithm <![CDATA[Average F1 value]]> cup - average calculation time Time efficiency ratio Bag-of-word 0.405 5.3s 0.0764 LSTM 0.792 10.31s 0.0768 AttentionbaseBi-LSTM 0.817 12.01s 0.0680 BERT-Attention 0.926 22.39s 0.0413 ACPmarked 0.903 12.81s 0.0704

[0119] Table 10 shows that the ACPmaeked model is far superior to the Bag-of-word, LSTM, and Attentionbased Bi-LSTM models in terms of average F1 value. It is far superior to the BERT-Attention model in terms of time efficiency, and the average time for cup to calculate each piece of data is about half that of the BERT-Attention model. Therefore, the ACPmaeked multi-semantic fusion architecture is used to process the digitized audit text.

[0120] Step 5: Security and data protection methods of the audit process.

[0121] The digitization of audit vouchers also increases the risk of information leakage. It is extremely important to ensure the data security of digital audit information during transmission. This embodiment uses the method of encrypted graph transmission to ensure the security of audit data during storage and transmission, while improving quality and efficiency while taking security into account. The encrypted graph transmission process is as follows:

[0122] a) First, scan the paper audit documents and process them using the OCA model and ACPmaeked architecture to obtain the digitized audit text

[0123] b) Audit the digitized audit text and obtain the audit work report

[0124] c) Convert the text of the audit work report into an image to form an image version of the audit work report, and encrypt and compress the resulting image

[0125] d) During the transmission process, encrypted and compressed images are used to transmit information

[0126] e) The customer receives an encrypted and compressed image and uses the OCA model to convert the image into text, thus obtaining an audit work report securely.

[0127] In summary, the method of using encrypted image transmission is to transmit information in the format of encrypted images. The client obtains information through image-to-text and text-to-image operations. This effectively prevents a large amount of data from being transmitted in the form of byte streams in cyberspace and being obtained by criminals.

[0128] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: (steps of the method), the storage medium, such as: ROM / RAM, disk, CD, etc.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for digitizing audit vouchers, characterized by comprising: Build an image repository for audit documents; Use the OCA multi-stage optical character recognition model to perform text recognition on images in the image resource library to obtain digital audit documents; Use the ACPmarked framework to classify the text of digital audit documents and remove text content that is not relevant to the audit; The OCA multi-stage optical character recognition model is used to enhance the image first, including correcting the problem image, highlighting the text features, and sharpening the text; then the Teacher text detection model and the Teacher text recognition model are used to train the Student text detection model and the Student text recognition model; finally, the Student text detection model and the Student text recognition model are used to detect and recognize text in the image; The Teacher text detection model uses VGG16 and ResNet50 network structures in parallel in the convolutional layer, and concatenates the operation results of the two. In the recurrent network layer, a bidirectional long short-term memory network based on the attention mechanism is used to extract serialized features. The operation results are obtained through the fully connected layer and the RPN network, and the knowledge is distilled to obtain soft knowledge. The Student text detection model uses the VGG16 network and adds a dropout mechanism. In the recurrent network layer, a bidirectional long short-term memory network based on an attention mechanism is used to obtain the calculation results through a single fully connected layer and an RPN network. The Teacher text recognition model uses the VGG16 convolutional neural network to extract image features in the convolutional layer; then uses a bidirectional long short-term memory network based on the attention mechanism to extract text serialization features; finally, the vector is passed to the transcription layer with a CTC loss function to obtain the result, and the result is subjected to knowledge distillation to obtain soft knowledge; The Student text recognition model uses a 3-layer CNN convolutional unit to extract image features, then passes the obtained vector through a bidirectional long short-term memory neural network based on an attention mechanism, and finally uses a transcription layer with a CTC loss function to obtain the vector result; Use the ACPmarked framework to perform text classification on digitized audit documents, including: Crawl text from the Internet as pre-training corpus for the Transformer module and use the word2vec model for training; introduce the position encoding matrix of the recognized text to obtain word vectors with position encoding; The attention-based bidirectional long short-term memory module uses pre-trained word vectors to encode the input text vectors and obtain word vector representations; Convolution and pooling modules are used to extract text features and classify text.

2. The audit voucher digitization method according to claim 1, characterized in that: The method to build an image resource library for audit documents is: Use crawler technology to crawl audit documents on the Internet; print audit documents; scan paper audit documents to obtain images of audit documents.

Images