Bill information automatic identification and error correction method

Abstract

The invention discloses a bill information automatic identification and error correction method, and relates to the technical field of image processing, and the method comprises the steps: extracting red channel information from an original bill image through a self-adaptive binarization and color channel separation method, determining a threshold value according to local statistics, and achieving image preprocessing and preliminary region segmentation; by using the method, the seal area and the table area can be distinguished, and the problem that a fixed threshold value is sensitive to illumination change and noise interference is effectively relieved; a seal segmentation model based on a convolutional neural network CNN is constructed, school bill sample training is adopted, and a cross entropy loss function is introduced to constrain seal blurring and color deviation; the model extracts seal features through local convolution operation, fine segmentation of a seal region is realized, accurate region information is provided for straightening characters in the seal, and the defect that a traditional method is high in rule dependence is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for automatic bill information recognition and error correction. Background Art

[0002] Invoice recognition technology is widely used in finance, enterprises, and school financial management. Traditional recognition methods mostly use OCR technology and image preprocessing algorithms. They use Canny edge detection, Hough transform, and color channel separation to extract seals, tables, and other areas in the invoice, and then use rule-based algorithms to complete character straightening and text rearrangement.

[0003] In actual application, paper receipts often experience increased image noise due to factors such as scanning angle, ambient light, and paper wear. Irregular arrangements and interference patterns are often encountered when locating and processing the seal area, leading to deviations in the extraction of key text information. Furthermore, the table portion of the receipt has a complex structure and diverse field arrangements. Traditional rearrangement methods based on a fixed row and column order can easily cause numbers, dates, and amounts to become disordered after recognition, which is extremely important for data management in schools. In the school's tuition collection and reimbursement documents, if the order of key information such as student numbers, names, or amounts becomes disordered, it will bring additional manual verification burdens to accounting processing and affect the final statistics and decision-making of the data.

[0004] To this end, some traditional solutions use statistical analysis and preset thresholds to deal with the above problems. However, their adaptability and robustness are still insufficient in the face of problems such as image blur, seal color attenuation, and unclear table boundaries. The automatic recognition process is difficult to take into account both details and overall logic, resulting in recognition errors or data misalignment in some bills during processing. Therefore, there is an urgent need for an automatic bill information recognition and error correction method that integrates multimodal information and targets the complex layout and interference factors in the bills, so as to truly meet the strict requirements of school financial management for key information such as tuition and reimbursement. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a method for automatic recognition and error correction of bill information to solve the problem that the traditional Canny and Hough transform-based methods are difficult to stably locate the circular seal in the current red seal area due to paper aging, color attenuation and background interference, resulting in a low recognition rate of key information in the seal; in addition, the information arrangement in the table area of ​​the tuition bill is complex, and the order of fields such as student number, name, amount, date, etc. is often disordered after OCR recognition, affecting subsequent automatic input.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The embodiment of the present invention provides a method for automatic identification and error correction of bill information, which includes:

[0009] Step S1, performing adaptive binarization and color channel separation on the bill image to obtain preliminary images of the seal area and the form area; and preprocessing the preliminary image, including noise suppression and brightness adjustment;

[0010] Step S2: constructing a seal segmentation model based on a convolutional neural network (CNN), using school receipt samples for training, and setting a loss function constraint to handle seal blur and color deviation during the training process;

[0011] Step S3, extracting the table border using morphological operations on the table area obtained in step S1; using optical character recognition (OCR) to recognize the text in the cells of the table and simultaneously recording the two-dimensional position information of the text;

[0012] Step S4: Utilize the document understanding LayoutLMv3 model and the gated recurrent unit (GRU) to construct a multimodal information fusion model, perform structural correction, and form a structured representation of the overall bill information.

[0013] Step S5: Record the structured bill information corrected in step S4 and establish data association with the existing school financial system or database.

[0014] As a preferred solution of the method for automatic bill information recognition and error correction of the present invention, in which: step S1, the step of adaptively binarizing and separating the color channels of the bill image is:

[0015] Input bill image For preprocessing, first extract the red channel from the image. The extraction process is expressed as:

[0016] ,

[0017] in, Represented in pixel coordinates The original image pixel value at Indicated in coordinates The red channel pixel value extracted at is the red channel identifier,

[0018] Using adaptive threshold right Perform binarization to obtain a binary image :

[0019] ,

[0020] in, Indicated in coordinates The binarization result at is the adaptive threshold calculated based on local statistics.

[0021] As a preferred solution of the method for automatic bill information recognition and error correction of the present invention, in step S2, the step of constructing a seal segmentation model based on a convolutional neural network CNN is as follows:

[0022] In the image preprocessed in step S1, the seal area is segmented using a convolutional neural network. The convolution operation is defined as:

[0023] ,

[0024] in, Indicates that the output feature map is at coordinate The value at represents the activation function, Indicates the convolution kernel in Row, No. The weight of the column, is the pixel value of the local area of ​​the input image, is the convolutional layer bias, is the size of the convolution kernel,

[0025] The cross entropy loss function is used during training:

[0026] ,

[0027] in, represents the total loss value, is the true label, indicating the coordinate Whether it belongs to the seal area, For the model in coordinates The predicted probability at ;

[0028] School receipt samples are selected during training, and the loss function is used to tune the model parameters to impose constraints on the blur and color deviation problems in the seal area.

[0029] As a preferred solution of the method for automatic bill information recognition and error correction described in the present invention, in step S3, the recognition result is input into the document image analysis Layout-Parser tool, and the text is rearranged according to the visual position and contextual relationship to form a text sequence in a logical order.

[0030] As a preferred solution of the method for automatic bill information recognition and error correction of the present invention, in step S3, the step of extracting the table border by using morphological operation on the table area obtained in step S1 is as follows:

[0031] For the table area obtained in step S1, the table border is first extracted using morphological operations. The morphological erosion operation is defined as:

[0032] ,

[0033] in, Indicates the result of the erosion operation at coordinates The value at The input image is at the offset position The pixel value at is a structural element, and is the corresponding offset in the structural element;

[0034] The morphological dilation operation is defined as:

[0035] ,

[0036] in, Indicates that the result of the expansion operation is in the coordinate The value at .

[0037] As a preferred embodiment of the method for automatic bill information recognition and error correction of the present invention, in step S3, the method of using optical character recognition (OCR) to recognize the text in the cell of the table and simultaneously record the two-dimensional position information of the text includes:

[0038] After extracting the table border using corrosion and expansion operations, each cell image is recorded as , ,in is the total number of cells, for each cell image Optical character recognition (OCR) algorithm is used for text recognition, denoted as ,

[0039] in, Represents a cell The recognized text in It is the OCR recognition function;

[0040] At the same time, the two-dimensional position information of the cell is recorded as ,

[0041] in, and Cells horizontal and vertical coordinates in the image;

[0042] Input all cell recognition results and their position information into the Layout-Parser tool to generate a rearranged text sequence:

[0043] in, Represents the overall text sequence after rearrangement, Layout-Parser function, used to rearrange text according to visual position and context logic.

[0044] As a preferred solution of the method for automatic identification and error correction of bill information described in the present invention, in which: in step S4, the multimodal information fusion model respectively receives the seal area text and boundary coordinates obtained in step S2 and the table area text and two-dimensional position information obtained in step S3; the fields in the bill involving student number, tuition fee and reimbursement amount are structurally corrected according to the school's financial management regulations to form a structured representation of the overall bill information.

[0045] As a preferred solution of the method for automatic bill information recognition and error correction described in the present invention, in step S4, the step of using the document understanding LayoutLMv3 model and the gated recurrent unit GRU to construct a multimodal information fusion model is as follows:

[0046] When constructing a multimodal information fusion model, the text information and the two-dimensional position information are jointly embedded. The joint embedding formula is:

[0047] ,

[0048] in, Indicates time The joint embedding of For the moment Text embedding, For the moment Layout embedding, i.e. two-dimensional position information embedding;

[0049] The gated recurrent unit GRU is used to model the joint embedding sequence, and its recursive formula is:

[0050] ,

[0051] in, Indicates time The hidden state of Indicates time The hidden state of To update the gate, To reset the gate, is the weight matrix of the current input is the weight matrix of the historical state, is the bias vector, is the hyperbolic tangent function.

[0052] As a preferred solution of the method for automatic bill information recognition and error correction described in the present invention, in step S4, the step of using the document understanding LayoutLMv3 model and the gated recurrent unit GRU to construct a multimodal information fusion model further includes:

[0053] Update Gate and reset gate The calculation formulas are:

[0054] ,

[0055] ,

[0056] in, and are the input weight matrices of the update gate and reset gate respectively, and are the state weight matrices of the update gate and reset gate respectively, and are the bias vectors for the update gate and reset gate, respectively. Represents the sigmoid function;

[0057] After multimodal information fusion, the fields in the bill, such as student ID, tuition, and reimbursement amount, are adjusted using a structural correction function:

[0058] like ,but ,otherwise ,

[0059] in, Represents the original field value, is the lower limit allowed for this field, is the upper limit allowed for this field. The corrected field value.

[0060] As a preferred solution of the method for automatic identification and error correction of bill information described in the present invention, the structured data of the structured bill information includes the content of each field and the corresponding position information.

[0061] The beneficial effects of the present invention are as follows: the present invention extracts red channel information from the original bill image through adaptive binarization and color channel separation methods, determines the threshold based on local statistics, and realizes image preprocessing and preliminary region segmentation; using this method, the seal area and the table area can be distinguished, effectively alleviating the problem that the fixed threshold is sensitive to illumination changes and noise interference.

[0062] The present invention constructs a seal segmentation model based on a convolutional neural network (CNN), uses school bill samples for training, and introduces a cross-entropy loss function to constrain seal blur and color deviation. The model extracts seal features through local convolution operations, achieves fine segmentation of the seal area, and provides accurate regional information for straightening characters inside the seal, overcoming the shortcomings of traditional methods that are highly rule-dependent.

[0063] For the table area, the present invention uses morphological erosion and dilation operations to extract the table border, and uses optical character recognition (OCR) technology to recognize the text on the extracted cell image, while recording the two-dimensional position information of each cell; then, the OCR recognition results are rearranged through the Layout-Parser tool to restore the correct logical order of each field in the bill, effectively avoiding text confusion.

[0064] The present invention uses the document understanding model LayoutLMv3 and the gated recurrent unit GRU to construct a multimodal information fusion model, and jointly encodes the text and position information of the seal area and the table area. In combination with the school's financial management requirements, a preset correction function is used to perform structural correction on fields such as the student number, tuition fee, and reimbursement amount to form a structured representation of the overall bill information. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 The figure is a flow chart of the method for automatic bill information recognition and error correction of the present invention. DETAILED DESCRIPTION

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0069] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0070] Example 1, with reference to Figure 1 This embodiment provides a method for automatic bill information recognition and error correction, including:

[0071] Step S1, performing adaptive binarization and color channel separation on the bill image to obtain preliminary images of the seal area and the form area; and preprocessing the preliminary image, including noise suppression and brightness adjustment;

[0072] Step S1, the steps of adaptively binarizing and separating the color channels of the bill image are as follows:

[0073] Input bill image For preprocessing, first extract the red channel from the image. The extraction process is expressed as:

[0074] ,

[0075] in, Represented in pixel coordinates The original image pixel value at Indicated in coordinates The red channel pixel value extracted at is the red channel identifier,

[0076] Using adaptive threshold right Perform binarization to obtain a binary image :

[0077] ,

[0078] in, Indicated in coordinates The binarization result at is the adaptive threshold calculated based on local statistics;

[0079] Specifically, in step S1, the bill image is preprocessed, and the seal and form areas are separated using the information of the red channel in the image. After red channel extraction, the obtained It can highlight the special color part of the bill and determine the adaptive threshold based on local brightness statistics. , so that the binarization operation adapts to the illumination changes in different areas, thereby generating a clear binary image ,This binary image provides clear boundary information for the subsequent fine segmentation of ,the seal and form areas;

[0080] Step S2: constructing a seal segmentation model based on a convolutional neural network (CNN), using school receipt samples for training, and setting a loss function constraint to handle seal blur and color deviation during the training process;

[0081] In step S2, the steps of constructing a seal segmentation model based on the convolutional neural network CNN are as follows:

[0082] In the image preprocessed in step S1, the seal area is segmented using a convolutional neural network. The convolution operation is defined as:

[0083] ,

[0084] in, Indicates that the output feature map is at coordinate The value at represents the activation function, Indicates the convolution kernel in Row, No. The weight of the column, is the pixel value of the local area of ​​the input image, is the convolutional layer bias, is the size of the convolution kernel,

[0085] The cross entropy loss function is used during training:

[0086] ,

[0087] in, represents the total loss value, is the true label, indicating the coordinate Whether it belongs to the seal area, For the model in coordinates The predicted probability at ;

[0088] During training, we selected school receipt samples and used this loss function to tune the model parameters, imposing constraints on blur and color deviation in the stamp area.

[0089] Specifically, in step S2, the seal area is segmented by a convolutional neural network, using local pixel information and convolution kernel parameters. Extract image features; introduce a cross-entropy loss function to measure the deviation between the model's predictions and the true labels, providing feedback for model training. Using school receipt samples as training data, the network can be effectively adjusted to address the blur and color deviation issues in the stamp area of ​​the receipts.

[0090] The seal region segmentation result obtained in this step provides regional information for subsequent straightening and recognition of characters inside the seal. Here, deep learning is used to automatically learn the characteristics of bill seals, overcoming the limitations of traditional methods that rely heavily on thresholds and rules.

[0091] Step S3, extracting the table border using morphological operations on the table area obtained in step S1; using optical character recognition (OCR) to recognize the text in the cells of the table and simultaneously recording the two-dimensional position information of the text;

[0092] In step S3, the recognition results are input into the document image analysis Layout-Parser tool, which rearranges the text according to the visual position and contextual relationship to form a text sequence that conforms to a logical order;

[0093] In step S3, the steps of extracting the table border by using morphological operations on the table area obtained in step S1 are as follows:

[0094] For the table area obtained in step S1, the table border is first extracted using morphological operations. The morphological erosion operation is defined as:

[0095] ,

[0096] in, Indicates the result of the erosion operation at coordinates The value at The input image is at the offset position The pixel value at is a structural element, and is the corresponding offset in the structural element;

[0097] The morphological dilation operation is defined as:

[0098] ,

[0099] in, Indicates that the result of the expansion operation is in the coordinate The value at

[0100] In step S3, optical character recognition (OCR) is used to recognize the text in the cells of the table and record the two-dimensional position information of the text. The method includes:

[0101] After extracting the table border using corrosion and expansion operations, each cell image is recorded as , ,in is the total number of cells, for each cell image Optical character recognition (OCR) algorithm is used for text recognition, denoted as ,

[0102] in, Represents a cell The recognized text in It is the OCR recognition function;

[0103] At the same time, the two-dimensional position information of the cell is recorded as ,

[0104] in, and Cells horizontal and vertical coordinates in the image;

[0105] Input all cell recognition results and their position information into the Layout-Parser tool to generate a rearranged text sequence:

[0106] in, Represents the overall text sequence after rearrangement, Layout-Parser function, used to rearrange text according to visual position and context logic;

[0107] Specifically, morphological operations are used here to extract cell borders from the table area, and the erosion operation formula is used to extract the cell borders from the structural elements. The offsets in and Take the local minimum value to filter the background noise, and the dilation operation takes the local maximum value to enhance the boundary information. The extracted cell image is recorded as Then, use OCR recognition function Convert the text in each cell and record its two-dimensional position information , through the Layout-Parser tool Rearrange the recognition results of each cell to obtain a text sequence with the correct logical order ;

[0108] Step S4: Utilize the document understanding LayoutLMv3 model and the gated recurrent unit (GRU) to construct a multimodal information fusion model, perform structural correction, and form a structured representation of the overall bill information.

[0109] In step S4, the multimodal information fusion model receives the seal area text and boundary coordinates obtained in step S2 and the table area text and two-dimensional position information obtained in step S3; the fields related to the student ID, tuition, and reimbursement amount in the bill are structurally corrected according to the school's financial management regulations to form a structured representation of the entire bill information;

[0110] In step S4, the steps of using the document understanding LayoutLMv3 model and the gated recurrent unit GRU to build a multimodal information fusion model are as follows:

[0111] When constructing a multimodal information fusion model, the text information and the two-dimensional position information are jointly embedded. The joint embedding formula is:

[0112] ,

[0113] in, Indicates time The joint embedding of For the moment Text embedding, For the moment Layout embedding, i.e. two-dimensional position information embedding;

[0114] The gated recurrent unit GRU is used to model the joint embedding sequence, and its recursive formula is:

[0115] ,

[0116] in, Indicates time The hidden state of Indicates time The hidden state of To update the gate, To reset the gate, is the weight matrix of the current input is the weight matrix of the historical state, is the bias vector, is the hyperbolic tangent function;

[0117] Update Gate and reset gate The calculation formulas are:

[0118] ,

[0119] ,

[0120] in, and are the input weight matrices of the update gate and reset gate respectively, and are the state weight matrices of the update gate and reset gate respectively, and are the bias vectors for the update gate and reset gate, respectively. Represents the sigmoid function;

[0121] After multimodal information fusion, the fields in the bill, such as student ID, tuition, and reimbursement amount, are adjusted using a structural correction function:

[0122] like ,but ,otherwise ,

[0123] in, Represents the original field value, is the lower limit allowed for this field, is the upper limit allowed for this field. is the corrected field value;

[0124] Specifically, step S4 uses text and layout information to construct a joint embedding. It is composed of text embedding and two-dimensional position information embedding, so that the representation of each moment contains both text semantics and position information; the sequence data is modeled by the GRU model, and the hidden state Reflecting the relationship between the current moment and the previous state, the update gate and reset gate in GRU respectively regulate the transmission and reset of historical information; the multimodal fusion model uniformly encodes the text information obtained from the seal area and the table area to represent the logical relationship between the fields in the bill. The structural correction function can judge the original field value by Whether abnormal data is uniformly adjusted within the preset upper and lower bounds to form structured data that complies with the school's financial management regulations;

[0125] Step S5, recording the structured bill information corrected in step S4 and establishing a data association with the existing school financial system or database;

[0126] Structured bill information, its structured data includes the content of each field and the corresponding location information.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatic bill information recognition and error correction, characterized by: include, Step S1, performing adaptive binarization and color channel separation on the bill image to obtain preliminary images of the seal area and the form area; and preprocessing the preliminary image, including noise suppression and brightness adjustment; Step S2: constructing a seal segmentation model based on a convolutional neural network (CNN), using school receipt samples for training, and setting a loss function constraint to handle seal blur and color deviation during the training process; Step S3, extracting the table border using morphological operations on the table area obtained in step S1; using optical character recognition (OCR) to recognize the text in the cells of the table and simultaneously recording the two-dimensional position information of the text; Step S4: Utilize the document understanding LayoutLMv3 model and the gated recurrent unit (GRU) to construct a multimodal information fusion model, perform structural correction, and form a structured representation of the overall bill information. Step S5: Record the structured bill information corrected in step S4 and establish data association with the existing school financial system or database.

2. The method for automatic bill information recognition and error correction according to claim 1, characterized in that: Step S1, the step of performing adaptive binarization and color channel separation on the bill image is as follows: Input bill image For preprocessing, first extract the red channel from the image. The extraction process is expressed as: ， in, Represented in pixel coordinates The original image pixel value at Indicated in coordinates The red channel pixel value extracted at is the red channel identifier, Using adaptive threshold right Perform binarization to obtain a binary image : ， in, Indicated in coordinates The binarization result at is the adaptive threshold calculated based on local statistics.

3. The method for automatic bill information recognition and error correction according to claim 2, characterized in that: In step S2, the step of constructing a seal segmentation model based on a convolutional neural network (CNN) is as follows: In the image preprocessed in step S1, the seal area is segmented using a convolutional neural network. The convolution operation is defined as: ， in, Indicates that the output feature map is at coordinate The value at represents the activation function, Indicates the convolution kernel in Row, No. The weight of the column, is the pixel value of the local area of ​​the input image, is the convolutional layer bias, is the size of the convolution kernel, The cross entropy loss function is used during training: ， in, represents the total loss value, is the true label, indicating the coordinate Whether it belongs to the seal area, For the model in coordinates The predicted probability at ; School receipt samples are selected during training, and the loss function is used to tune the model parameters to impose constraints on the blur and color deviation problems in the seal area.

4. The method for automatic bill information recognition and error correction according to claim 3, characterized in that: In step S3, the recognition results are input into the document image analysis Layout-Parser tool, which rearranges the text according to the visual position and contextual relationship to form a text sequence that conforms to a logical order.

5. The method for automatic bill information recognition and error correction according to claim 4, characterized in that: In step S3, the step of extracting the table border by using morphological operation on the table area obtained in step S1 is as follows: For the table area obtained in step S1, the table border is first extracted using morphological operations. The morphological erosion operation is defined as: ， in, Indicates the result of the erosion operation at coordinates The value at The input image is at the offset position The pixel value at is a structural element, and is the corresponding offset in the structural element; The morphological dilation operation is defined as: ， in, Indicates that the result of the expansion operation is in the coordinate The value at .

6. The method for automatic bill information recognition and error correction according to claim 5, characterized in that: In step S3, the method of using optical character recognition (OCR) to recognize the text in the cell of the table and simultaneously recording the two-dimensional position information of the text includes: After extracting the table border using corrosion and expansion operations, each cell image is recorded as , ,in is the total number of cells, for each cell image Optical character recognition (OCR) algorithm is used for text recognition, denoted as , in, Represents a cell The recognized text in It is the OCR recognition function; At the same time, the two-dimensional position information of the cell is recorded as , in, and Cells horizontal and vertical coordinates in the image; Input all cell recognition results and their position information into the Layout-Parser tool to generate a rearranged text sequence: in, Represents the overall text sequence after rearrangement, It is a Layout-Parser function used to rearrange text according to visual position and contextual logic.

7. The method for automatic bill information recognition and error correction according to claim 6, characterized in that: In step S4, the multimodal information fusion model receives the seal area text and boundary coordinates obtained in step S2 and the table area text and two-dimensional position information obtained in step S3 respectively; the fields related to student number, tuition fee and reimbursement amount in the bill are structurally corrected according to the school's financial management regulations to form a structured representation of the overall bill information.

8. The method for automatic bill information recognition and error correction according to claim 7, characterized in that: In step S4, the step of using the document understanding LayoutLMv3 model and the gated recurrent unit GRU to construct a multimodal information fusion model is as follows: When constructing a multimodal information fusion model, the text information and the two-dimensional position information are jointly embedded. The joint embedding formula is: ， in, Indicates time The joint embedding of For the moment Text embedding, For the moment Layout embedding; The gated recurrent unit GRU is used to model the joint embedding sequence, and its recursive formula is: ， in, Indicates time The hidden state of Indicates time The hidden state of To update the gate, To reset the gate, is the weight matrix of the current input is the weight matrix of the historical state, is the bias vector, is the hyperbolic tangent function.

9. The method for automatic bill information recognition and error correction according to claim 8, characterized in that: In step S4, the step of constructing a multimodal information fusion model using the document understanding LayoutLMv3 model and the gated recurrent unit GRU further includes: Update Gate and reset gate The calculation formulas are: ， ， in, and are the input weight matrices of the update gate and reset gate respectively, and are the state weight matrices of the update gate and reset gate respectively, and are the bias vectors for the update gate and reset gate, respectively. Represents the sigmoid function; After multimodal information fusion, the fields in the bill, such as student ID, tuition, and reimbursement amount, are adjusted using a structural correction function: like ,but ,otherwise , in, Represents the original field value, is the lower limit allowed for this field, is the upper limit allowed for this field. The corrected field value.

10. The method for automatic bill information recognition and error correction according to claim 9, characterized in that: The structured data of the structured bill information includes the content of each field and the corresponding position information.

Images