Image correction model acquisition method, processing method, device, equipment and medium

Abstract

The present disclosure provides an acquisition method, processing method, device, equipment and medium for an image correction model, which relates to the field of artificial intelligence, especially to the fields of big data, computer vision and image processing. The specific implementation scheme is as follows: inputting training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point; the training sample data includes a deformed document image and annotation information; the annotation information includes information associated with the positions of the initial deformation point and the initial reference point respectively; based on the annotation information, the first prediction information, the second prediction information and the preset weight of the initial deformation point, a prediction loss value is determined; the deformed document image contains a preset initial deformation point located in the edge area of ​​the image, as well as other remaining initial deformation points other than the preset initial deformation point; the weight of the preset initial deformation point is greater than the weight of the other remaining initial deformation points; and the network parameters of the preset network model are adjusted according to the prediction loss value.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the fields of big data, computer vision, and image processing, and specifically provides a method for obtaining an image correction model, a processing method, an apparatus, a device, and a medium. Background Art

[0002] Automatic document image recognition has become an indispensable function in various business processes. However, due to factors such as the physical deformation of the paper, the shooting environment, and the camera angle, the captured document images often exhibit tilt, distortion, and wrinkles. This makes it difficult to detect and recognize the document content in the image, seriously affecting the accuracy of downstream tasks such as text detection and recognition.

[0003] To reduce the impact of document image deformation on downstream tasks, the document content in the document image needs to be corrected to improve the accuracy of downstream tasks. Currently, deep learning-based document correction methods can be divided into optical flow prediction methods and initial deformation point prediction methods based on the correction principle. Among them, the optical flow prediction method is based on an encoder and decoder. This method has a large number of parameters, a long prediction time, and its performance often does not meet user requirements. The initial deformation point prediction method inputs the complete image into the correction model to predict the initial deformation point, and then corrects the document content in the image based on the predicted initial deformation point.

[0004] Initial deformation point prediction methods introduce a cluttered background, requiring the correction model to locate the foreground and background in the image. Furthermore, these methods rely heavily on specific datasets for the application scenario. Because insurance claim document images differ from conventional images in terms of document content, size, and background, models trained on datasets containing conventional images often produce poor correction results. Model training often uses a relatively small number of document images of a given type, making it difficult to guarantee correction results using models trained on a small number of document images. Summary of the Invention

[0005] The present disclosure provides an acquisition method, processing method, device, equipment and medium for an image correction model.

[0006] The present disclosure solves the above technical problems through the following technical solutions:

[0007] According to one aspect of the present disclosure, a method for obtaining a document image correction model is provided, the method comprising:

[0008] Acquire training sample data; wherein the training sample data includes deformed document images and annotation information;

[0009] The annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with positions of the initial deformation point and the initial reference point, respectively;

[0010] Inputting the training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point;

[0011] Determining a prediction loss value based on the annotation information, the first prediction information, the second prediction information, and a preset weight of the initial deformation point;

[0012] The deformed document image includes a plurality of preset deformation points located in the edge area of ​​the image, and other remaining initial deformation points except the plurality of preset deformation points;

[0013] The weight of the preset deformation point is greater than the weights of other remaining initial deformation points;

[0014] The network parameters of the preset network model are adjusted according to the predicted loss value.

[0015] According to another aspect of the present disclosure, a document image processing method is provided, the document image processing method comprising:

[0016] Obtaining a document image correction model using the above acquisition method;

[0017] Acquire the document image to be corrected;

[0018] Obtaining, through the document image correction, first target information of an initial deformation point in the document to be corrected and second target information of an initial reference point;

[0019] The document image to be corrected is corrected according to the first target information and the second target information.

[0020] According to another aspect of the present disclosure, a device for acquiring a document image correction model is provided, the device comprising:

[0021] A training sample acquisition module, configured to acquire training sample data; wherein the training sample data includes a deformed document image and annotation information;

[0022] The annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with positions of the initial deformation point and the initial reference point, respectively;

[0023] A first prediction module, configured to input the training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point;

[0024] a loss value determination module, configured to determine a predicted loss value based on the annotation information, the first prediction information, the second prediction information, and a preset weight of the initial deformation point;

[0025] The deformed document image includes a plurality of preset deformation points located in the edge area of ​​the image, and other remaining initial deformation points except the plurality of preset deformation points;

[0026] The weight of the preset deformation point is greater than the weights of other remaining initial deformation points;

[0027] A parameter adjustment module is used to adjust the network parameters of the preset network model according to the predicted loss value.

[0028] According to another aspect of the present disclosure, there is provided a document image processing apparatus, the document image processing apparatus comprising:

[0029] A model training module, used to obtain a document image correction model using the above-mentioned acquisition device;

[0030] An image acquisition module, used for acquiring an image of a document to be corrected;

[0031] A second prediction module is configured to obtain, through the document image correction model, first target information of an initial deformation point in the document to be corrected and second target information of an initial reference point;

[0032] The document image correction module is used to correct the document image to be corrected according to the first target information and the second target information.

[0033] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0034] at least one processor; and

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0037] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the above method.

[0038] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0039] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0041] Figure 1 A flowchart of a method for obtaining a document image correction model provided in an embodiment of the present disclosure;

[0042] Figure 2 A flowchart for calculating a first predicted loss value provided in an embodiment of the present disclosure;

[0043] Figure 3 A flowchart for calculating a second predicted loss value provided by an embodiment of the present disclosure;

[0044] Figure 4 A flowchart of calculating a reference point loss value provided in an embodiment of the present disclosure;

[0045] Figure 5 A network architecture diagram of a preset network model provided in an embodiment of the present disclosure;

[0046] Figure 6 A flowchart of obtaining training sample data provided in an embodiment of the present disclosure;

[0047] Figure 7 A schematic diagram of an initial reference point and a corresponding initial deformation point provided in an embodiment of the present disclosure;

[0048] Figure 8 A flowchart of a document image processing method provided by an embodiment of the present disclosure;

[0049] Figure 9 A schematic diagram of an original captured image provided by an embodiment of the present disclosure;

[0050] Figure 10 A schematic diagram of a corrected document image provided by an embodiment of the present disclosure;

[0051] Figure 11 A schematic diagram of the insurance claim process provided by an embodiment of the present disclosure;

[0052] Figure 12 A schematic diagram of the process of correcting medical bill images provided in an embodiment of the present disclosure;

[0053] Figure 13 A schematic diagram of a module of a device for acquiring a document image correction model provided by an embodiment of the present disclosure;

[0054] Figure 14 A schematic diagram of modules of a document image processing device provided by an embodiment of the present disclosure;

[0055] Figure 15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0057] Figure 1 The present disclosure provides a method for obtaining a document image correction model. Figure 1 , the method for obtaining the document image correction model of this embodiment includes:

[0058] S101, obtaining training sample data;

[0059] The training sample data includes a deformed document image and annotation information, wherein the annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with the positions of the initial deformation point and the initial reference point, respectively.

[0060] It should be noted that the position of the initial reference point in the deformed document image is used to represent the position of the initial deformation point after correction.

[0061] S102: Input the training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point;

[0062] The prediction information includes the initial deformation point and reference position-related information.

[0063] S103, determining a prediction loss value based on the annotation information, the first prediction information, the second prediction information, and a preset weight of the initial deformation point;

[0064] The deformed document image includes a plurality of preset deformation points located at the edge of the image and other remaining initial deformation points except the preset deformation points. The weight of the preset deformation points is greater than the weight of the other remaining initial deformation points.

[0065] It should be noted that the image edge area can be set according to actual needs.

[0066] S104: Adjust the network parameters of the preset network model according to the predicted loss value.

[0067] The above acquisition method can obtain a document image correction model that meets the preset requirements by iteratively training the preset network model, that is, repeatedly executing steps S102 to S104.

[0068] In this embodiment, a preset network model is used to obtain prediction information of initial reference points corresponding to initial deformation points in a deformed document image. A prediction loss value is calculated based on the prediction information and the corresponding annotation information, and the network parameters of the preset network model are then adjusted based on the prediction loss value. Specifically, initial deformation points located in the image edge region of the deformed document image are used as preset deformation points, and preset weights of several preset deformation points are set to be greater than the preset weights of the remaining initial deformation points. This improves the regression accuracy of initial deformation points located in the image edge region of the deformed document image, thereby improving the prediction accuracy of initial deformation points located in the image edge region of the deformed document image, thereby ensuring the final correction effect of the deformed document image.

[0069] In one feasible solution, the preset weight of the preset deformation point is positively correlated with the first distance; wherein the first distance is the distance from the preset deformation point to the center of the deformed document image.

[0070] That is, the farther the preset deformation point is from the midpoint of the deformed document image, the greater its preset weight, which can further improve the regression accuracy of the preset position association information of the initial deformation point located at the super-edge of the deformed document image, and then improve the prediction accuracy of the super-edge of the deformed document image, so as to ensure that the image contour of the document image obtained by subsequent correction is accurate, the image resolution is clear, and the document content is complete.

[0071] In one feasible solution, step S103 determines the deformation point loss value and the reference point loss value based on the annotation information, the first prediction information, the second prediction information and the preset weight of the initial deformation point, and then calculates the predicted loss value based on the deformation point loss value and the reference point loss value.

[0072] Specifically, step S103 includes:

[0073] S1031: Determine a deformation point loss value based on the first prediction information of the initial deformation point, the first position association information, and a preset weight;

[0074] S1032: Determine a reference point loss value based on the second prediction information and the second position association information of the initial reference point;

[0075] S1033. Calculate a predicted loss value based on the deformation point loss value and the reference point loss value.

[0076] In this scheme, increasing the preset weights of several preset deformation points located in the image edge area of ​​the deformed document image can improve the regression accuracy of the prediction information of the initial deformation points located at the edge of the deformed document image, thereby improving the prediction accuracy of the initial deformation points located at the edge of the deformed document image.

[0077] In order to obtain the deformation point loss value, in an implementable solution, the first prediction information of the initial deformation point includes the first predicted position of the initial deformation point, and the first position association information of the initial deformation point includes the first marked position of the initial deformation point.

[0078] Based on the first prediction information and the first position association information of the initial deformation point, the loss value between the first prediction position and the first marked position of each initial deformation point is calculated. Figure 2 As shown, step S1031 includes:

[0079] S201, calculating and obtaining a first prediction deviation value for each initial deformation point; wherein the first prediction deviation value is the difference between the first prediction position and the corresponding first marked position;

[0080] S202: Determine a first predicted loss value according to a first predicted deviation value and a preset weight of each initial deformation point.

[0081] The first prediction deviation value is actually the distance between the first predicted position and the corresponding first marked position.

[0082] In this solution, a first prediction loss value is calculated based on the difference between the first predicted position of the initial deformation point and the corresponding first marked position, which intuitively demonstrates the prediction effect of the current preset network model on the initial deformation point.

[0083] Specifically, step S202 includes:

[0084] S2021: Obtain a comparison result between the first predicted deviation value and a preset deviation threshold, and determine a first initial loss value for each initial deformation point based on the comparison result;

[0085] S2022. Calculate and obtain a weighted sum of a first initial loss value of each initial deformation point and a corresponding preset weight;

[0086] S2023. Calculate a first average value of the weighted sum, and use the first average value as a first predicted loss value.

[0087] In this solution, different first initial loss values ​​are assigned to the initial deformation point according to the first predicted deviation value between the first predicted position of the initial deformation point and the corresponding first marked position, and the regression accuracy and regression efficiency of the model can be controlled by presetting the deviation threshold.

[0088] For example, there are N initial deformation points in the deformed document image, the first predicted position of the initial deformation point is represented as P1, the first annotated position of the initial deformation point is represented as P2, and the preset deviation threshold is represented as β. When the first initial loss value of each initial deformation point is calculated in sequence, the first initial loss value Z of the initial deformation point calculated for the i-th time is i The calculation formula is:

[0089]

[0090] Wherein, a is the first preset coefficient.

[0091] In one feasible solution, the value of a is 0.5, and the value of β is 1.

[0092] Then the calculation formula for the first prediction loss value L1 is:

[0093]

[0094] Among them, γ i is the preset weight corresponding to the i-th calculated initial deformation point.

[0095] In another feasible solution, the loss value of the distance between each initial deformation point and its adjacent other initial deformation points can be calculated based on the first prediction information and the first position association information of the initial deformation point. Figure 3 As shown, step S1031 also includes:

[0096] S301: Obtain the labeled neighbor distance of each initial deformation point according to the first labeled position of the initial deformation point; wherein the labeled neighbor distance is the sum of the labeled distances from the initial deformation point to each adjacent initial deformation point;

[0097] S302: Obtain a predicted neighbor distance of each initial deformation point based on a first predicted position of the initial deformation point; wherein the predicted neighbor distance is the sum of the predicted distances from the initial deformation point to each adjacent initial deformation point;

[0098] S303: Determine a second prediction loss value according to the marked neighbor distance and the predicted neighbor distance of each initial deformation point.

[0099] In this solution, based on the first prediction information of the initial deformation point, the predicted position relationship between each initial deformation point and its other adjacent initial deformation points is analyzed, the predicted position relationship of the initial deformation point is compared with the labeled position relationship between each initial deformation point and its other adjacent initial deformation points, and the loss function is optimized to improve the prediction effect of the preset network model on the initial deformation point.

[0100] Specifically, step S303 includes:

[0101] S3031. Calculate a second prediction deviation value for each initial deformation point; wherein the second prediction deviation value is the difference between the marked neighbor distance and the corresponding predicted neighbor distance;

[0102] S3032. Calculate a second predicted loss value based on the second predicted deviation value of each initial deformation point; wherein the second predicted loss value is a second average value corresponding to each initial deformation point, and the second average value is obtained by averaging the sum of the squares of the second predicted deviation values ​​of each initial deformation point.

[0103] Exemplarily, there are N initial deformation points in the deformed document image, the first predicted position of the initial deformation point is represented as P1, the first annotated position of the initial deformation point is represented as P2, it is assumed that each initial deformation point has k adjacent initial deformation points, and the first predicted position of the initial deformation point's adjacent initial deformation point is represented as P1', and the first annotated position of the initial deformation point's adjacent initial deformation point is represented as P2'.

[0104] The calculation formula of the predicted neighbor distance δ1 of the initial deformation point is:

[0105]

[0106] The calculation formula of the δ2 ​​distance between the labeled neighbors of the initial deformation point is:

[0107]

[0108] Then the calculation formula of the second prediction loss value L2 is:

[0109]

[0110] In this scheme, the loss of the distance between the initial deformation point and its other adjacent initial deformation points is introduced into the deformation point loss value. Combined with the positional relationship between the initial deformation point and its other adjacent initial deformation points, the regression accuracy of the prediction information of the initial deformation point can be further improved.

[0111] In addition, in order to obtain the reference point loss value, in an implementable scheme, the second prediction information of the initial reference point includes the predicted horizontal distance and the predicted vertical distance between the initial reference points, and the second position association information of the initial reference point includes the marked horizontal distance and the marked vertical distance between the initial reference points.

[0112] Based on the second prediction information and the second position association information of the initial reference point, the loss value between the predicted horizontal distance of the initial reference point and the marked horizontal distance, and the loss value between the predicted vertical distance of the initial reference point and the marked vertical distance are calculated respectively. Figure 4 As shown, step S1032 includes:

[0113] S401: Calculate and obtain a horizontal prediction deviation value of the initial reference point; wherein the horizontal prediction deviation value is the difference between the predicted horizontal distance and the corresponding marked horizontal distance;

[0114] S402: Calculate and obtain a vertical prediction deviation value of the initial reference point; wherein the vertical prediction deviation value is the difference between the predicted vertical distance and the corresponding marked vertical distance;

[0115] S403: Determine a reference point loss value according to the horizontal prediction deviation value and the vertical prediction deviation value.

[0116] In this scheme, the horizontal distances between the initial reference points are set to equal values, and the vertical distances between the initial reference points are set to equal values. It is only necessary to calculate the difference between the predicted horizontal distance and the marked horizontal distance, and the difference between the predicted vertical distance and the marked vertical distance, respectively, to obtain the initial reference point prediction loss. It is no longer necessary to calculate the loss value for each initial deformation point, which greatly reduces the amount of calculation and improves the overall speed of model training.

[0117] Specifically, step S403 includes: calculating a third average value of the sum of the horizontal prediction deviation value and the vertical prediction deviation value, and using the third average value as the reference point loss value.

[0118] For example, the predicted horizontal distance of the initial reference point is represented as D3, the predicted vertical distance is represented as D4, the marked horizontal distance is represented as D5, and the marked vertical distance is represented as D6. The calculation formula of the reference point loss value L3 is:

[0119]

[0120] In one feasible solution, the preset network model belongs to a convolutional neural network model, including but not limited to a UNet model (a convolutional neural network model).

[0121] Specifically, the model structure of the preset network model is as follows Figure 5 As shown, it includes multiple layers of first convolutional neural networks connected in sequence, and a fully connected neural network and a second convolutional neural network respectively connected to the last layer of the first convolutional neural network.

[0122] Based on Figure 5 In the preset network model shown, step S102 specifically includes:

[0123] S1021, performing feature extraction on the deformed document image through a multi-layer first convolutional neural network to obtain a feature extraction result;

[0124] S1022: Input the feature extraction result into a second convolutional neural network to obtain first prediction information of the initial deformation point;

[0125] S1023: Input the feature extraction result into a fully connected neural network to obtain second prediction information of the initial reference point.

[0126] In this solution, the UNet model is used as the preset network model. The UNet model architecture is simple and easy to implement, with high prediction accuracy, and can retain image detail information during feature extraction.

[0127] In addition, the number of deformed document images that can be used for training is limited, that is, the amount of training sample data that can be annotated is limited. Therefore, in order to expand the amount of training sample data, the present disclosure provides a specific process for obtaining training sample data.

[0128] In one feasible solution, step S101 includes:

[0129] Step S1011: Acquire a sample document image corresponding to the target type;

[0130] Step S1012: deform the sample document image according to at least one preset deformation type to obtain training sample data corresponding to the target type.

[0131] In this solution, one or more training sample data can be obtained by deformation based on a small number of sample document images corresponding to the target type, thereby providing sufficient training sample data for model training.

[0132] Specifically, the first position association information of the initial deformation point includes the first marked position of the initial deformation point. Figure 6 As shown, step S1012 includes:

[0133] S601, dividing the sample document image into a number of area blocks of the same size;

[0134] S602: Set the vertex of each area block as an initial reference point, and obtain a second marked position of each initial reference point according to the size information of the area block;

[0135] S603: randomly selecting at least one target reference point from the initial reference points, and randomly selecting a corresponding deformation control point from the sample document image for each target reference point, to obtain a third annotated position of the deformation control point;

[0136] S604: Determine the deformation control parameters of each initial reference point according to the second marked position of the target reference point, the third marked position of the deformation control point, and the preset deformation type;

[0137] Among them, the deformation control parameters are negatively correlated with the deformation influence distance;

[0138] The deformation influence distance is the distance from the initial reference point to the target line, and the target line consists of the target reference point and the corresponding deformation control point;

[0139] S605: Calculate the first marked position of the corresponding initial deformation point based on the second marked position of each initial reference point and the deformation control parameter;

[0140] S606, generating a deformed document image according to the first marked position of each initial deformation point and the second marked position of the corresponding initial reference point;

[0141] S607: Generate training sample data according to the first marked position of each initial deformation point, the deformed document image, and the size information of the region block.

[0142] It should be noted that the shape of the region block can be selected according to actual needs. The region block is usually rectangular or square. For example, when the region block is rectangular, the size information includes the width and height of the region block.

[0143] In this solution, for any sample document image, corresponding deformed document images can be generated according to different preset deformation types, and then used as training sample data. In other words, a large amount of training sample data can be prepared for training the preset network model based on a small number of sample document images.

[0144] The degree of deformation of the generated deformed document image can also be controlled based on the number of target reference points selected as needed. Typically, only a small number of target reference points, such as one, two, or three, are required. Therefore, too many target reference points can result in a severe degree of deformation of the deformed document image, affecting the image quality of the final deformed document image and increasing the difficulty of model training. Furthermore, the deformation direction and strength of each initial reference point can be controlled based on the target reference points and deformation control points. By comprehensively considering the influence of each target reference point and deformation control point on the deformation direction and strength of each initial reference point, the final deformed document image can be made more realistic and closer to the document image on actual wrinkled or curled paper.

[0145] by Figure 7 Taking the initial reference point A(X,Y) shown in the figure as an example, the target reference point B and the target reference point C are randomly selected. The target reference point B and the corresponding deformation control point B'(X1,Y1) form the target line BB', and the target reference point C and the corresponding deformation control point C'(X2,Y2) form the target line CC'.

[0146] First, the deformation influence distances D1 and D2 of the initial reference point A are calculated. Deformation influence distance D1 is the distance from the initial reference point A to the target line BB', and deformation influence distance D2 is the distance from the initial reference point A to the target line CC'. Then, based on the deformation formula corresponding to the preset deformation type, deformation influence distances D1 and D2, as well as the second annotated position of the deformation control point corresponding to deformation influence distance D1 and deformation influence distance D2, the deformation control parameter of the initial reference point A is calculated. Finally, based on the second annotated position of the initial reference point A and the corresponding deformation control parameter, the first annotated position of the corresponding initial deformation point is calculated.

[0147] If the preset deformation type of the sample document image is folding, the deformation control parameters W1 and W2 of the initial reference point A are calculated according to the following folding deformation formula:

[0148]

[0149]

[0150] Among them, α is the preset deformation parameter.

[0151] Then the first annotation position (X3, Y3) of the initial deformation point A1 corresponding to the initial reference point A is:

[0152] (X3, Y3) = (X, Y) + W1 + W2;

[0153] If the preset deformation type of the sample document image is curl, the deformation control parameters W3 and W4 of the initial reference point A are calculated according to the following curl deformation formula:

[0154] W3=(1-D1 α )×(X1,Y1),

[0155] W4=(1-D2 α )×(X2,Y2);

[0156] Then the first annotation position (X4, Y4) of the initial deformation point A2 corresponding to the initial reference point A is:

[0157] (X4,Y4)=(X,Y)+W3+W4.

[0158] It should be noted that the deformation control parameters include a first control parameter for the horizontal coordinate and a second control parameter for the vertical coordinate. When calculating the first annotation position of the initial deformation point, it is necessary to calculate the sum of the horizontal coordinate of the second annotation position of the initial reference point and the first control parameter, as well as to calculate the sum of the vertical coordinate of the second annotation position of the initial reference point and the second control parameter.

[0159] Taking the deformation control parameter W1 as an example, the first control parameter is The second control parameter is By analogy, the first control parameter in the deformation control parameter W2 is The second control parameter is

[0160] Then the first marked position of the initial deformation point A1 is:

[0161]

[0162]

[0163] In actual applications, the movement of the initial reference point may result in missing pixels in the sample document image. To ensure the integrity of the deformed document image, a preset interpolation method may be used to fill in the missing pixels. Therefore, in another feasible solution, step S606 specifically includes:

[0164] S6061. Obtain a first matrix of a sample document image;

[0165] S6062: Generate a second matrix of the initial deformed document image according to the first marked position of the initial deformation point and the second marked position of the initial reference point;

[0166] S6063: forming a sparse map based on the first matrix and the second matrix; wherein the sparse map is used to represent a situation where the initial deformed document image has pixels with missing pixel values ​​compared to the sample document image;

[0167] S6064, determining the target position of the missing pixel according to the sparse mapping;

[0168] S6065. Fill the target pixel points using a preset interpolation method to obtain a deformed document image.

[0169] Specifically, the sample document image is converted into a first matrix, and the elements at each position of the matrix correspond to the pixel values ​​of the corresponding pixels in the sample document image. The pixel points at the initial reference point are moved to the corresponding initial deformation points to obtain the second matrix of the initial deformed document image. After the pixels are moved, the pixel values ​​of some pixels in the second matrix are empty, and the target pixel points of the missing pixel values ​​can be determined based on the sparse mapping between the first matrix and the second matrix. Then, a preset interpolation method is used to fill the target positions of the missing pixels in the second matrix according to the first matrix to obtain a third matrix, that is, the final deformed document image, and a dense mapping is formed between the first matrix and the third matrix. For example, for a pixel point with a pixel value of 0 (0 represents black), the average value of multiple pixel values ​​around this pixel point in the first matrix is ​​calculated as the pixel value of this pixel point in the second matrix.

[0170] In this solution, missing pixels are filled in by a preset interpolation method to ensure the image clarity and image quality of the deformed document image.

[0171] Based on the above process of obtaining training sample data, different sample document images corresponding to the target type can be selected according to actual needs, thereby generating a large amount of training sample data corresponding to the target type.

[0172] In one feasible solution, the target type is a regular text document, that is, the corresponding sample document image is a text document image.

[0173] In another feasible solution, the target type is a medical bill, that is, the corresponding sample document image is a medical bill image.

[0174] Due to the particularity of medical bill images, it is difficult to obtain a sufficient number of deformed medical bill images in practical applications. Therefore, the above-mentioned training sample data acquisition process can be used to generate a large number of deformed medical bill images based on a small number of medical bill images, and then a model with higher accuracy for correcting medical bill images can be trained.

[0175] Figure 8 A document image processing method provided by the present disclosure is shown. Figure 8 , the document image processing method of this embodiment includes:

[0176] S801, using the above acquisition method to obtain a document image correction model;

[0177] S802, obtaining a document image to be corrected;

[0178] S803: Obtaining first target information of an initial deformation point and second target information of an initial reference point in the document to be corrected through a document image correction model;

[0179] S804: Correct the document image to be corrected according to the first target information and the second target information.

[0180] In this embodiment, the above-mentioned acquisition method is used to iteratively train the preset network model to obtain a document image correction model, and then the document image to be corrected is input into the document image correction model to obtain the target information of the initial deformation point and the initial reference point in the document image to be corrected, and then the initial deformation point is corrected to the position of the initial reference point to achieve correction of the document image.

[0181] However, in actual applications, the original image, including the document image to be corrected, is typically captured using a handheld device such as a mobile phone or camera and directly fed into the prediction network model. Because the original image often includes a cluttered background, the pre-set network model also needs to separately locate the foreground and background in the original image. Predicting the initial deformation point at the edge of the document image to be corrected is inherently difficult. If the document image to be corrected and the background information need to be separated, this can significantly impact the contour, clarity, and quality of the corrected image.

[0182] Therefore, in order to filter the background information in the original captured image, in one feasible solution, step S802 includes:

[0183] S8021, obtaining the original captured image;

[0184] S8022. Input the original captured image into the trained preset segmentation network model to remove background information in the original captured image to obtain the document image to be corrected.

[0185] In this scheme, the background information in the original captured image of the preset segmentation network model area is used to extract the document image to be corrected, thereby reducing the difficulty of predicting the initial deformation point located at the edge of the document image to be corrected and improving the prediction effect and efficiency of the preset network model for the initial deformation point.

[0186] In one feasible solution, the preset segmentation network uses a lightweight semantic segmentation network model.

[0187] Exemplarily, the preset segmentation network model selects the U2-Net model (a semantic segmentation network model), and the original captured image is input into the preset segmentation network. The preset segmentation network predicts a probability map of the document image to be corrected, and then binarizes the probability map through a preset deviation threshold, and finally obtains the document image to be corrected with background information removed.

[0188] Specifically, the preset segmentation network model can be trained based on an independent binary cross entropy loss function. For example, the binary cross entropy loss function L4 is:

[0189]

[0190] Among them, Y i Indicates the label category of the i-th pixel in the original captured image, Y i When it is 1, it means that the label category of the i-th pixel is foreground, Y i When it is 0, it means that the label category of the i-th pixel is background; P3 i represents the probability that the i-th pixel in the original captured image is predicted to be the foreground, and N is the number of pixels in the original captured image.

[0191] In a specific application scenario, Figure 9 Take the document image shown as an example:

[0192] Get Figure 9 The original photographed image shown is input into the trained preset segmentation network model to remove the background information in the original photographed image and extract the document image to be corrected. The document image to be corrected is then input into the document image correction model obtained by the above acquisition method to obtain the first target information of the initial deformation point and the second target information of the initial reference point. Finally, the document image to be corrected is corrected based on the first target information and the second target information, and the following is obtained: Figure 10 The corrected document image is shown.

[0193] In another specific application scenario, for example Figure 11 The automated review process for the insurance claims process shown:

[0194] When applying for medical insurance claims, users take photos and upload images of various expense details, invoices and other medical documents as claim vouchers. The insurance claims adjuster automatically reviews the user's uploaded claim vouchers through text detection and recognition, and then automatically verifies whether a claim is made for the user and the specific amount of the claim.

[0195] Due to the influence of factors such as the physical deformation of the paper, the shooting environment, and the camera angle during the shooting process of medical bills, the captured medical bill images often have problems such as tilt, distortion, and wrinkles. This makes the bill content in the medical bill images difficult to detect and recognize, seriously affecting the accuracy of downstream tasks (such as text detection and recognition), and hindering the intelligent and automated insurance claims.

[0196] To train a pre-set network model specifically for obtaining prediction information about the initial deformation point and initial reference point of medical bills, sample document images corresponding to the medical bill type can be collected and, according to step S1012, a large number of deformed document images corresponding to the medical bill type can be generated. The deformed document images are then annotated, with the initial deformation point and the corresponding initial reference point information for each deformed document image being annotated. Finally, steps S102-S104 are repeatedly performed, i.e., the pre-set network model is trained using a large number of deformed document images until the prediction loss value of the pre-set network model meets the model's usage requirements.

[0197] After training a preset network model that meets the model usage requirements, it is applied to the insurance claims process. Upon receiving the original captured image containing the medical receipt image uploaded by the user, the background information in the original captured image is first filtered according to steps S8021 and S8022 above to obtain the medical receipt image. The medical receipt image is then input into the trained preset network model to obtain target information of the initial deformation point and initial reference point of the medical receipt image. Finally, based on the predicted information of the initial deformation point and initial reference point obtained from the medical receipt image, the initial deformation point is corrected to the corresponding initial reference point to achieve correction of the medical receipt image.

[0198] See Figure 12 After filtering the background information in the original captured image, the deformed medical bill image corresponding to (a) is obtained. The medical bill image is input into the preset network model corresponding to (b) to obtain the target information of the initial deformation point corresponding to (c) and the target information of the initial reference point corresponding to (d). That is, the target predicted position of each initial reference point is calculated based on the predicted horizontal distance and predicted vertical distance between the initial reference points. Then, based on the target information of the initial reference point and the corresponding initial deformation point, a sparse mapping of the medical bill image before and after deformation is generated. The sparse mapping is converted into a dense mapping through interpolation methods such as TPS (thin plate spline interpolation) and Linear (linear difference). Based on the dense mapping, the deformed medical bill image is corrected to obtain the corrected medical bill image corresponding to (e).

[0199] Figure 13 The present invention provides a device for obtaining a document image correction model. Figure 13 , the device for acquiring the document image correction model of this embodiment includes:

[0200] The training sample acquisition module 1301 is used to obtain training sample data;

[0201] The training sample data includes a deformed document image and annotation information, wherein the annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with the positions of the initial deformation point and the initial reference point, respectively.

[0202] It should be noted that the position of the initial reference point in the deformed document image is used to represent the position of the initial deformation point after correction.

[0203] A first prediction module 1302 is configured to input training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point; wherein the prediction information includes position-related information of the initial deformation point and the reference;

[0204] a loss value determining module 1303 for determining a predicted loss value based on the annotation information, the first predicted information, the second predicted information, and preset weights of the initial deformation points; wherein the deformed document image includes a plurality of preset deformation points located in an edge region of the image and other remaining initial deformation points other than the plurality of preset deformation points;

[0205] The weight of the preset deformation point is greater than the weight of other remaining initial deformation points;

[0206] The parameter adjustment module 1304 is used to adjust the network parameters of the preset network model according to the predicted loss value.

[0207] The acquisition device can obtain a document image correction model that meets preset requirements by iteratively training the preset network model.

[0208] In this embodiment, a preset network model is used to obtain prediction information of initial reference points corresponding to initial deformation points in a deformed document image. A prediction loss value is calculated based on the prediction information and the corresponding annotation information, and the network parameters of the preset network model are then adjusted based on the prediction loss value. Specifically, initial deformation points located in the image edge region of the deformed document image are used as preset deformation points, and preset weights of several preset deformation points are set to be greater than the preset weights of the remaining initial deformation points. This improves the regression accuracy of initial deformation points located in the image edge region of the deformed document image, thereby improving the prediction accuracy of initial deformation points located in the image edge region of the deformed document image, thereby ensuring the final correction effect of the deformed document image.

[0209] It should be noted that the image edge area can be set according to actual needs.

[0210] In one feasible solution, the preset weight of the preset deformation point is positively correlated with the first distance; wherein the first distance is the distance from the preset deformation point to the center of the deformed document image.

[0211] That is, the farther the preset deformation point is from the midpoint of the deformed document image, the greater its preset weight, which can further improve the regression accuracy of the preset position association information of the initial deformation point located at the super-edge of the deformed document image, and then improve the prediction accuracy of the super-edge of the deformed document image, so as to ensure that the image contour of the document image obtained by subsequent correction is accurate, the image resolution is clear, and the document content is complete.

[0212] The loss value determination module 1303 is mainly used to determine the deformation point loss value and the reference point loss value based on the annotation information, the first prediction information, the second prediction information and the preset weight of the initial deformation point, and then calculate the predicted loss value based on the deformation point loss value and the reference point loss value.

[0213] Specifically, the loss value determination module 1303 includes:

[0214] a first loss determining unit, configured to determine a loss value of the deformation point according to first prediction information of the initial deformation point, first position association information, and a preset weight;

[0215] a second loss determining unit, configured to determine a reference point loss value based on the second prediction information and the second position association information of the initial reference point;

[0216] The predicted loss determination unit is used to calculate the predicted loss value based on the deformation point loss value and the reference point loss value.

[0217] In this scheme, increasing the preset weights of several preset deformation points located in the image edge area of ​​the deformed document image can improve the regression accuracy of the prediction information of the initial deformation points located at the edge of the deformed document image, thereby improving the prediction accuracy of the initial deformation points located at the edge of the deformed document image.

[0218] In order to obtain the deformation point loss value, in an implementable solution, the first prediction information of the initial deformation point includes the first predicted position of the initial deformation point, and the first position association information of the initial deformation point includes the first marked position of the initial deformation point.

[0219] Based on the first prediction information and the first position association information of the initial deformation point, the loss value between the first predicted position and the first marked position of each initial deformation point is calculated. Specifically, the first loss determination unit includes:

[0220] A first calculation subunit is configured to calculate a first prediction deviation value for each initial deformation point; wherein the first prediction deviation value is a difference between a first predicted position and a corresponding first marked position;

[0221] The first determining subunit is configured to determine a first predicted loss value according to a first predicted deviation value and a preset weight of each initial deformation point.

[0222] The first prediction deviation value is actually the distance between the first predicted position and the corresponding first marked position.

[0223] In this solution, a first prediction loss value is calculated based on the difference between the first predicted position of the initial deformation point and the corresponding first marked position, which intuitively demonstrates the prediction effect of the current preset network model on the initial deformation point.

[0224] Specifically, the first determination subunit is specifically used to obtain the comparison result between the first predicted deviation value and the preset deviation threshold, and determine the first initial loss value of each initial deformation point according to the comparison result; calculate the weighted sum of the first initial loss value of each initial deformation point and the corresponding preset weight; calculate the first average value of the weighted sum, and use the first average value as the first predicted loss value.

[0225] In this solution, different first initial loss values ​​are assigned to the initial deformation point according to the first predicted deviation value between the first predicted position of the initial deformation point and the corresponding first marked position, and the regression accuracy and regression efficiency of the model can be controlled by presetting the deviation threshold.

[0226] For example, there are N initial deformation points in the deformed document image, the first predicted position of the initial deformation point is represented as P1, the first annotated position of the initial deformation point is represented as P2, and the preset deviation threshold is represented as β. When the first initial loss value of each initial deformation point is calculated in sequence, the first initial loss value Z of the initial deformation point calculated for the i-th time is i The calculation formula is:

[0227]

[0228] Wherein, a is the first preset coefficient.

[0229] In one feasible solution, the value of a is 0.5, and the value of β is 1.

[0230] Then the calculation formula for the first prediction loss value L1 is:

[0231]

[0232] Among them, γ i is the preset weight corresponding to the i-th calculated initial deformation point.

[0233] In another feasible solution, the loss value of the distance between each initial deformation point and its adjacent other initial deformation points can be calculated based on the first prediction information and the first position association information of the initial deformation point. The first loss determination unit further includes:

[0234] A second calculation subunit is configured to obtain a labeled neighbor distance of each initial deformation point based on the first labeled position of the initial deformation point; wherein the labeled neighbor distance is the sum of the labeled distances from the initial deformation point to each adjacent initial deformation point;

[0235] a third calculation subunit, configured to obtain a predicted neighbor distance of each initial deformation point based on the first predicted position of the initial deformation point; wherein the predicted neighbor distance is the sum of the predicted distances from the initial deformation point to each adjacent initial deformation point;

[0236] The second determining subunit is configured to determine a second prediction loss value according to the labeled neighbor distance and the predicted neighbor distance of each initial deformation point.

[0237] In this solution, based on the first prediction information of the initial deformation point, the predicted position relationship between each initial deformation point and its other adjacent initial deformation points is analyzed, the predicted position relationship of the initial deformation point is compared with the labeled position relationship between each initial deformation point and its other adjacent initial deformation points, and the loss function is optimized to improve the prediction effect of the preset network model on the initial deformation point.

[0238] Among them, the second determination subunit is specifically used to calculate the second prediction deviation value of each initial deformation point respectively; the second prediction deviation value is the difference between the marked neighbor distance and the corresponding predicted neighbor distance; according to the second prediction deviation value of each initial deformation point, the second prediction loss value is calculated; wherein, the second prediction loss value is the second average value corresponding to each initial deformation point, and the second average value is obtained by averaging the sum of the squares of the second prediction deviation values ​​of each initial deformation point.

[0239] Exemplarily, there are N initial deformation points in the deformed document image, the first predicted position of the initial deformation point is represented as P1, the first annotated position of the initial deformation point is represented as P2, it is assumed that each initial deformation point has k adjacent initial deformation points, and the first predicted position of the initial deformation point's adjacent initial deformation point is represented as P1', and the first annotated position of the initial deformation point's adjacent initial deformation point is represented as P2'.

[0240] The calculation formula of the predicted neighbor distance δ1 of the initial deformation point is:

[0241]

[0242] The calculation formula of the annotation neighbor distance δ2 of the initial deformation point is:

[0243]

[0244] Then the calculation formula of the second prediction loss value L2 is:

[0245]

[0246] In this scheme, the loss of the distance between the initial deformation point and its other adjacent initial deformation points is introduced into the deformation point loss value. Combined with the positional relationship between the initial deformation point and its other adjacent initial deformation points, the regression accuracy of the prediction information of the initial deformation point can be further improved.

[0247] In addition, in order to obtain the reference point loss value, in an implementable scheme, the second prediction information of the initial reference point includes the predicted horizontal distance and the predicted vertical distance between the initial reference points, and the second position association information of the initial reference point includes the marked horizontal distance and the marked vertical distance between the initial reference points.

[0248] Based on the second prediction information and the second position association information of the initial reference point, the loss value between the predicted horizontal distance of the initial reference point and the marked horizontal distance, and the loss value between the predicted vertical distance of the initial reference point and the marked vertical distance are calculated respectively. The second loss determination unit includes:

[0249] The fourth calculation subunit is configured to calculate a horizontal prediction deviation value of the initial reference point; wherein the horizontal prediction deviation value is the difference between the predicted horizontal distance and the corresponding marked horizontal distance;

[0250] A fifth calculation subunit is configured to calculate a vertical prediction deviation value of the initial reference point; wherein the vertical prediction deviation value is the difference between the predicted vertical distance and the corresponding marked vertical distance;

[0251] The third determining subunit is configured to determine a reference point loss value according to the horizontal prediction deviation value and the vertical prediction deviation value.

[0252] In this scheme, the horizontal distances between the initial reference points are set to equal values, and the vertical distances between the initial reference points are set to equal values. It is only necessary to calculate the difference between the predicted horizontal distance and the marked horizontal distance, and the difference between the predicted vertical distance and the marked vertical distance, respectively, to obtain the initial reference point prediction loss. It is no longer necessary to calculate the loss value for each initial deformation point, which greatly reduces the amount of calculation and improves the overall speed of model training.

[0253] The third determining subunit is specifically configured to calculate a third average value of the sum of the horizontal prediction deviation value and the vertical prediction deviation value, and use the third average value as the reference point loss value.

[0254] For example, the predicted horizontal distance of the initial reference point is represented as D3, the predicted vertical distance is represented as D4, the marked horizontal distance is represented as D5, and the marked vertical distance is represented as D6. The calculation formula of the reference point loss value L3 is:

[0255]

[0256] The preset network model used in the first prediction module is a convolutional neural network model, including but not limited to the encoder in the UNet model (a convolutional neural network model).

[0257] Specifically, the model structure of the preset network model is as follows Figure 4 As shown, it includes multiple layers of first convolutional neural networks connected in sequence, and a fully connected neural network and a second convolutional neural network respectively connected to the last layer of the first convolutional neural network.

[0258] Based on Figure 4 In the preset network model shown, the first prediction module includes:

[0259] a feature extraction unit, configured to extract features from the deformed document image using a multi-layer first convolutional neural network to obtain a feature extraction result;

[0260] an initial deformation point prediction unit, configured to input the feature extraction result into a second convolutional neural network to obtain first prediction information of the initial deformation point;

[0261] The initial reference point prediction unit is used to input the feature extraction result into the fully connected neural network to obtain second prediction information of the initial reference point.

[0262] In this solution, the UNet model is used as the preset network model. The UNet model architecture is simple and easy to implement, with high prediction accuracy, and can retain image detail information during feature extraction.

[0263] The number of deformed document images that can be used for training is limited, that is, the number of training sample data that can be annotated is limited. Therefore, in order to expand the amount of training sample data, the training sample acquisition module 1301 provided in the present disclosure can obtain deformed document images corresponding to preset deformation types based on sample document images.

[0264] In one feasible solution, the training sample acquisition module 1301 includes:

[0265] A first image acquisition unit, configured to acquire a sample document image corresponding to a target type;

[0266] The second image acquisition unit is configured to deform the sample document image according to at least one preset deformation type to obtain training sample data corresponding to the target type.

[0267] In this solution, one or more training sample data can be obtained by deformation based on a small number of sample document images corresponding to the target type, thereby providing sufficient training sample data for model training.

[0268] Specifically, the second image acquisition unit includes:

[0269] A region division subunit, used for dividing the sample document image into a number of region blocks of the same size;

[0270] An initial reference point marking subunit, configured to set the vertex of each area block as an initial reference point and obtain a second marking position of each initial reference point based on the size information of the area block;

[0271] a deformation determination subunit, configured to randomly select at least one target reference point from the plurality of initial reference points, and randomly select a corresponding deformation control point from the sample document image for each target reference point, and obtain a third annotated position of the deformation control point;

[0272] a parameter calculation subunit, configured to determine a deformation control parameter of each initial reference point according to the second marked position of the target reference point, the third marked position of the deformation control point, and a preset deformation type;

[0273] Among them, the deformation control parameters are negatively correlated with the deformation influence distance;

[0274] The deformation influence distance is the distance from the initial reference point to the target line, and the target line consists of the target reference point and the corresponding deformation control point;

[0275] an initial deformation point marking subunit, configured to calculate a first marking position of each initial deformation point based on a second marking position of each target reference point and a deformation control parameter;

[0276] a deformation subunit, configured to generate a deformed document image according to a first annotated position of each initial deformation point and a second annotated position of a corresponding initial reference point;

[0277] The training sample generating subunit is used to generate training sample data according to the first marked position of each initial deformation point, the deformed document image, and the size information of the region block.

[0278] It should be noted that the shape of the region block can be selected according to actual needs. The region block is usually rectangular or square. For example, when the region block is rectangular, the size information includes the width and height of the region block.

[0279] In this solution, for any sample document image, corresponding deformed document images can be generated according to different preset deformation types, and then used as training sample data. In other words, a large amount of training sample data can be prepared for training the preset network model based on a small number of sample document images.

[0280] The degree of deformation of the generated deformed document image can also be controlled based on the number of target reference points selected as needed. Typically, only a small number of target reference points are required, such as one, two, or three. Excessive target reference points can lead to severe deformation of the deformed document image, affecting the image quality of the final deformed document image and increasing the difficulty of model training.

[0281] Moreover, the deformation direction and deformation strength of each initial reference point can be controlled according to the target reference point and the deformation control point. By comprehensively considering the influence of each deformation control point on the deformation direction and deformation strength of each initial reference point, the final deformed document image can be made more realistic and closer to the document image on actual wrinkled or curled paper.

[0282] by Figure 3 Taking the initial reference point A(X,Y) shown as an example, the target reference point B and the corresponding deformation control point B'(X1,Y1) form the target line BB', and the target reference point C and the corresponding deformation control point C'(X2,Y2) form the target line CC'.

[0283] First, the deformation influence distances D1 and D2 of the initial reference point A are calculated. Deformation influence distance D1 is the distance from the initial reference point A to the target line BB', and deformation influence distance D2 is the distance from the initial reference point A to the target line CC'. Then, based on the deformation formula corresponding to the preset deformation type, deformation influence distances D1 and D2, as well as the second annotated position of the deformation control point corresponding to deformation influence distance D1 and deformation influence distance D2, the deformation control parameter of the initial reference point A is calculated. Finally, based on the second annotated position of the initial reference point A and the corresponding deformation control parameter, the first annotated position of the corresponding initial deformation point is calculated.

[0284] If the preset deformation type of the sample document image is folding, the deformation control parameters W1 and W2 of the initial reference point A are calculated according to the following folding deformation formula:

[0285]

[0286]

[0287] Among them, α is the preset deformation parameter.

[0288] Then the first annotation position (X3, Y3) of the initial deformation point A1 corresponding to the initial reference point A is:

[0289] (X3, Y3) = (X, Y) + W1 + W2;

[0290] If the preset deformation type of the sample document image is curl, the deformation control parameters W3 and W4 of the initial reference point A are calculated according to the following curl deformation formula:

[0291] W3=(1-D1 α )×(X1,Y1),

[0292] W4=(1-D2 α )×(X2,Y2);

[0293] Then the first annotation position (X4, Y4) of the initial deformation point A2 corresponding to the initial reference point A is:

[0294] (X4,Y4)=(X,Y)+W3+W4.

[0295] It should be noted that the deformation control parameters include a first control parameter for the horizontal coordinate and a second control parameter for the vertical coordinate. When calculating the first annotation position of the initial deformation point, it is necessary to calculate the sum of the horizontal coordinate of the second annotation position of the initial reference point and the first control parameter, as well as to calculate the sum of the vertical coordinate of the second annotation position of the initial reference point and the second control parameter.

[0296] Taking the deformation control parameter W1 as an example, the first control parameter is The second control parameter is By analogy, the first control parameter in the deformation control parameter W2 is The second control parameter is

[0297] Then the first marked position of the initial deformation point A1 is:

[0298]

[0299]

[0300] In practical applications, the movement of the initial reference point may result in missing pixels in the sample document image. In order to ensure the integrity of the deformed document image, a preset interpolation method can be used to fill the missing pixels.

[0301] Therefore, in another feasible scheme, the deformation subunit is specifically used to obtain a first matrix of the sample document image; generate a second matrix of the initial deformed document image based on the first annotated position of the initial deformation point and the second annotated position of the initial reference point; form a sparse mapping based on the first matrix and the second matrix; wherein the sparse mapping is used to characterize the situation where the initial deformed document image has pixel points with missing pixel values ​​compared to the sample document image; determine the target position of the missing pixel based on the sparse mapping; and fill the target pixel points by a preset interpolation method to obtain the deformed document image.

[0302] Specifically, the sample document image is converted into a first matrix, and the elements at each position of the matrix correspond to the pixel values ​​of the corresponding pixels in the sample document image. The pixel points at the initial reference point are moved to the corresponding initial deformation points to obtain the second matrix of the initial deformed document image. After the pixels are moved, the pixel values ​​of some pixels in the second matrix are empty, and the target pixel points of the missing pixel values ​​can be determined based on the sparse mapping between the first matrix and the second matrix. Then, a preset interpolation method is used to fill the target positions of the missing pixels in the second matrix according to the first matrix to obtain a third matrix, that is, the final deformed document image, and a dense mapping is formed between the first matrix and the third matrix. For example, for a pixel point with a pixel value of 0 (0 represents black), the average value of multiple pixel values ​​around this pixel point in the first matrix is ​​calculated as the pixel value of this pixel point in the second matrix.

[0303] In this solution, missing pixels are filled in by a preset interpolation method to ensure the image clarity and image quality of the deformed document image.

[0304] Based on the above process of obtaining training sample data, different sample document images corresponding to the target type can be selected according to actual needs, thereby generating a large amount of training sample data corresponding to the target type.

[0305] In one feasible solution, the target type is a regular text document, that is, the corresponding sample document image is a text document image.

[0306] In another feasible solution, the target type is a medical bill, that is, the corresponding sample document image is a medical bill image.

[0307] Due to the particularity of medical bill images, it is difficult to obtain a sufficient number of deformed medical bill images in practical applications. Therefore, the above-mentioned training sample data acquisition process can be used to generate a large number of deformed medical bill images based on a small number of medical bill images, and then a model with higher accuracy for correcting medical bill images can be trained.

[0308] Figure 14 A document image processing device provided by the present disclosure is shown. Figure 14 , the document image processing device of this embodiment includes:

[0309] Model training module 1401, used to obtain a document image correction model using the above-mentioned acquisition device;

[0310] An image acquisition module 1402 is used to acquire an image of a document to be corrected;

[0311] The second prediction module 1403 is configured to obtain first target information of an initial deformation point in the document to be corrected and second target information of an initial reference point through a document image correction model;

[0312] The document image correction module 1404 is configured to correct the document image to be corrected according to the first target information and the second target information.

[0313] In this scheme, the above-mentioned acquisition device is used to iteratively train the preset network model to obtain a document image correction model, and then the document image to be corrected is input into the document image correction model to obtain the target information of the initial deformation point and the initial reference point in the document image to be corrected, and then the initial deformation point is corrected to the position of the initial reference point to achieve correction of the document image.

[0314] However, in actual applications, the original image, including the document image to be corrected, is typically captured using a handheld device such as a mobile phone or camera and directly fed into the prediction network model. Because the original image often includes a cluttered background, the pre-set network model also needs to separately locate the foreground and background in the original image. Predicting the initial deformation point at the edge of the document image to be corrected is inherently difficult. If the document image to be corrected and the background information need to be separated, this can significantly impact the contour, clarity, and quality of the corrected image.

[0315] Therefore, in order to filter the background information in the original captured image, in one feasible solution, the image acquisition module 1402 includes:

[0316] A third image acquisition unit, configured to acquire an original captured image;

[0317] The background removal unit is used to input the original captured image into the trained preset segmentation network model to remove the background information in the original captured image to obtain the document image to be corrected.

[0318] In this scheme, the background information in the original captured image of the preset segmentation network model area is used to extract the document image to be corrected, thereby reducing the difficulty of predicting the initial deformation point located at the edge of the document image to be corrected and improving the prediction effect and efficiency of the preset network model for the initial deformation point.

[0319] In one feasible solution, the preset segmentation network uses a lightweight semantic segmentation network model.

[0320] Exemplarily, the preset segmentation network model selects the U2-Net model (a semantic segmentation network model), and the original captured image is input into the preset segmentation network. The preset segmentation network predicts a probability map of the document image to be corrected, and then binarizes the probability map through a preset deviation threshold, and finally obtains the document image to be corrected with background information removed.

[0321] Specifically, the preset segmentation network model can be trained based on an independent binary cross entropy loss function. For example, the binary cross entropy loss function L4 is:

[0322]

[0323] Among them, Y i Indicates the label category of the i-th pixel in the original captured image, Y i When it is 1, it means that the label category of the i-th pixel is foreground, Y i When it is 0, it means that the label category of the i-th pixel is background; P3 i represents the probability that the i-th pixel in the original captured image is predicted to be the foreground, and N is the number of pixels in the original captured image.

[0324] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0325] Figure 15 A schematic block diagram of an example electronic device 1500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0326] like Figure 15 As shown, device 1500 includes a computing unit 1501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1502 or a computer program loaded from a storage unit 15015 into a random access memory (RAM) 1503. Various programs and data required for the operation of device 1500 can also be stored in RAM 1503. Computing unit 1501, ROM 1502, and RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to bus 1504.

[0327] Various components in device 1500 are connected to I / O interface 1505, including: an input unit 1506, such as a keyboard, mouse, etc.; an output unit 1508, such as various types of displays, speakers, etc.; a storage unit 1508, such as a magnetic disk, optical disk, etc.; and a communication unit 1509, such as a network card, modem, wireless communication transceiver, etc. Communication unit 1509 allows device 1500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0328] The computing unit 1501 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1501 performs the various methods and processes described above, such as the above-mentioned method. For example, in some embodiments, the above-mentioned method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 1508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1500 via the ROM 1502 and / or the communication unit 1509. When the computer program is loaded into the RAM 1503 and executed by the computing unit 1501, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 1501 can be configured to perform the above-mentioned method by any other appropriate means (e.g., by means of firmware).

[0329] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0330] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0331] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0332] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0333] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0334] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0335] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0336] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for obtaining a document image correction model, the method comprising: Obtain training sample data; Wherein, the training sample data includes deformed document images and annotation information; The annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with positions of the initial deformation point and the initial reference point, respectively; Inputting the training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point; Determining a predicted loss value based on the labeled information, the first predicted information, the second predicted information, and a preset weight of the initial deformation point; The deformed document image includes a plurality of preset initial deformation points located in an edge region of the image, and other remaining initial deformation points other than the plurality of preset initial deformation points; The weight of the preset initial deformation point is greater than the weights of other remaining initial deformation points; Adjusting the network parameters of the preset network model according to the predicted loss value; The step of determining the prediction loss value based on the labeling information, the first prediction information, the second prediction information, and the preset weight of the initial deformation point includes: determining a deformation point loss value according to the first prediction information of the initial deformation point, the first position association information, and the preset weight; determining a reference point loss value according to the second prediction information and the second position association information of the initial reference point; Calculating the predicted loss value based on the deformation point loss value and the reference point loss value; The first prediction information includes a first predicted position of the initial deformation point, and the first position association information includes a first marked position of the initial deformation point; The step of determining the deformation point loss value according to the first prediction information, the first position association information, and the preset weight of the initial deformation point includes: Calculating a first predicted deviation value for each of the initial deformation points; Wherein, the first prediction deviation value is the difference between the first predicted position and the corresponding first marked position; A first prediction loss value is determined according to the first prediction deviation value and the preset weight of each of the initial deformation points.

2. The acquisition method according to claim 1, wherein the preset weight of the preset initial deformation point is positively correlated with the first distance; in, The first distance is the distance from the preset initial deformation point to the center of the deformed document image.

3. The acquisition method according to claim 1, wherein the step of determining the first predicted loss value based on the first predicted deviation value and the preset weight of each initial deformation point comprises: Obtaining a comparison result between the first predicted deviation value and a preset deviation threshold, and determining a first initial loss value for each of the initial deformation points according to the comparison result; Calculating a weighted sum of the first initial loss value of each initial deformation point and the corresponding preset weight; A first average value of the weighted sums is calculated and used as the first predicted loss value.

4. The acquisition method according to claim 1, wherein the step of determining the deformation point loss value based on the first prediction information, the first position association information, and the preset weight of the initial deformation point further comprises: Obtaining a marked neighbor distance of each of the initial deformation points according to the first marked position of the initial deformation point; The annotated neighbor distance is the sum of the annotated distances from the initial deformation point to each adjacent initial deformation point; Obtaining a predicted neighbor distance of each of the initial deformation points according to the first predicted position of the initial deformation point; The predicted neighbor distance is the sum of the predicted distances from the initial deformation point to each of the adjacent initial deformation points; A second prediction loss value is determined according to the marked neighbor distance and the predicted neighbor distance of each of the initial deformation points.

5. The acquisition method according to claim 4, wherein the step of determining the second prediction loss value based on the annotated neighbor distance and the predicted neighbor distance of each initial deformation point comprises: Calculating and obtaining a second predicted deviation value for each of the initial deformation points; Wherein, the second prediction deviation value is the difference between the marked neighbor distance and the corresponding predicted neighbor distance; Calculating the second predicted loss value according to the second predicted deviation value of each of the initial deformation points; The second predicted loss value is a second average value corresponding to each of the initial deformation points, and the second average value is obtained by averaging the sum of the squares of the second predicted deviation values ​​of each of the initial deformation points.

6. The acquisition method according to claim 1, wherein the second prediction information includes a predicted horizontal distance and a predicted vertical distance between the initial reference points, and the second position association information includes a marked horizontal distance and a marked vertical distance between the initial reference points; The step of determining the reference point loss value according to the second prediction information and the second position association information of the initial reference point includes: Calculating a horizontal prediction deviation value of the initial reference point; Wherein, the horizontal prediction deviation value is the difference between the predicted horizontal distance and the corresponding marked horizontal distance; Calculating a vertical prediction deviation value of the initial reference point; Wherein, the vertical prediction deviation value is the difference between the predicted vertical distance and the corresponding marked vertical distance; The reference point loss value is determined according to the horizontal prediction deviation value and the vertical prediction deviation value.

7. The acquisition method according to claim 1, wherein the preset network model comprises a first convolutional neural network with multiple layers connected sequentially, and a fully connected neural network and a second convolutional neural network respectively connected to the last layer of the first convolutional neural network; The step of inputting the training sample data into the preset network model to obtain prediction information of the initial deformation point and the corresponding initial reference point in the deformed document image comprises: Performing feature extraction on the deformed document image through the multi-layer first convolutional neural network to obtain a feature extraction result; Inputting the feature extraction result into the second convolutional neural network to obtain the first prediction information of the initial deformation point; The feature extraction result is input into the fully connected neural network to obtain the second prediction information of the initial reference point.

8. The acquisition method according to any one of claims 1 to 7, wherein the step of acquiring training sample data comprises: Obtaining a sample document image corresponding to a target type; The sample document image is deformed according to at least one preset deformation type to obtain the training sample data corresponding to the target type.

9. The acquisition method according to claim 8, wherein the step of deforming the sample document image according to at least one preset deformation type to obtain the training sample data corresponding to the target type comprises: Dividing the sample document image into a plurality of area blocks of the same size; Setting the vertex of each of the area blocks as the initial reference point, and obtaining a second marked position of each of the initial reference points according to the size information of the area block; Randomly selecting at least one target reference point from the initial reference points, and randomly selecting a corresponding deformation control point from the sample document image for each target reference point, to obtain a third annotated position of the deformation control point; determining a deformation control parameter for each of the initial reference points according to the second marked position of the target reference point, the third marked position of the deformation control point, and the preset deformation type; Wherein, the deformation control parameter is negatively correlated with the deformation influence distance; The deformation impact distance is the distance from the initial reference point to the target straight line; The target straight line is composed of the target reference point and the corresponding deformation control point; Calculating the first marked position of the corresponding initial deformation point according to the second marked position of each initial reference point and the deformation control parameter; generating the deformed document image according to the first marked position of each of the initial deformation points and the second marked position of the corresponding initial reference point; The training sample data is generated according to the first marked position of each initial deformation point, the deformed document image, and the size information of the region block.

10. The acquisition method according to claim 9, wherein the step of generating the deformed document image according to the first marked position of each initial deformation point and the second marked position of the corresponding initial reference point comprises: Acquire a first matrix of the sample document image; generating a second matrix of the initial deformed document image according to the first marked position of the initial deformed point and the second marked position of the initial reference point; forming a sparse mapping according to the first matrix and the second matrix; The sparse mapping is used to characterize the situation where the initial deformed document image has pixel points with missing pixel values ​​compared to the sample document image; Determine a target pixel point according to the sparse mapping; The target pixel points are filled in a preset interpolation manner to obtain the deformed document image. The acquisition method according to claim 8 , wherein the target type is a medical bill.

12. A document image processing method, comprising: Acquire a document image correction model using the acquisition method described in any one of claims 1 to 11; Acquire the document image to be corrected; Obtaining, through the document image correction model, first target information of an initial deformation point in the document to be corrected and second target information of an initial reference point; The document image to be corrected is corrected according to the first target information and the second target information.

13. The document image processing method according to claim 12, wherein the step of obtaining the document image to be corrected comprises: Get the original captured image; The original photographed image is input into a trained preset segmentation network model, and background information in the original photographed image is removed to obtain the document image to be corrected.

14. A device for acquiring a document image correction model, the device comprising: A training sample acquisition module is used to obtain training sample data; Wherein, the training sample data includes deformed document images and annotation information; The annotation information includes an initial deformation point and an initial reference point annotated based on the deformed document image, and information associated with positions of the initial deformation point and the initial reference point, respectively; A first prediction module, configured to input the training sample data into a preset network model to obtain first prediction information of the initial deformation point and second prediction information of the initial reference point; a loss value determination module, configured to determine a predicted loss value based on the annotation information, the first prediction information, the second prediction information, and a preset weight of the initial deformation point; The deformed document image includes a plurality of preset deformation points located in the edge area of ​​the image, and other remaining initial deformation points except the plurality of preset deformation points; The weight of the preset deformation point is greater than the weights of other remaining initial deformation points; A parameter adjustment module, configured to adjust the network parameters of the preset network model according to the predicted loss value; The loss value determination module includes: a first loss determining unit, configured to determine a loss value of a deformation point according to the first prediction information of the initial deformation point, the first position association information, and the preset weight; a second loss determining unit, configured to determine a reference point loss value according to the second prediction information and the second position association information of the initial reference point; A predicted loss determining unit, configured to calculate the predicted loss value based on the deformation point loss value and the reference point loss value; The first prediction information includes a first predicted position of the initial deformation point, and the first position association information includes a first marked position of the initial deformation point; The first loss determining unit includes: A first calculation subunit, configured to calculate and obtain a first prediction deviation value for each of the initial deformation points; Wherein, the first prediction deviation value is the difference between the first predicted position and the corresponding first marked position; The first determining subunit is configured to determine a first predicted loss value according to the first predicted deviation value and the preset weight of each of the initial deformation points.

15. The acquisition device according to claim 14, wherein the preset weight of the preset inflection point is positively correlated with the first distance; in, The first distance is the distance from the preset deformation point to the center of the deformed document image.

16. According to the acquisition device according to claim 14, the first determination subunit is used to obtain the comparison result of the first predicted deviation value and the preset deviation threshold, and determine the first initial loss value of each initial deformation point according to the comparison result; calculate the first initial loss value of each initial deformation point and the weighted sum of the corresponding preset weight; calculate the first average value of the weighted sum, and use the first average value as the first predicted loss value.

17. The acquisition device according to claim 14, wherein the first loss determination unit further comprises: a second calculation subunit, configured to obtain a marked neighbor distance of each of the initial deformation points according to the first marked position of the initial deformation point; The annotated neighbor distance is the sum of the annotated distances from the initial deformation point to each adjacent initial deformation point; a third calculation subunit, configured to obtain a predicted neighbor distance of each of the initial deformation points according to the first predicted position of the initial deformation point; The predicted neighbor distance is the sum of the predicted distances from the initial deformation point to each of the adjacent initial deformation points; The second determining subunit is configured to determine a second prediction loss value according to the marked neighbor distance and the predicted neighbor distance of each of the initial deformation points.

18. The acquisition device according to claim 17, wherein the second determining subunit is configured to calculate a second prediction deviation value for each of the initial deformation points; the second prediction deviation value is the difference between the annotated neighbor distance and the corresponding predicted neighbor distance; and the second prediction loss value is calculated based on the second prediction deviation value for each of the initial deformation points; in, The second predicted loss value is a second average value corresponding to each of the initial deformation points, and the second average value is obtained by averaging the sum of the squares of the second predicted deviation values ​​of each of the initial deformation points.

19. The acquisition device according to claim 14, wherein the second prediction information includes a predicted horizontal distance and a predicted vertical distance between the initial reference points, and the second position association information includes a marked horizontal distance and a marked vertical distance between the initial reference points; The second loss determining unit includes: A fourth calculation subunit, configured to calculate a horizontal prediction deviation value of the initial reference point; Wherein, the horizontal prediction deviation value is the difference between the predicted horizontal distance and the corresponding marked horizontal distance; A fifth calculation subunit, configured to calculate a vertical prediction deviation value of the initial reference point; Wherein, the vertical prediction deviation value is the difference between the predicted vertical distance and the corresponding marked vertical distance; The third determining subunit is configured to determine the reference point loss value according to the horizontal prediction deviation value and the vertical prediction deviation value.

20. The acquisition device according to claim 14, wherein the preset network model comprises multiple layers of first convolutional neural networks connected in sequence, and a fully connected neural network and a second convolutional neural network respectively connected to the last layer of the first convolutional neural network; The first prediction module includes: a feature extraction unit, configured to perform feature extraction on the deformed document image through the multi-layer first convolutional neural network to obtain a feature extraction result; an initial deformation point prediction unit, configured to input the feature extraction result into the second convolutional neural network to obtain the first prediction information of the initial deformation point; An initial reference point prediction unit is used to input the feature extraction result into the fully connected neural network to obtain the second prediction information of the initial reference point.

21. The acquisition device according to any one of claims 15 to 20, wherein the training sample acquisition module comprises: A first image acquisition unit, configured to acquire a sample document image corresponding to a target type; The second image acquisition unit is configured to deform the sample document image according to at least one preset deformation type to obtain the training sample data corresponding to the target type.

22. The acquisition device according to claim 21, wherein the second image acquisition unit comprises: A region division subunit, configured to divide the sample document image into a plurality of region blocks of the same size; an initial reference point marking subunit, configured to set a vertex of each of the area blocks as the initial reference point, and obtain a second marking position of each of the initial reference points according to size information of the area block; a deformation determination subunit, configured to randomly select at least one target reference point from the plurality of initial reference points, and randomly select a corresponding deformation control point from the sample document image for each target reference point, and obtain a third annotated position of the deformation control point; a parameter calculation subunit, configured to determine a deformation control parameter of each of the initial reference points according to the second marked position of the target reference point, the third marked position of the deformation control point, and the preset deformation type; Wherein, the deformation control parameter is negatively correlated with the deformation influence distance; The deformation influence distance is the distance from the initial reference point to the target straight line, and the target straight line is composed of the target reference point and the corresponding deformation control point; an initial deformation point marking subunit, configured to calculate the first marking position of each of the initial deformation points according to the second marking position of each of the target reference points and the deformation control parameter; a deformation subunit, configured to generate the deformed document image according to the first marked position of each of the initial deformation points and the second marked position of the corresponding initial reference point; The training sample generating subunit is configured to generate the training sample data according to the first marked position of each initial deformation point, the deformed document image, and the size information of the region block.

23. The acquisition device according to claim 22, wherein the deformation subunit is configured to acquire a first matrix of the sample document image; generate a second matrix of the initial deformed document image based on the first annotated position of the initial deformation point and the second annotated position of the initial reference point; and form a sparse mapping based on the first matrix and the second matrix; wherein, The sparse mapping is used to characterize the situation where the initial deformed document image has pixel points with missing pixel values ​​compared to the sample document image; determine the target pixel points based on the sparse mapping; and fill the target pixel points using a preset interpolation method to obtain the deformed document image. The acquisition device according to claim 21 , wherein the target type is a medical bill.

25. A document image processing device, comprising: A model training module, configured to acquire a document image correction model using an acquisition device according to any one of claims 14 to 24; An image acquisition module, used for acquiring an image of a document to be corrected; A second prediction module is configured to obtain, through the document image correction model, first target information of an initial deformation point in the document to be corrected and second target information of an initial reference point; The document image correction module is used to correct the document image to be corrected according to the first target information and the second target information.

26. The document image processing apparatus according to claim 25, wherein the image acquisition module comprises: A third image acquisition unit, configured to acquire an original captured image; The background removal unit is used to input the original photographed image into a trained preset segmentation network model to remove background information in the original photographed image to obtain the document image to be corrected.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11; or the method of claim 12 or 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 11; or the method according to claim 12 or 13.

29. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 11; or the method according to claim 12 or 13.

Images