A medical document image correction and recognition method and system based on grid points

Abstract

The present invention relates to a medical document image correction and recognition method and system based on grid points, the method comprising the following steps: step A: obtaining the image to be corrected of the medical document and its corresponding real image, and constructing a test data set; step B: constructing a medical document image correction model based on grid points and training it through a training data set to obtain a trained medical document image correction model; step C: testing the trained medical document image correction model using the test data set, and the one that is evaluated as qualified is the final medical document image correction model; step D: applying the final medical document image correction model to medical document image correction to obtain a corrected medical document image; then performing text recognition on the corrected medical document image to obtain a recognized medical text. The method and system are conducive to quickly and accurately correcting medical document images and performing text recognition on them.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a medical document image correction and recognition method and system based on grid points. Background Art

[0002] Medical documents are common documents for patients to see a doctor, and they are used as evidence of their condition and subsequent treatment. In the face of the organization and storage of a large number of medical documents, hospital informatization is needed to achieve this. The electronic medical record system is used to integrate and share information within the hospital, and authorized physicians can retrieve the corresponding documents from the system for review and editing. This approach avoids the paper consumption and storage space occupation brought about by traditional paper documents, but the security issues of patients' privacy data also arise. In addition, the storage form of the data depends on the internal design of the system and cannot be universalized. Due to the limited hospital database, these data cannot be saved for a long time, which brings inconvenience to patients who want to view their medical document data. And because the information formats of medical documents in different hospitals are different, it is very difficult to make comprehensive use of medical document resources and it is difficult to share them between hospitals.

[0003] Medical document digitization tasks are increasingly being performed using smartphones instead of more traditional flatbed scanners. The appearance of these haphazardly captured medical document images is often very different from flatbed scans due to different camera angles, unconstrained lighting conditions, and physical deformations of the paper (such as folds, wrinkles, and bends). These appearance changes pose problems for post-processing steps such as Optical Character Recognition (OCR). Therefore, medical document image correction is an important step in modernizing the medical document digitization process to bring layout extraction and OCR performance on par with traditional processes. Summary of the invention

[0004] The object of the present invention is to provide a method and system for medical document image correction and recognition based on grid points, which method and system are conducive to quickly and accurately correcting medical document images and performing text recognition on them.

[0005] In order to achieve the above object, the technical solution of the present invention is: a medical document image correction and recognition method based on grid points, comprising the following steps:

[0006] Step A: Obtain the image to be corrected of the medical document and its corresponding real image to construct a test data set;

[0007] Step B: constructing a medical document image correction model based on grid points; the medical document image correction model includes a multi-layer convolution module, a Resnet network, a bridge module, a convolution fusion module and a prediction position point module. For input data, features of different scales are extracted through the multi-layer convolution module, and more complex features are extracted through Resnet learning, and then multi-scale features are integrated through the bridge module, and the prediction results are output after convolution fusion; the medical document image correction model is trained through the training data set to obtain a trained medical document image correction model;

[0008] Step C: Use the test data set to test the trained medical document image correction model to evaluate its performance on unknown data. The one that passes the evaluation is the final medical document image correction model;

[0009] Step D: Apply the final medical document image correction model to medical document image correction to obtain a corrected medical document image; then perform text recognition on the corrected medical document image to obtain a recognized medical text.

[0010] Furthermore, in step A, images of medical documents are acquired by methods including camera shooting, mobile phone shooting and flatbed scanning, wherein the images acquired by camera shooting and mobile phone shooting are used as images to be corrected, and the images acquired by flatbed scanning are used as corresponding real images; then, all images to be corrected and the corresponding real images are combined to establish a test data set.

[0011] Furthermore, the medical documents include medical records, medical invoices, prescriptions and physical examination reports.

[0012] Furthermore, the step B specifically includes the following steps:

[0013] Step B1: Construct a medical document image correction model based on grid points; load the data of the training data set and perform data processing;

[0014] Step B2: Train the constructed medical document image correction model using the data after data processing to obtain a trained medical document image correction model.

[0015] Furthermore, in step B1, the data of the training data set is loaded and data processing is performed, specifically:

[0016] Initialize the training data set data loading, set the data loading path, data processing method, image size and grid size;

[0017] Load the distorted text images from the training dataset Image 0 and the two-dimensional grid map corresponding to the image G 2D_0, the two-dimensional grid map G 2D_0 Crop to the set size to get the processed two-dimensional grid image G 2D_1 , the distorted text image Image 0 is converted into RGB format and its background information is randomly replaced to obtain the converted distorted text image Image 1. Then, the converted distorted text image Image 1. Perform tight cropping until the edge of the text is at the edge of the image to remove unnecessary edges and obtain the processed distorted text image. Image 2;

[0018] Finally, the processed two-dimensional grid map G 2D_1 And the distorted text image after processing Image 2 Perform bilinear interpolation to obtain the real text image Image bm .

[0019] Furthermore, the step B2 specifically includes the following steps:

[0020] Step B21: Load and process the distorted text images from the training dataset Image 2 Input into the medical document image correction model;

[0021] Step B22: After the data passes through the first convolution layer of the multi-layer convolution module, it is input into the first batch normalization layer to normalize the output of the first convolution layer. F conv_1 , and then the first ReLU activation function introduces nonlinearity to help the model learn more complex features; the data output by the first ReLU activation function passes through the second convolutional layer to further extract features and downsample, and then enters the second batch normalization layer to normalize the output of the second convolutional layer F conv_2 , the output obtained is F conv_2 It is concatenated with the input data and then passed through a second ReLU activation function. Then, the ResNet network is used to further extract features and gradually reduce the spatial dimension of the features. In the ResNet network, the data is extracted through multiple residual blocks and then processed through multiple layers of dilated convolution, batch normalization, and activation functions. The dilation rate of each layer increases continuously to obtain the features. F grid1 , F grid2 ,......, F gridn ;

[0022] Step B23: The features obtained by bridging the module F grid1 , F grid2 ,......, F gridn Splicing is performed in the channel dimension, and features are obtained through the first bridge module. F bright1 , after the second bridge module, the feature F bright2 , ..., after the eighth bridge module, the feature F bright8 , the bridge module includes two dilated convolutional layers, a batch normalization layer and a ReLU activation function;

[0023] Step B24: Concatenate the outputs of the eight bridge modules in the channel dimension to obtain features F bright_concat0 , and then compress the channel dimension through the convolution layer of the convolution fusion module, reduce the number of parameters and integrate the features to obtain the feature F bright_concat1 , and then the output of the convolutional layer is normalized through the batch normalization layer to obtain the feature F bright_concat2 , and then the normalized features F bright_concat2 Apply the ReLU activation function to get the feature F bright_concat3 ;

[0024] Step B25: Add features F bright_concat3 In the input prediction location point module, the first convolution layer is used to extract features and compress channels, and then the batch normalization layer is used to normalize the features. F bright_concat3_pred , and then the normalized features F bright_concat3_pred Apply the PReLU activation function to introduce nonlinearity; finally, the feature data is converted into the final two-dimensional point position prediction through the second convolution layer, and the predicted two-dimensional feature map is output G pred_2D , two-dimensional feature map G pred_2D The two channel values ​​of each pixel in represents the corresponding two-dimensional point position prediction;

[0025] Step B26: The predicted two-dimensional feature map G pred_2D Image with distorted text Image 2 Perform bilinear interpolation to obtain the text image after model correction Image"pred , and then with the real text image Image bm , 2D grid graph G 2D_1 and 2D feature map G pred_2D Input them into the loss function together to calculate the loss L recon , back propagates into the model, and corrects the text image of the model Image" pred and real text images Image bm Calculate the mean square error (MSE) and save the model with the smallest MSE, which is the trained medical document image correction model. P model ; The loss function is expressed as:

[0026]

[0027] in, α and β is the set hyperparameter, L 1 loss Represents the mean absolute error MAE, L 2 loss Stands for mean square error MSE.

[0028] Furthermore, the specific implementation method of step C is:

[0029] The image to be rectified in the test dataset Image pred_distorted Input the trained medical document image correction model P model , get the predicted two-dimensional feature map G" pred_2D , the predicted two-dimensional feature map G" pred_2D The image to be corrected Image pred_distorted Perform bilinear interpolation to obtain the image corrected by the model Image pred ;

[0030] The image after the model is rectified Image pred The image to be corrected Image pred_distorted The corresponding real images are compared to evaluate the performance of the model on the test dataset, and the one that passes the evaluation is the final medical document image correction model.

[0031] Furthermore, in step D, the open source PyTessart tool is used to perform OCR recognition on the corrected medical document image, and the text recognized by OCR is highlighted to assist medical personnel in quickly locating data to obtain the recognized medical text.

[0032] Furthermore, the medical document image correction and recognition method is applied to recognize printed medical documents.

[0033] The present invention also provides a medical document image correction and recognition system based on grid points, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0034] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a medical document image correction and recognition method and system based on grid points. The method uses dilated convolution and eight bridging modules to predict the grid points of medical document images, which can effectively capture the dependencies of different distance spaces without adding convolution layers. The present invention uses grid points to correct medical document images, thereby improving the readability of the corrected medical document images. At the same time, in order to improve the accuracy of text recognition, the present invention adopts the method of correcting the image first and then performing text recognition, thereby improving the text recognition performance. The present invention conducts experiments on real medical document image datasets and compares them with other methods. The experimental results show that the present invention has achieved better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0036] Figure 2 is a flowchart for implementing training of a medical document image correction model based on grid points in an embodiment of the present invention;

[0037] Figure 3 is a structural block diagram of a bridge module in an embodiment of the present invention;

[0038] Figure 4 is a correction effect diagram of the method on a test data in an embodiment of the present invention;

[0039] Figure 5 is a correction effect diagram of the method in the embodiment of the present invention on another test data;

[0040] Figure 6 This is a result diagram of OCR recognition of the corrected medical document image in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Medical documents are common documents used by patients when they seek medical treatment, as evidence of their condition and subsequent treatment. Due to the limited size of hospital databases, these data cannot be saved for a long time, which brings inconvenience to patients who want to view their own medical document data. In addition, since the information formats of medical documents in different hospitals are different, it is very difficult to make comprehensive use of medical document resources and to share them between hospitals.

[0045] In order to solve this problem, the present invention introduces a medical document image correction and recognition method and system based on grid points. The present invention uses grid points to correct medical document images, thereby improving the readability of the corrected medical document images. At the same time, in order to improve the accuracy of text recognition, the present invention adopts the method of first correcting the image and then performing text recognition, thereby improving the text recognition performance.

[0046] like Figure 1 As shown, this embodiment provides a medical document image correction and recognition method based on grid points, comprising the following steps:

[0047] Step A: Obtain the image to be corrected of the medical document and its corresponding real image to construct a test data set;

[0048] Step B: constructing a medical document image correction model based on grid points; the medical document image correction model includes a multi-layer convolution module, a Resnet network, a bridge module, a convolution fusion module and a prediction position point module. For input data, features of different scales are extracted through the multi-layer convolution module, and more complex features are extracted through Resnet learning, and then multi-scale features are integrated through the bridge module, and the prediction results are output after convolution fusion; the medical document image correction model is trained through the training data set to obtain a trained medical document image correction model;

[0049] Step C: Use the test data set to test the trained medical document image correction model to evaluate its performance on unknown data. The one that passes the evaluation is the final medical document image correction model;

[0050] Step D: Apply the final medical document image correction model to medical document image correction to obtain a corrected medical document image; then perform text recognition on the corrected medical document image to obtain a recognized medical text.

[0051] Recognized medical texts can be highlighted to help medical staff quickly locate data, thereby improving work efficiency.

[0052] In this embodiment, in step A, images of medical documents are obtained by camera shooting, mobile phone shooting and flatbed scanning, wherein the images obtained by camera shooting and mobile phone shooting are used as images to be corrected, and the images obtained by flatbed scanning are used as corresponding real images. Among them, medical documents include medical records, medical invoices, prescriptions and physical examination reports. Then, a test data set is established by combining all the images to be corrected and the corresponding real images.

[0053] Figure 2 It is the architecture of the grid point-based medical document image correction model in this embodiment and the principle diagram of its training, which shows the process of the model processing the medical document image to output a two-dimensional grid.

[0054] In this embodiment, the specific implementation steps of step B are as follows.

[0055] Step B1: First, a medical document image correction model based on grid points is constructed; then, the data of the training data set is loaded and the data is processed. The specific implementation is as follows.

[0056] First, initialize the data loading of the training data set and set the data loading path, data processing method, image size, and grid size.

[0057] Then, load the distorted text images from the training dataset Image 0 and the two-dimensional grid map corresponding to the image G 2D_0 , the two-dimensional grid map G 2D_0 Crop to the set size (256*256) to get the processed two-dimensional grid image G 2D_1 , the distorted text image Image 0 is converted into RGB format and its background information is randomly replaced to obtain the converted distorted text image Image 1. Then, the converted distorted text image Image1. Perform tight cropping until the edge of the text is at the edge of the image to remove unnecessary edges and obtain the processed distorted text image. Image 2.

[0058] Finally, the processed two-dimensional grid map G 2D_1 And the distorted text image after processing Image 2 Perform bilinear interpolation to obtain the real text image Image bm .

[0059] Step B2: Train the constructed medical document image correction model with the data after data processing to obtain a trained medical document image correction model. The specific implementation steps are as follows.

[0060] Step B21: Load and process the distorted text images from the training dataset Image 2 Input into the medical document image correction model. Set the kernel size and stride of the convolution layer. In this implementation, the kernel size of all convolution layers is 3 and the stride is 1.

[0061] Step B22: After the data passes through the first convolution layer of the multi-layer convolution module, it is input into the first batch normalization layer to normalize the output of the first convolution layer. F conv_1 , and then the first ReLU activation function introduces nonlinearity to help the model learn more complex features; the data output by the first ReLU activation function passes through the second convolutional layer to further extract features and downsample, and then enters the second batch normalization layer to normalize the output of the second convolutional layer F conv_2 , the output obtained is F conv_2 The data is concatenated with the input data and then passed through a second ReLU activation function. The ResNet network is then used to further extract features and gradually reduce the spatial dimension of the features. In the ResNet network, the data passes through a three-layer sequence of residual blocks, and features are extracted through multiple residual blocks. Then, the data is processed through multiple layers of dilated convolution, batch normalization, and activation functions. The dilation rate of each layer increases continuously to obtain features. F grid1 , F grid2 ,......, F gridn .

[0062] Step B23: The features obtained by bridging the module F grid1 , F grid2 ,......,F gridn Splicing is performed in the channel dimension, and features are obtained through the first bridge module. F bright1 , after the second bridge module, the feature F bright2 , ..., after the eighth bridge module, the feature F bright8 .like Figure 3 As shown, the bridge module includes two dilated convolutional layers, a batch normalization layer and a ReLU activation function.

[0063] Step B24: Concatenate the outputs of the eight bridge modules in the channel dimension to obtain features F bright_concat0 , and then compress the channel dimension through the convolution layer of the convolution fusion module, reduce the number of parameters and integrate the features to obtain the feature F bright_concat1 , and then the output of the convolutional layer is normalized through the batch normalization layer to obtain the feature F bright_concat2 , and then the normalized features F bright_concat2 Apply the ReLU activation function to get the feature F bright_concat3 .

[0064] Step B25: Add features F bright_concat3 In the input prediction location point module, the first convolution layer is used to extract features and compress channels, and then the batch normalization layer is used to normalize the features. F bright_concat3_pred , and then the normalized features F bright_concat3_pred Apply the PReLU activation function to introduce nonlinearity; finally, the feature data is converted into the final two-dimensional point position prediction through the second convolution layer, and the predicted two-dimensional feature map is output G pred_2D , two-dimensional feature map G pred_2D The two channel values ​​of each pixel in represents the corresponding two-dimensional point position prediction.

[0065] Step B26: The predicted two-dimensional feature map G pred_2D Image with distorted text Image 2 Perform bilinear interpolation to obtain the text image after model correction Image" pred , and then with the real text image Image bm , 2D grid graph G2D_1 and 2D feature map G pred_2D Input them into the loss function together to calculate the loss L recon , back propagates into the model, and corrects the text image of the model Image" pred and real text images Image bm Calculate the mean square error (MSE) and save the model with the smallest MSE, which is the trained medical document image correction model. P model ; The loss function is expressed as:

[0066]

[0067] in, α and β is the set hyperparameter, L 1 loss Represents the mean absolute error MAE, L 2 loss Stands for mean square error MSE.

[0068] In this embodiment, the specific implementation method of step C is:

[0069] The image to be rectified in the test dataset Image pred_distorted Input the trained medical document image correction model P model , get the predicted two-dimensional feature map G" pred_2D , the predicted two-dimensional feature map G" pred_2D The image to be corrected Image pred_distorted Perform bilinear interpolation to obtain the image corrected by the model Image pred ;

[0070] The image after the model is rectified Image pred The image to be corrected Image pred_distorted The corresponding real images are compared to evaluate the performance of the model on the test dataset, and the one that passes the evaluation is the final medical document image correction model.

[0071] In the step D, the open source PyTessart tool is used to perform OCR recognition on the corrected medical document image, and the text recognized by OCR is highlighted to assist medical personnel in quickly locating data to obtain the recognized medical text.

[0072] In this embodiment, real medical document image data is used for testing. The images are taken from different mobile phones, with a total of 2,000 copies. The layout of the medical documents is not fixed. The shooting mobile phone models include Xiaomi 7, Huawei mate60, and iPhone 14pro. For all experiments, training and testing are performed under the same hardware environment. The training data set uses the public data set Doc3d, which contains 100,000 pictures and corresponding optical flow maps, ultraviolet maps, three-dimensional coordinate maps, two-dimensional grid maps, depth maps, etc. This embodiment uses the Adam optimizer for training, with an initial learning rate of 0.001 and a batch size of 2. This embodiment uses data standardization and data enhancement techniques during the training process. Data enhancement includes random rotation, random cropping, and random horizontal flipping, where the random rotation amplitude is within ten degrees, and the randomly cropped pictures are 256*256 size pictures. This embodiment uses two common evaluation indicators: Multi-Scale Structural Similarity (MS-SSIM) and Local Distortion (LD). In OCR quality evaluation, Edit Distance (ED) and Character Error Rate (CER) are used to measure the performance of the model.

[0073] Figure 4-5 The correction effect of the method in this embodiment on the test data set is shown. Figure 4 This is a diagram showing the correction effect of a contract. Figure 5 This is a diagram showing the correction effect of another contract. To protect privacy, sensitive data has been anonymized. Figure 4-5 It can be seen that this method can correct and recognize medical document images well and is not limited to one type of medical document image.

[0074] Figure 6 The results of OCR recognition of the corrected medical document image in this embodiment are shown. Figure 6 It can be seen that this method has a better recognition effect.

[0075] This embodiment also provides a medical document image correction and recognition system based on grid points, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0076] Since the detection of doctor's handwriting in medical documents is relatively complicated, the method and system provided by the present invention are mainly used for identifying printed medical documents.

[0077] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A medical document image correction and recognition method based on grid points, characterized in that: The following steps are involved: Step A: Obtain the image to be corrected of the medical document and its corresponding real image to construct a test data set; Step B: constructing a medical document image correction model based on grid points; the medical document image correction model includes a multi-layer convolution module, a Resnet network, a bridge module, a convolution fusion module and a prediction position point module. For input data, features of different scales are extracted through the multi-layer convolution module, and more complex features are extracted through Resnet learning, and then multi-scale features are integrated through the bridge module, and the prediction results are output after convolution fusion; the medical document image correction model is trained through the training data set to obtain a trained medical document image correction model; Step C: Use the test data set to test the trained medical document image correction model to evaluate its performance on unknown data. The one that passes the evaluation is the final medical document image correction model; Step D: applying the finally obtained medical document image correction model to medical document image correction to obtain a corrected medical document image; then performing text recognition on the corrected medical document image to obtain a recognized medical text; The step B specifically comprises the following steps: Step B1: Construct a medical document image correction model based on grid points; load the data of the training data set and perform data processing; Step B2: training the constructed medical document image correction model using the data after data processing to obtain a trained medical document image correction model; In step B1, the data of the training data set is loaded and data processing is performed, specifically: Initialize the training data set data loading, set the data loading path, data processing method, image size and grid size; Load the distorted text images from the training dataset Image 0 and the two-dimensional grid map corresponding to the image G 2D_0 , the two-dimensional grid map G 2D_0 Crop to the set size to get the processed two-dimensional grid image G 2D_1 , the distorted text image Image 0 is converted into RGB format and its background information is randomly replaced to obtain the converted distorted text image Image 1. Then, the converted distorted text image Image 1. Perform tight cropping until the edge of the text is at the edge of the image to remove unnecessary edges and obtain the processed distorted text image. Image 2; Finally, the processed two-dimensional grid map G 2D_1 And the distorted text image after processing Image 2 Perform bilinear interpolation to obtain the real text image Image bm ; The step B2 specifically comprises the following steps: Step B21: Load and process the distorted text images from the training dataset Image 2 Input into the medical document image correction model; Step B22: After the data passes through the first convolution layer of the multi-layer convolution module, it is input into the first batch normalization layer to normalize the output of the first convolution layer. F conv_1 , and then the first ReLU activation function introduces nonlinearity to help the model learn more complex features; the data output by the first ReLU activation function passes through the second convolutional layer to further extract features and downsample, and then enters the second batch normalization layer to normalize the output of the second convolutional layer F conv_2 , the output obtained is F conv_2 It is concatenated with the input data and then passed through a second ReLU activation function. Then, the ResNet network is used to further extract features and gradually reduce the spatial dimension of the features. In the ResNet network, the data is extracted through multiple residual blocks and then processed through multiple layers of dilated convolution, batch normalization, and activation functions. The dilation rate of each layer increases continuously to obtain the features. F grid1 , F grid2 ,......, F gridn ; Step B23: The features obtained by bridging the module F grid1 , F grid2 ,......, F gridn Splicing is performed in the channel dimension, and features are obtained through the first bridge module. F bright1 , after the second bridge module, the feature F bright2 , ..., after the eighth bridge module, the feature F bright8 , the bridge module includes two dilated convolutional layers, a batch normalization layer and a ReLU activation function; Step B24: Concatenate the outputs of the eight bridge modules in the channel dimension to obtain features F bright_concat0 , and then compress the channel dimension through the convolution layer of the convolution fusion module, reduce the number of parameters and integrate the features to obtain the feature F bright_concat1 , and then the output of the convolutional layer is normalized through the batch normalization layer to obtain the feature F bright_concat2 , and then the normalized features F bright_concat2 Apply the ReLU activation function to get the feature F bright_concat3 ; Step B25: Add features F bright_concat3 In the input prediction location point module, the first convolution layer is used to extract features and compress channels, and then the batch normalization layer is used to normalize the features. F bright_concat3_pred , and then the normalized features F bright_concat3_pred Apply the PReLU activation function to introduce nonlinearity; finally, the feature data is converted into the final two-dimensional point position prediction through the second convolution layer, and the predicted two-dimensional feature map is output G pred_2D , two-dimensional feature map G pred_2D The two channel values ​​of each pixel in represents the corresponding two-dimensional point position prediction; Step B26: The predicted two-dimensional feature map G pred_2D Image with distorted text Image 2 Perform bilinear interpolation to obtain the text image after model correction Image" pred , and then with the real text image Image bm , 2D grid graph G 2D_1 and a two-dimensional feature map G pred_2D Input them into the loss function together to calculate the loss L recon , back propagates into the model, and corrects the text image of the model Image" pred and real text images Image bm Calculate the mean square error (MSE) and save the model with the smallest MSE, which is the trained medical document image correction model. P model ; The loss function is expressed as: ;in, α and β is the set hyperparameter, L 1 loss Represents the mean absolute error MAE, L 2 loss Stands for mean square error MSE.

2. According to the grid point-based medical document image correction and recognition method of claim 1, it is characterized in that: In step A, images of medical documents are acquired by methods including camera shooting, mobile phone shooting and flatbed scanning, wherein the images acquired by camera shooting and mobile phone shooting are used as images to be corrected, and the images acquired by flatbed scanning are used as corresponding real images; then, a test data set is established by combining all the images to be corrected and the corresponding real images.

3. According to the grid point based medical document image correction and recognition method of claim 2, it is characterized in that: The medical documents include medical records, medical invoices, prescriptions and physical examination reports.

4. The method for correcting and recognizing medical document images based on grid points according to claim 1, characterized in that: The specific implementation method of step C is: The image to be rectified in the test dataset Image pred_distorted Input the trained medical document image correction model P model , get the predicted two-dimensional feature map G" pred_2D , the predicted two-dimensional feature map G" pred_2D The image to be corrected Image pred_distorted Perform bilinear interpolation to obtain the image corrected by the model Image pred ; The image after the model is rectified Image pred The image to be corrected Image pred_distorted The corresponding real images are compared to evaluate the performance of the model on the test dataset, and the one that passes the evaluation is the final medical document image correction model.

5. The method for correcting and recognizing medical document images based on grid points according to claim 1, characterized in that: In the step D, the open source PyTessart tool is used to perform OCR recognition on the corrected medical document image, and the text recognized by OCR is highlighted to assist medical personnel in quickly locating data to obtain the recognized medical text.

6. The method for correcting and recognizing medical document images based on grid points according to claim 1, characterized in that: The medical document image correction and recognition method is applied to recognize printed medical documents.

7. A medical document image correction and recognition system based on grid points, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps as described in any one of claims 1 to 6 can be implemented.

Images