Image edge detection and res-crnn-based paper medical record digitization information extraction method, system, device and medium

Abstract

The paper medical record digitization information extraction method, system, equipment and medium based on image edge detection and Res-CRNN, the method comprises: sequentially performing gray processing, edge sharpening processing, image line recognition and rotation correction, mask processing and projection transformation on the original image with color information, generating handwritten digital training data through morphological transformation, and performing character recognition on the processed image based on the generated handwritten digital training data and Res-CRNN deep learning model to obtain the final text digital recognition result; the system, equipment and medium are used for realizing the paper medical record digitization information extraction method based on image edge detection and Res-CRNN; the original image is processed through gray processing, edge sharpening processing, image line recognition and rotation correction, and the digitization information extraction of the medical record image is realized through Res-CRNN, so that the paper medical record digitization information extraction method has the characteristics of efficient and accurate extraction of case image information and simple operation steps.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition and medical application, and particularly relates to a paper medical record digitization information extraction method, system, device and medium based on image edge detection and Res-CRNN. BACKGROUND

[0002] The existing cases are mostly saved in the form of paper version, and this saving mode needs a large amount of physical space for storage, and a large amount of manpower and time for saving, management, classification and retrieval protection, and there is a risk of loss, damage and tampering, which may lead to irrecoverable or serious consequences. The paper case is generally in one copy, which is saved in the hands of a doctor or a patient, cannot be shared and cooperated, and cannot be managed digitally. If manual input is selected, a large amount of resources is needed, and the risk of errors and omissions is increased. The existing image segmentation and handwriting recognition technology has no universality, that is, the processing and technology required for different data and requirements are different. The existing image segmentation is based on finding the corresponding table points as the segmentation criterion, but due to the uncontrollable distance, angle and direction of image shooting and pdf printing, and the high requirement for the shooting device, simultaneous digital recognition of multiple images will lead to the failure to find the end points of the table, or the recognition of wrong points.

[0003] The patent with the patent name of "a table recognition method, device, electronic equipment and storage medium", the publication number [CN115759259A] provides a table recognition method, device, electronic equipment and storage medium, wherein the table recognition method comprises: acquiring a to-be-recognized image; extracting a straight line segment end point in the to-be-recognized image; for each straight line segment end point, acquiring the to-be-fitted point of each straight line segment end point according to a preset to-be-fitted point screening rule; fitting the straight line segment end point and the to-be-fitted point of the straight line segment end point to acquire a first candidate cell corner point; according to a candidate corner point merging rule, the first candidate cell corner point is merged to acquire a second candidate cell corner point; and based on the second candidate cell corner point, the table in the to-be-recognized image is reconstructed. However, since the recognition method of the patent needs to be established on the basis of a relatively perfect table image, in the way of image data acquisition by shooting, there are uncontrollable distance, angle and direction, and high requirement for the shooting device, and simultaneous digital recognition of multiple images will lead to the failure to find the end points of the table, or the recognition of wrong points, which cannot be performed through the merging of the straight line segment and the end point.

[0004] The invention with the patent name "a form handwritten number recognition model based on convolutional neural network" and the publication number [CN115527225A] provides a form handwritten number recognition model based on convolutional neural network. The form handwritten number recognition model based on convolutional neural network learns and studies the structural characteristics of LeNet-5 model, builds a convolutional neural network model based on TensorFlow open source platform and Keras framework, so that when the model recognizes form handwritten numbers, the handwritten number pictures of the form are input at the input end and the recognized Arabic numerals are output at the output end. However, the recognition method of the patent can only predict the model for fixed-length numbers one by one, and cannot realize end-to-end prediction of indefinite-length numbers, which is also a defect of traditional convolutional neural network. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the purpose of the present application is to provide a paper medical record digitization information extraction method, system, equipment and medium based on image edge detection and Res-CRNN, which can recognize the table image with errors and has the characteristics of high recognition efficiency and high recognition accuracy. The method comprises the following steps:

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is:

[0007] The paper medical record digitization information extraction method based on image edge detection and Res-CRNN comprises the following steps:

[0008] Step 1, gray-scale processing is performed on the original image with color information to obtain an image with single brightness information;

[0009] Step 2, Canny edge detection is performed on the image with single brightness information obtained in step 1 to obtain an edge sharpening processed image;

[0010] Step 3, line recognition and rotation correction are performed on the edge sharpening processed image obtained in step 2 to obtain a rotation processed image;

[0011] Step 4, mask processing is performed on the rotation processed image obtained in step 3, and the horizontal line and vertical line of the image are found according to morphology, and the intersection points are recognized and marked on the original image;

[0012] Step 5, projection transformation is performed on the original image marked in step 4 to obtain a projection transformed image.

[0013] Step 6, generate handwritten digital training data through morphological transformation, and then perform character recognition on the projected image obtained in step 5 based on the handwritten digital training data and the Res-CRNN deep learning model to obtain the final text and digital recognition result.

[0014] The formula of the gray processing in step 1 is specifically:

[0015] The final gray value is obtained by using the three color channels in the picture, and the gray value = 0.2989 * X red + 0.5870 * X green + 0.1140 * X blue , wherein X red represents the red channel value separated in the image, X green represents the green channel value separated in the image, and X blue represents the green channel value separated in the image.

[0016] The edge sharpening processing in step 2 is specifically:

[0017] The image with single brightness information is converted into an image after edge sharpening processing by using the Canny function of the cv2 library in python;

[0018] The Canny function needs three parameters:

[0019] image, which is the image to be edge detected, and is represented as an image with single brightness information here;

[0020] threshold1, which is the hysteresis threshold, if the gradient of a pixel is greater than this threshold, it is considered as an edge;

[0021] threshold2, which is the main threshold, if the gradient of a pixel is greater than it and connected with the first threshold, it is considered as an edge;

[0022] Step 2.2, the values of the parameters threshold1 and threshold2 in step 2.1 are confirmed respectively by grid tuning.

[0023] The edge sharpening processing in step 3 is specifically:

[0024] Step 3.1, based on cumulative probability Hough transform, and using the HoughLinesP function of the cv2 library in python, the straight lines of the image after edge processing are recognized;

[0025] Step 3.2, the parameters in the HoughLinesP function are determined by grid adjustment, and the straight lines identified in step 3.1 are adjusted, the parameters include: rho, distance accuracy in pixels; theta, angle accuracy in radians; threshold, threshold parameter of the accumulation plane; minLineLength, minimum line length; maxLineGap, maximum distance allowed to connect the same row of points with points;

[0026] Step 3.3, filter out the straight lines with a slope in the range of (-pi / 4, pi / 4) from the straight lines adjusted in step 3.2 and define them as horizontal lines, and perform a weighted average of the slopes on the straight lines with a slope in the range of (-pi / 4, pi / 4) to obtain the slope k, and then rotate the entire table to offset the effect of the slope k.

[0027] The step 4 is specifically:

[0028] Step 4.1, based on the image after rotation processing, a mask area of the table is constructed;

[0029] Step 4.2, in the mask area of the table constructed in step 4.1, the parameters of the specific mask area created in step 4.1 are adjusted respectively by the getStructuringElement function of the opencv library: shape, esize, so as to obtain a full horizontal line graph and a full vertical line graph, wherein, shape is the shape of the kernel, set to rectangle, esize is the size of the kernel;

[0030] Step 4.3, the full horizontal line graph and the full vertical line graph obtained in step 4.2 are overlapped, the intersection points are marked in the overlap of the full horizontal line graph and the vertical line graph image, the grid is oriented, and then the rectangular frame is screened out and marked on the original image.

[0031] The step 5 is specifically:

[0032] Step 5.1, by obtaining the mask code and the marked intersection points, the largest rectangular frame corresponding to the medical record is established on the original image after marking, and the bitwise_or function in the OpenCV computer vision library is used to separate the extracted mask source code in the original image after marking, and the projection transformation is used for correction to obtain the final preprocessed image;

[0033] Step 5.2, in the projection transformation of the last pre-processed image obtained in step 5.1, first, a coordinate system of a target image needs to be defined, and then each pixel point in the original image is mapped to the corresponding position in the target image, and this mapping relationship can be described by a projection matrix, which is determined by a set of corresponding points in the last pre-processed image obtained in step 5.1 and the target image, and the corresponding transformation formula is:

[0034]

[0035] In the formula, u, v represent the coordinates of the original picture, is a transformation matrix, which is composed of four parts, wherein represents a linear transformation;[a 31 a 32 ] represents a translation transformation;[a 13 a 23 ] T represents a perspective transformation, and the transformation formula becomes:

[0036]

[0037]

[0038] In the formula, x, y represent the coordinates of the transformed picture, and x, y are calculated by u, v to realize the projection transformation of the image.

[0039] The step 6 is specifically:

[0040] Step 6.1, by downloading the handwritten digits in the training set and test set in the Mnist data set in the network, the image and label information are classified, each handwritten digit image in the Mnist data is a 0-9 gray handwritten digit image composed of 28x28 pixel points, black background and white characters, and the image pixel value is 0-255;

[0041] Step 6.2, the Mnist data set downloaded in step 6.1 and the data in the existing medical record image are spliced and combined to form an indefinite length handwritten data set, in the splicing process, the negative sign and the decimal point are substituted into the indefinite length handwritten data for splicing, and the existing handwritten data is segmented and combined with the handwritten digit data set;

[0042] Step 6.3, the indefinite length handwritten data set obtained in step 6.2 is processed for robustness:

[0043] Step 6.3.1, data enhancement: according to the specific medical record identification requirements, by reading the indefinite length handwritten data, the handwritten digit image in the Mnist data is randomly extracted and cut in the corresponding folder for the single digit to be extracted; by binarizing the handwritten digit image in the cut Mnist data, the pixel points where the single handwritten digit image is located are found, cut and reshaped; the images that have been cut and reshaped are rotated by a random degree and scaled by a multiple, and the data is spliced; the spliced data is rotated and scaled as a whole, and a random number of pixel points are added to the top, bottom, left and right of the generated data image, and the image is resized in size to ensure that the image size entering the training is uniform; finally, the indefinite length handwritten digit image after data enhancement is obtained;

[0044] Step 6.3.2, adding noise to the indefinite length handwritten digit image after data enhancement generated in step 6.3.1: adding random-valued Gaussian noise and salt and pepper noise to the indefinite length handwritten digit image after data enhancement generated in step 6.3.1 to simulate the noise influence in the real scene;

[0045] Step 6.4, using the indefinite length handwritten data set after robustness processing in step 6.3 as data support, using CRNN network to recognize the projected image, the whole CRNN network structure contains three parts, which are indefinite length handwritten data into CNN convolution layer, indefinite length handwritten data from CNN convolution layer to RNN recursive layer and indefinite length handwritten data from RNN convolution layer to CTC transcription layer:

[0046] Step 6.4.1, indefinite length handwritten data into CNN convolution layer:

[0047] The CNN convolution layer is composed of four maximum pooling layers, which can extract useful feature information from the input indefinite length handwritten data, and the network structure information is as follows: the network reduces the height of the initial image by half four times, but only reduces the width by half twice, and retains the information with larger width and smaller height in the text image. After determining the network structure information, the feature vector sequence is extracted from the feature information, each feature vector contains 512-dimensional features, and the feature vector sequence represents the connection of specific feature image pixels. These feature vectors form a sequence and are used as input data for RNN recursive layer;

[0048] Step 6.4.2, indefinite length handwritten data from CNN convolution layer to RNN recursive layer:

[0049] In the processing of the sequence information of the image, the RNN recursive layer receives the indefinite length handwritten data processed by the CNN convolutional layer in step 6.4.1, and is transferred to the RNN recursive layer, in which an LSTM network is used as the recursive layer, one LSTM is passed forward, the other LSTM is passed backward, and they are combined into a bidirectional LSTM, so as to obtain a plurality of 512-dimensional feature vectors, and then the probability distribution of each input is obtained by predicting the matrix region corresponding to the feature vector, and the vector corresponding to the probability distribution is taken as the input result of the CTC transcription layer;

[0050] Step 6.4.3, the indefinite length handwritten data is transferred from the RNN recursive layer to the CTC transcription layer:

[0051] The CTC transcription layer receives the indefinite length handwritten data processed by the RNN recursive layer in step 6.4.2;

[0052] As an end-to-end network, the CRNN has all possible output characters, such as numbers, decimal points and negative signs, which are represented by set, so the total character set of the recognition task is represented as: set'=set∪{null},

[0053] Suppose the output corresponding to the LSTM network is a sequence of length T: f=(f1,f2,…,f T In the CTC transcription layer, the length T sequence obtained by training is translated to obtain the final recognition result, and the CTC transcription layer needs to decode the information obtained by the CNN convolutional layer and the RNN recursive layer to obtain reference data information. First, a character in the character set set' is assigned to each f t The probability of each possible sequence in the entire decoding process is represented as:

[0054]

[0055] After removing the repeated characters and null in the sequence π, the final predicted character I can be obtained, that is, B(π t )=I, and after the one-to-many mapping is de-duplicated and de-empty, the probability of each mapping for I is summed up, that is, the result of each possible output is obtained;

[0056] Step 6.5, the CRNN network for recognizing the projected image in step 6.4 is improved to optimize the recognition of the projected image to obtain the image recognition result based on the Res-CRNN network. In the Res-CRNN, the RNN recursive layer in the CRNN is improved, so that the output of the RNN recursive layer is changed to:

[0057] F(X)=F main(X) + F res (X),

[0058] Wherein, F main (X) is the output of two layers of convolutional networks in the main network, and F res (X) is the output after the residual connection passes through a layer of convolutional layer, and the result of the image after the projection transformation is obtained after the Res-CRNN network training.

[0059] The paper medical record digital information extraction system based on image edge detection and Res-CRNN comprises the following modules:

[0060] The image preprocessing module: medical record image detection preprocessing, including gray processing, Canny edge detection, line recognition and rotation correction of the original image;

[0061] The image correction and positioning module: medical record image correction and image region positioning, including mask processing and projection transformation of the medical record image preprocessed by the image preprocessing module;

[0062] The digital information extraction module: based on Res-CRNN, the medical record image corrected and region positioned by the image correction and positioning module is subjected to digital information extraction.

[0063] The paper medical record digital information extraction device based on image edge detection and Res-CRNN comprises:

[0064] The memory: used for storing the computer program for realizing the paper medical record digital information extraction method based on image edge detection and Res-CRNN;

[0065] The processor: used for realizing the paper medical record digital information extraction method based on image edge detection and Res-CRNN when the computer program is executed.

[0066] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize the steps of the paper medical record digital information extraction method based on image edge detection and Res-CRNN.

[0067] Compared with the prior art, the beneficial effects of the present application are:

[0068] 1、The present application converts the color information image into a single luminance information image by carrying out gray processing on the original image, which can reduce irrelevant information in pixels and eliminate the influence of redundant information on processing speed in the table line recognition process.

[0069] 2、The present application can compensate the outline of the image, enhance the edge and gray level jump of the image, make the image clear, eliminate the error existing in the image shooting and actual, and improve the recognition accuracy by edge sharpening processing of the image to determine the region boundary.

[0070] 3、The present application can efficiently complete the recognition task when the table image has error by image line recognition and rotation correction.

[0071] 4、The present application can accurately obtain the specified region of the table by intersection recognition, and improve the recognition efficiency of the characters in the frame line region.

[0072] 5、The present application can form the indefinite length handwritten data set by splicing and combining the Mnist data set and the data in the existing medical record image, provide training data support for the training of Res-CRNN, effectively assist in recognizing the characters in the frame line region, and then realize the digital information extraction of the medical record image through Res-CRNN, effectively improve the recognition speed and accuracy of the characters in the frame line region, and simplify the operation steps.

[0073] In summary, the present application has the characteristics of fast recognition speed, high accuracy and simple operation steps by gray processing, edge sharpening processing of the original image, image line recognition and rotation correction, intersection recognition, and Res-CRNN for realizing the digital information extraction of the medical record image. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 The CRNN flowchart of the present application.

[0075] Figure 2 The residual connection schematic diagram with convolution layer of the present application.

[0076] Figure 3 The original image of the embodiment of the present application.

[0077] Figure 4 (a) is the image before gray processing of the embodiment of the present application, Figure 4 (b) is the image after gray processing of the embodiment of the present application.

[0078] Figure 5 (a) is the image before edge sharpening of the embodiment of the present application, Figure 5 (b) is the image with the edge sharpening parameter [30, 150] of the embodiment of the present application, Figure 5 (c) is the image with the edge sharpening parameter [30, 100] of the embodiment of the present application, Figure 5 (d) is the image with the edge sharpening parameter [100, 150] of the embodiment of the present application.

[0079] Figure 6(a) is a table for the horizontal line screening of the embodiment of the present application, Figure 6 (b) is a table for the horizontal line screening of the embodiment of the present application.

[0080] Figure 7 (a) is a pre-rotation correction effect diagram of the embodiment of the present application, Figure 7 (b) is a post-rotation correction effect diagram of the embodiment of the present application.

[0081] Figure 8 (a) is a full horizontal line diagram of the embodiment of the present application, Figure 8 (b) is a full vertical line diagram of the embodiment of the present application, Figure 8 (c) is a horizontal line and vertical line superimposed diagram of the embodiment of the present application, Figure 8 is an intersection extraction diagram of the embodiment of the present application.

[0082] Figure 9 (a) is a pre-projection transformation effect diagram of the embodiment of the present application, Figure 9 (b) is a post-projection transformation effect diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0083] The present application will be described in detail below with reference to the accompanying drawings.

[0084] The paper medical record digital information extraction method based on image edge detection and Res-CRNN includes the following steps:

[0085] Step 1, performing gray processing on the original image with color information to obtain an image with single brightness information;

[0086] Step 2, performing Canny edge detection on the image with single brightness information obtained in step 1 to obtain an edge sharpening processed image;

[0087] Step 3, performing line recognition and rotation correction on the edge sharpening processed image obtained in step 2 to obtain a rotation processed image;

[0088] Step 4, performing mask processing on the rotation processed image obtained in step 3, finding the horizontal line and vertical line of the image according to morphology, and identifying the intersection and marking on the original image;

[0089] Step 5, performing projection transformation on the original image marked in step 4 to obtain a projection transformed image;

[0090] Step 6, generating handwritten digital training data through morphological transformation, and performing character recognition on the projection transformed image obtained in step 5 based on the handwritten digital training data and the Res-CRNN deep learning model to obtain the final text and digital recognition result.

[0091] The formula of the gray processing in step 1 is specifically as follows:

[0092] The final gray value is obtained by using the three color channels in the picture, and the gray value = 0.2989 * X red + 0.5870 * X green + 0.1140 * X blue , wherein X red represents the red channel value in the image separation, X green represents the green channel value in the image separation, and X blue represents the green channel value in the image separation.

[0093] The edge sharpening processing in step 2 is specifically as follows:

[0094] The image with single luminance information is converted into an image after edge sharpening processing by using the Canny function of the cv2 library in python;

[0095] The Canny function is a function for edge detection, and it itself needs three parameters:

[0096] image, that is, the image to be subjected to edge detection, which is represented as the image with single luminance information here;

[0097] threshold1, that is, the hysteresis threshold value, if the gradient of a pixel is greater than the threshold value, it is considered to be an edge;

[0098] threshold2, that is, the main threshold value, if the gradient of a pixel is greater than it and connected with the first threshold value, it is considered to be an edge;

[0099] Step 2.2, the values of the parameters threshold1 and threshold2 in step 2.1 are confirmed by grid tuning.

[0100] The edge sharpening processing in step 3 is specifically as follows:

[0101] Step 3.1, based on cumulative probability Hough transformation, and by using the HoughLinesP function of the cv2 library in python, the straight lines of the image after edge processing are recognized;

[0102] Step 3.2, the parameters in the HoughLinesP function are determined by grid tuning, and the straight lines recognized in step 3.1 are adjusted, and the parameters include: rho, distance precision in pixels; theta, angle precision in radians; threshold, threshold parameter of the cumulative plane; minLineLength, minimum line segment length; maxLineGap, maximum distance allowed to connect the same row of points with points;

[0103] Step 3.3, in the straight line adjusted in step 3.2, the straight line with the slope in the range of (-π / 4, π / 4) is screened and defined as a horizontal line, and the weighted average of the slope on the straight line with the slope in the range of (-π / 4, π / 4) is obtained The slope k, and then the whole table is rotated to offset the influence of the slope k.

[0104] The step 4 is specifically:

[0105] Step 4.1, based on the image after the rotation processing, a mask area of the table is constructed, that is, the region of interest during operation;

[0106] Step 4.2, in the mask area of the table constructed in step 4.1, the parameters of the specific mask area created in step 4.1 are adjusted respectively by the getStructuringElement function of the opencv library: shape, esize, so as to obtain a full horizontal line graph and a full vertical line graph, wherein shape is the shape of the kernel, which is set to rectangle, and esize is the size of the kernel;

[0107] Step 4.3, the full horizontal line graph and the full vertical line graph obtained in step 4.2 are overlapped, the intersection points are marked in the coincidence of the full horizontal line graph and the vertical line image, the grid is oriented, and then the rectangular frame is screened out and marked on the original image.

[0108] The step 5 is specifically:

[0109] Step 5.1, by obtaining the mask and the marked intersection points, the largest rectangular frame corresponding to the medical record is established on the original image after marking, and the bitwise_or function in the OpenCV computer vision library is used to separate the extracted mask source code in the original image after marking, and the projection transformation is corrected to obtain the final preprocessed image;

[0110] Step 5.2, in the projection transformation of the final preprocessed image obtained in step 5.1, a coordinate system of a target image needs to be defined first, and then each pixel point in the original image is mapped to the corresponding position in the target image. This mapping relationship can be described by a projection matrix, which is determined by a set of corresponding points in the final preprocessed image obtained in step 5.1 and the target image. The corresponding transformation formula is:

[0111]

[0112] In the formula, u, v represent the coordinates of the original picture, is a transformation matrix, which is composed of four parts, wherein represents a linear transformation;[a 31 a 32 ] represents a translation transformation;[a 13 a 23 ] T represents a perspective transformation, the transformation formula becomes:

[0113]

[0114]

[0115] In the formula, x, y represent the coordinates of the transformed picture, and x, y are calculated by u, v to realize the projection transformation of the image.

[0116] The step 6 is specifically:

[0117] Step 6.1, by downloading the handwritten digits in the training set and test set in the Mnist data set in the network, classifying the image and label information, each handwritten digit image in the Mnist data is a 0-9 gray handwritten digit image composed of 28*28 pixel points, black background and white characters, and the image pixel value is 0-255;

[0118] Step 6.2, the Mnist data set downloaded in step 6.1 and the data in the existing medical record image are spliced and combined to form an indefinite length handwritten data set. In the splicing process, the minus sign and the decimal point are substituted into the indefinite length handwritten data for splicing, and the existing handwritten data is merged and spliced with the handwritten digit data set.

[0119] Step 6.3, the indefinite length handwritten data set obtained in step 6.2 is processed for robustness:

[0120] Step 6.3.1, data enhancement: according to the specific medical record recognition requirement, by reading the indefinite length handwritten data, the handwritten digit image in the Mnist data is randomly extracted and cut in the corresponding folder; by carrying out binary processing on the cut Mnist data, the pixel points where the single handwritten digit image is located are found, cut and reshaped; the image that has been cut and reshaped is rotated by a random number of degrees left and right and scaled by a multiple, and the data is spliced; the spliced data is rotated and scaled as a whole, and the generated data image is expanded by a random number of pixels up, down, left and right, and then the image is resized in size to ensure that the image size entering the training is uniform; finally, the indefinite length handwritten digit image after data enhancement is obtained;

[0121] Step 6.3.2, adding noise to the data-enhanced indefinite-length handwritten digit images generated in step 6.3.1: In order to make the generated data more robust, Gaussian noise and salt and pepper noise with random values are added to the data-enhanced indefinite-length handwritten digit images generated in step 6.3.1 to simulate the noise effects in real scenarios;

[0122] Referring to Figure 1 , step 6.4, using the CRNN network to recognize the projected images based on the indefinite-length handwritten data set that has been robustly processed in step 6.3, the entire CRNN network structure includes three parts, which are the conversion of indefinite-length handwritten data into a CNN convolutional layer, the conversion of indefinite-length handwritten data from a CNN convolutional layer to an RNN recurrent layer, and the conversion of indefinite-length handwritten data from an RNN convolutional layer to a CTC transcription layer:

[0123] Step 6.4.1, conversion of indefinite-length handwritten data into a CNN convolutional layer:

[0124] The CNN convolutional layer is composed of four max-pooling layers, which can extract useful feature information from the input indefinite-length handwritten data. The network structure information is as follows: the network retains the information with a larger width and a smaller height in the text image by reducing the height of the initial image by half four times but reducing the width by half only twice. After determining the network structure information, the feature vector sequence is extracted from the feature information, each feature vector contains 512-dimensional features, and the feature vector sequence represents the connection of specific feature image pixels. These feature vectors form a sequence and are used as input data for the RNN recurrent layer;

[0125] Step 6.4.2, conversion of indefinite-length handwritten data from a CNN convolutional layer to an RNN recurrent layer:

[0126] In processing the sequence information of the image, the RNN recurrent layer receives the indefinite-length handwritten data processed by the CNN convolutional layer in step 6.4.1 and converts it into the RNN recurrent layer. In the RNN recurrent layer, an LSTM network is used as the recurrent layer. One LSTM is passed forward, another LSTM is passed backward, and they are combined into a bidirectional LSTM to obtain multiple 512-dimensional feature vectors. The probability distribution of each input is obtained by predicting the matrix region corresponding to these feature vectors, and the vector corresponding to this probability distribution is used as the input result of the CTC transcription layer;

[0127] Step 6.4.3, conversion of indefinite-length handwritten data from an RNN recurrent layer to a CTC transcription layer:

[0128] The CTC transcription layer receives the indefinite-length handwritten data processed by the RNN recurrent layer in step 6.4.2;

[0129] CRNN as an end-to-end network, all possible output characters are numbers, decimal points and negative signs, denoted by set, then the total character set of the recognition task is represented as: set'=set∪{null},

[0130] Suppose the output corresponding to the LSTM network is a sequence of length T: f=(f1,f2,…,f T ), the length T sequence obtained by training in the CTC transcription layer is translated to obtain the final recognition result, and the CTC transcription layer needs to decode the information obtained by the CNN convolution layer and the RNN recursive layer to obtain reference data information. First, assign a character in the character set set' to each f t , then the probability of each possible sequence in the entire decoding process is represented as:

[0131]

[0132] After removing the repeated characters and null in the sequence π, the final predicted character I can be obtained, that is, B(π t )=I, and after the one-to-many mapping is removed and the empty operation is performed, the probability of each mapping for I is summed up, that is, the result of each possible output is obtained.

[0133] Referring to Figure 2 , step 6.5, the CRNN network for recognizing the image after the projection transformation of step 6.4 is improved to optimize the recognition of the image after the projection transformation to obtain the image recognition result based on the Res-CRNN network. In Res-CRNN, the RNN recursive layer in CRNN is improved, so that the output of the RNN recursive layer is changed to:

[0134] F(X)=F main (X)+F res (X),

[0135] Where F main (X) is the output of the two convolutional networks in the main network, and F res (X) is the output of the residual connection after passing through a convolutional layer, and then the result of the image after the projection transformation is obtained after the Res-CRNN network is trained.

[0136] The paper-based medical record digital information extraction system based on image edge detection and Res-CRNN includes the following modules:

[0137] The image preprocessing module: medical record image detection preprocessing, including gray processing, Canny edge detection, line recognition and rotation correction of the original image, for realizing steps 1-3 of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN of the application;

[0138] The image correction and positioning module: medical record image correction and image region positioning, including mask processing and projection transformation of the medical record image preprocessed by the image preprocessing module, for realizing steps 4-5 of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN of the application;

[0139] The digitization information extraction module: based on Res-CRNN, the medical record image corrected and region positioned by the image correction and positioning module is subjected to digitization information extraction, for realizing step 6 of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN of the application.

[0140] The paper medical record digitization information extraction equipment based on image edge detection and Res-CRNN, comprising:

[0141] The memory: for storing the computer program for realizing the paper medical record digitization information extraction method based on image edge detection and Res-CRNN;

[0142] The processor: for realizing the paper medical record digitization information extraction method based on image edge detection and Res-CRNN when the computer program is executed.

[0143] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the equipment specifically related to the paper medical record digitization information extraction method based on image edge detection and Res-CRNN, and connects each part of the entire equipment specifically related to the paper medical record digitization information extraction method based on image edge detection and Res-CRNN through various interfaces and lines.

[0144] The processor implements the steps of the above-mentioned paper medical record digitization information extraction method based on image edge detection and Res-CRNN, for example: step 1, performing gray processing on the original image with color information to obtain an image with single brightness information; step 2, performing Canny edge detection on the image with single brightness information obtained in step 1 to obtain an edge sharpening processed image; step 3, performing line recognition and rotation correction on the edge sharpening processed image obtained in step 2 to obtain a rotation processed image; step 4, performing mask processing on the rotation processed image obtained in step 3, and finding horizontal lines and vertical lines of the image according to morphology, and performing intersection recognition and marking on the original image; step 5, performing projection transformation on the original image marked in step 4 to obtain a projection transformed image; step 6, generating handwritten digital training data through morphological transformation, and performing text recognition on the projection transformed image obtained in step 5 based on the handwritten digital training data and the Res-CRNN deep learning model to obtain a final text digital recognition result; and the paper medical record digitization information extraction method based on image edge detection and Res-CRNN is implemented.

[0145] Alternatively, the processor implements the functions of each module in the above-mentioned system when executing the computer program, for example: an image preprocessing module: medical record image detection preprocessing, including gray processing, Canny edge detection, line recognition and rotation correction on the original image; an image correction and positioning module: medical record image correction and image region positioning, including mask processing and projection transformation on the medical record image preprocessed by the image preprocessing module; a digitization information extraction module: based on Res-CRNN, performing digitization information extraction on the medical record image corrected and region positioned by the image correction and positioning module; and outputting the result of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN.

[0146] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing preset functions, which are used to describe the execution process of the computer program in the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device. For example, the computer program can be divided into an image preprocessing module, an image correction and positioning module, and a digitalization information extraction module, and the specific functions of each module are as follows: the image preprocessing module: medical record image detection preprocessing, including gray processing, Canny edge detection, line recognition and rotation correction of the original image; the image correction and positioning module: medical record image correction and image region positioning, including mask processing and projection transformation of the medical record image preprocessed by the image preprocessing module; the digitalization information extraction module: based on Res-CRNN, the medical record image corrected and region positioned by the image correction and positioning module is subjected to digitalization information extraction; and the result of the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN is output.

[0147] The paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above is an example of the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device, and does not constitute a limitation on the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device. It can include more components than the above, or combine certain components, or different components, for example, the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device can also include an input / output device, a network access device, a bus and the like.

[0148] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the paper medical record digitalization information extraction method based on image edge detection and Res-CRNN device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory.

[0149] The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data (such as audio data, a phone book, etc.) created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0150] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN.

[0151] The modules / units of the system integration of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN can be stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products.

[0152] The application realizes all or part of the processes of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN, and can also be completed by instructing related hardware through a computer program, the computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the paper medical record digitization information extraction method based on image edge detection and Res-CRNN when being executed by a processor. The computer program includes computer program codes, and the computer program codes can be in the form of source codes, object codes, executable files or preset intermediate forms, etc.

[0153] The computer readable storage medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. that can carry the computer program codes.

[0154] It should be noted that the computer readable storage medium contains contents that can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0155] It should be noted that the embodiments of the present application can be realized by hardware, software or a combination of software and hardware. The hardware part can be realized by special logic; the software part can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or a specially designed hardware.

[0156] Those skilled in the art can understand that the above-mentioned devices and methods can be realized by computer executable instructions and / or contained in processor control code, for example, such code is provided on a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware) or a data carrier such as an optical or electronic signal carrier. The device of the present application and its modules can be realized by hardware circuit such as ultra large scale integrated circuit or gate array, semiconductor such as logic chip, transistor, or programmable hardware device such as field programmable gate array, programmable logic device, etc., can also be realized by software executed by various types of processors, and can also be realized by a combination of the above hardware circuit and software, such as firmware.

[0157] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any modification, equivalent replacement and improvement made by those skilled in the art within the technical range disclosed by the present application, as long as it is within the spirit and principle of the present application, should be covered within the protection scope of the present application.

[0158] Embodiment:

[0159] Referring to Figure 3 , the data comes from the orthokeratology medical records of an orthokeratology center of a hospital (the identity information has been desensitized), the image size is 1984x2976, the number is 500, and the image segmentation and processing are based on OpenCV computer vision library.

[0160] The paper medical record digital information extraction method based on image edge detection and Res-CRNN includes the following steps:

[0161] Referring to Figure 4 (a) and Figure 4 (b), step 1, the original image is processed by gray scale, and the comparison chart before and after gray scale is shown in Figure 4 (a) and Figure 4 (b), wherein, Figure 4(a) is the image before the grayscale processing, which is a three-channel image, and Figure 4 (b) is the image after the grayscale processing, which is a single-channel image, which simplifies the complexity of the processing and highlights the image details;

[0162] Referring to Figure 5 (a) to (d), step 2, the image after the grayscale processing in step 1 is subjected to Canny edge sharpening processing. It should be noted that the minimum threshold and the maximum threshold need to be provided during the Canny edge detection process. The range of the threshold value can control the accuracy of the edge detection. The smaller the threshold value, the more details will be obtained, and more edges of the image will be extracted. Figure 5 (a) is the image before the edge processing, Figure 5 (b) the corresponding threshold range is [30, 150], the image (c) corresponds to the threshold range [30, 100], and the image (d) corresponds to the threshold range [100, 150]. As can be seen from the figure, in the medical record data this time, the image (d) with the threshold value [100, 150] can provide all the information as much as possible while avoiding the existence of redundant information.

[0163] Referring to Figure 6 (a), Figure 6 (b), step 3, the straight lines in the image after the edge processing in step 2 are identified by performing Hough transform on the image. In order to exclude redundant lines, we adjust the controllable distance accuracy, angle accuracy, threshold parameters of the accumulated plane and minimum length and other parameters to obtain effective line identification pictures. Considering that the requirement is to identify the horizontal line slope on the table, a relatively loose strategy is adopted in the threshold selection. Only the table frame line of each row needs to be identified and marked in the form of a red line. In order to convert the random rotation part in the image to be positive, we filter out the straight lines with a slope of ±π / 4 and mark them on the original image. Referring to Figure 7 (a), Figure 7 (b) corrects the image according to the average offset degree of the straight lines to obtain the image after the rotation processing.

[0164] Referring to Figure 8(a) to (d), step 4, after the image after the rotation processing obtained in step 3 is processed by using the getStructuringElement function. Unlike the fixed threshold, the getStructuringElement function determines different thresholds according to the weighted average value of the image pixel points around each pixel point, so as to effectively control the change of the threshold in different regions. The advantage of this is that the brightness of different regions can be adaptively adjusted according to the distribution of the pixel points, so as to control the brightness of the image regions by the height of the threshold, and by using the image erosion and expansion operation after the image after the getStructuringElement function, the mask corresponding to the image is obtained, and the full horizontal line graph Figure 8 (a), the full vertical line graph Figure 8 (b), the horizontal line and vertical line superimposed graph Figure 8 (c) and the extracted intersection graph Figure 9 (d).

[0165] Referring to Figure 9 (a), Figure 9 (b), step 5, by obtaining the mask and extracting the intersection, the largest rectangular frame corresponding to the medical record is established, the original image is separated from the extracted mask source by using the bitwise_or function in the OpenCV computer vision library, and the original image after the labeling in step 4 is corrected by projection transformation to obtain the image after the projection transformation, as shown in ​ (b), the white color is the effect diagram after the frame line is extracted. On this basis, since the medical record template is certain, after the medical record is fixed in the specified region, the corresponding cell ratio obtained is consistent, and the content of the specified cell can be obtained by cutting.

[0166] Step 6, the digital information extraction of the cell content obtained in step 5 is performed. The target of the network model training in the digital information extraction process is to minimize the CTC loss function. When training CRNN and Res-CRNN, 20000 pictures are generated in total, 15000 of which are generated pictures, which are used as a training set, 4000 pictures are used as a verification set, and 1000 pictures are used as a test set. The experimental results are shown in the CRNN and Res-CRNN result comparison table. The Res-CRNN of the present application is superior to the CRNN network in terms of loss and recognition rate.

[0167] CRNN and Res-CRNN result comparison table

[0168]

Claims

1. A method for extracting information from paper medical records based on image edge detection and Res-CRNN, characterized in that, The method comprises the following steps: Step 1, gray processing is performed on an original image with color information to obtain an image with single brightness information; Step 2, Canny edge detection is performed on the image with single brightness information obtained in step 1 to obtain an edge sharpening processed image; Step 3, line recognition and rotation correction are performed on the edge sharpening processed image obtained in step 2 to obtain a rotation processed image; Step 4, mask processing is performed on the rotation processed image obtained in step 3, and horizontal lines and vertical lines of the image are found according to morphology, and the intersection points are identified and marked on the original image, and the specific steps of step 4 are as follows: Step 4.1, based on the image after rotation processing, a mask area of the table is constructed; Step 4.2, in the mask area of the table constructed in step 4.1, the parameters of the specific mask area created in step 4.1 are adjusted through the getStructuringElement function of the opencv library: shape, esize, to obtain a full horizontal line image and a full vertical line image, wherein shape is the shape of the kernel, and esize is the size of the kernel; Step 4.3, the full horizontal line image and the full vertical line image obtained in step 4.2 are overlapped, the intersection points are marked in the overlap of the full horizontal line image and the vertical line image, the grid is oriented, and then the rectangular frame is screened out and marked on the original image; Step 5, projection transformation is performed on the original image marked in step 4 to obtain a projection transformed image, and the specific steps of step 5 are as follows: Step 5.1, a maximum rectangular frame corresponding to the medical record is established on the marked original image through the obtained mask and the marked intersection points, the extracted mask source code is separated out in the marked original image by using the bitwise_or function in the OpenCV computer vision library, and the last preprocessed image is obtained through projection transformation; Step 5.2, in the projection transformation of the last preprocessed image obtained in step 5.1, a coordinate system of a target image needs to be defined first, and then each pixel point in the original image is mapped to the corresponding position in the target image, and the mapping relationship can be described by a projection matrix, which is determined by a set of corresponding points in the last preprocessed image obtained in step 5.1 and the target image, and the corresponding transformation formula is as follows: where u, v represent the coordinates of the original picture, is the transformation matrix, which is composed of four parts, wherein represents a linear transformation;[a 31 a 32 ] represents a translation transformation;[a 13 a 23 ] T represents a perspective transformation, and the transformation formula becomes: In the formula, x and y represent the coordinates of the transformed picture, and x and y are calculated through u and v to realize the projection transformation of the image; Step 6, handwritten digital training data are generated through morphological transformation, and then the projection transformed image obtained in step 5 is subjected to character recognition based on the handwritten digital training data and a Res-CRNN deep learning model to obtain a final text and digital recognition result. 2.The method of claim 1, wherein, The formula of the gray processing in step 1 is specifically as follows: The final gray value is obtained using the three color channels in the picture, gray value = 0.2989 * X red + 0.5870 * X green + 0.1140 * X blue wherein X red represents the red channel value in the image separation, X green represents the green channel value in the image separation, X blue represents the green channel value in the image separation. 3.The method of claim 1, wherein, The edge sharpening processing in step 2 is specifically as follows: Step 2.1, the Canny function of the cv2 library in python is used to convert the image with single brightness information into an edge sharpening processed image; The Canny function needs three parameters: image: the image to be edge detected, represented as an image of single luminance information here; threshold1: the hysteresis threshold, if the gradient of a pixel is greater than this threshold, it is considered as an edge; threshold2: the main threshold, if the gradient of a pixel is greater than it and connected with the first threshold, it is considered as an edge; Step 2.2, the values of the parameters threshold1 and threshold2 in step 2.1 are determined by grid tuning. 4.The method of claim 1, wherein, The edge sharpening processing in step 3 is specifically: Step 3.1, based on cumulative probability Hough transform, and using the HoughLinesP function of the cv2 library in python, the straight lines of the image processed by the edge are identified; Step 3.2, the parameters in the HoughLinesP function are determined by grid tuning to adjust the straight lines identified in step 3.1, including: rho, distance precision in pixels; theta, angle precision in radians; threshold, threshold parameter of the accumulation plane; minLineLength, minimum line length; maxLineGap, the maximum distance allowed to connect the same row of points with points; Step 3.3, in the straight lines adjusted in step 3.2, the straight lines with a slope in the range of (-π / 4, π / 4) are selected and defined as horizontal lines, and the weighted average of the slope of the straight lines with a slope in the range of (-π / 4, π / 4) is obtained to obtain the slope k, and the entire table is rotated to offset the influence of the slope k. 5.The method of claim 1, wherein, The step 6 is specifically: Step 6.1, by downloading the handwritten digits in the training set and test set in the Mnist dataset in the network, the image and label information are classified, each handwritten digit image in the Mnist data is a 0-9 gray handwritten digit image composed of 28x28 pixel points, black background and white characters, the image pixel value is 0-255; Step 6.2, the Mnist dataset downloaded in step 6.1 and the data in the existing medical record image are spliced and combined to form an indefinite length handwritten data set, in the splicing process, the negative sign and decimal point are substituted into the indefinite length handwritten data for splicing, and the existing handwritten data is merged and spliced with the handwritten digit dataset; Step 6.3, the indefinite length handwritten data set obtained in step 6.2 is processed for robustness: Step 6.3.1, data enhancement: according to the specific medical record identification requirements, by reading the indefinite length handwritten data, the handwritten digit images in the Mnist data are randomly extracted and cut in the corresponding folder for the single digit to be extracted; by binarizing the handwritten digit images in the cut Mnist data, the pixel points where the single handwritten digit image is located are found, cut and reshaped; the images that have been cut and reshaped are rotated by a random degree and scaled by a multiple, and the data is spliced; the spliced data is rotated and scaled as a whole, and a random number of pixel points are added to the top, bottom, left and right of the generated data image, and the image is resized to ensure that the image size entering the training is uniform; finally, the indefinite length handwritten digit images after data enhancement are obtained; Step 6.3.2, adding noise to the indefinite length handwritten digit images after data enhancement generated in step 6.3.1: adding random-valued Gaussian noise and salt and pepper noise to the indefinite length handwritten digit images after data enhancement generated in step 6.3.1 to simulate the noise influence in the real scene; Step 6.4, using the indefinite length handwritten data set after robustness processing in step 6.3 as data support, using CRNN network to recognize the projected transformed image, the whole CRNN network structure contains three parts, which are indefinite length handwritten data into CNN convolution layer, indefinite length handwritten data from CNN convolution layer to RNN recursive layer and indefinite length handwritten data from RNN convolution layer to CTC transcription layer: Step 6.4.1, indefinite length handwritten data into CNN convolution layer: The CNN convolution layer is composed of four max pooling layers, which can extract useful feature information from the input indefinite length handwritten data, and the network structure information is as follows: the network retains the information of the text image with larger width and smaller height by reducing the height of the initial image by half four times and reducing the width by half only twice. After determining the network structure information, the feature vector sequence is extracted from the feature information, each feature vector contains 512-dimensional features, and the feature vector sequence represents the connection of specific feature image pixels. These feature vectors form a sequence and are used as input data for the RNN recursive layer; Step 6.4.2, indefinite length handwritten data from CNN convolution layer to RNN recursive layer: In processing the sequence information of the image, the RNN recursive layer receives the indefinite length handwritten data processed by the CNN convolution layer in step 6.4.1 and enters the RNN recursive layer. In the RNN recursive layer, an LSTM network is used as the recursive layer. One LSTM is passed forward and the other LSTM is passed backward, and they are combined into a bidirectional LSTM to obtain multiple 512-dimensional feature vectors. The matrix region corresponding to these feature vectors is predicted to obtain the probability distribution of each input, and the vector corresponding to the probability distribution is used as the input result of the CTC transcription layer; Step 6.4.3, the indefinite length handwritten data is converted into the CTC transcription layer by the RNN recursive layer: The CTC transcription layer receives the indefinite length handwritten data processed by the RNN recursive layer in step 6.4.2; As an end-to-end network, the CRNN has all possible output characters, including numbers, decimal points and negative signs, denoted by set, so the total character set of the recognition task is represented as: set'=set∪{null}, Assuming that the output of the corresponding LSTM network is a sequence of length T: f = (f1, f2,..., f T ), the final recognition result is translated from the trained sequence of length T in the CTC transcription layer. The CTC transcription layer needs to decode the information obtained by the CNN convolution layer and the RNN recursive layer to obtain reference data information. First, assign a character in the character set set' to each f t , and the probability of each possible sequence in the prediction sequence is represented as: After removing the repeated characters and null in sequence π, the final predicted character I, i.e. B(π t ) = I, can be obtained. After removing the repeated and null mapping, the probability of each mapping for I is added, i.e. the result of each possible output is obtained. Step 6.5, the CRNN network for recognizing the image after the projection transformation in step 6.4 is improved to optimize the recognition of the image after the projection transformation, and the image recognition result based on the Res-CRNN network is obtained, in the Res-CRNN, the RNN recursive layer in the CRNN is improved, so that the output of the RNN recursive layer is changed to: F(X) = F main (X) + F res (X), where F main (X) is the output of the two-layer convolutional network in the main network, and F res (X) is the output of the residual connection after a layer of convolutional layer, and the result of the image after the projection transformation is obtained.

6. The paper medical record digitization information extraction system based on image edge detection and Res-CRNN, characterized in that, Comprise the following modules: Image preprocessing module: medical record image detection preprocessing, including gray processing, Canny edge detection, line recognition and rotation correction of the original image; Image correction and positioning module: medical record image correction and image region positioning, including mask processing and projection transformation of the medical record image preprocessed by the image preprocessing module; Digital information extraction module: based on Res-CRNN, the medical record image corrected and region positioned by the image correction and positioning module is used to extract digital information.

7. The paper medical record digitization information extraction device based on image edge detection and Res-CRNN, characterized in that, Comprise: Memory: for storing the computer program for implementing the paper medical record digital information extraction method based on image edge detection and Res-CRNN according to any one of claims 1-5; Processor: for executing the computer program to implement the paper medical record digital information extraction method based on image edge detection and Res-CRNN according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the paper medical record digital information extraction method based on image edge detection and Res-CRNN according to any one of claims 1-5.

Images