Medical bill identification method based on multi-classification text detection
By adopting multi-classified text detection method and Transformer structure in the medical bill recognition system, the problem that existing systems cannot accurately distinguish different types of text information is solved, efficient identification of complex bills is achieved, and recognition accuracy and robustness are improved. .
Patent Information
- Application Number
- CN202411410813.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-01-17
AI Technical Summary
When the existing medical bill identification system processes complex bills, it is impossible to accurately distinguish different types of text information, resulting in insufficient recognition effect.
Using a multi-classified text detection method, a multi-classified text detection model suitable for medical receipts is designed through the DBNet framework, and a Transformer structure is introduced into the CRNN feature extraction network to enhance feature extraction capabilities and long-distance dependency capture capabilities.
It improves the diversified and complex identification efficiency of medical bills, and can detect multiple different types of information on the bills at one time, enhancing the accuracy and robustness of identification.
Smart Images
Figure CN119323795A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing and recognition, and in particular to a method for medical bill recognition based on multi-classification text detection. Background Art
[0002] With the continuous advancement of deep learning technology, the field of bill recognition is undergoing a transformation from traditional recognition methods to deep learning-based technologies. These emerging methods are deeply rooted in text detection and recognition technology in natural scenes, and continue to evolve on this basis to build an end-to-end processing flow that integrates text detection and recognition. In view of the significant differences between the unique background templates and text characteristics of bill images and natural scene text images, as well as the high standards for recognition accuracy, the recognition research on specific bill types is increasing. At present, leading bill OCR projects such as Baidu OCR and Kuangshi OCR have demonstrated excellent performance in the recognition of regular bill images. However, these systems have high requirements for bill image quality and text neatness. When faced with complex bills or interference between texts, their recognition effect is still insufficient and needs further optimization and improvement.
[0003] In the current research field of medical bill recognition, in-depth research on complex bills is still insufficient. In the currently commonly used bill recognition system, most of them rely on binary classification detection algorithms in the text detection stage. Although such algorithms can effectively locate text areas, they cannot accurately distinguish different types of text information on bills, thus facing limitations when processing diversified and complex medical bills. Therefore, a method for medical bill recognition based on multi-classification text detection is provided. Summary of the invention
[0004] The purpose of the present invention is to provide a method for medical bill recognition based on multi-classification text detection, so as to solve the problem raised in the above background technology that in the current research field of medical bill recognition, in-depth research on complex bills is still insufficient. In the currently commonly used bill recognition system, most of them rely on binary classification detection algorithms in the text detection stage. Although this type of algorithm can effectively locate the text area, it cannot accurately distinguish different types of text information on the bill, and thus faces limitations when processing diversified and complex medical bills.
[0005] To achieve the above object, the present invention aims to provide a method for medical bill recognition based on multi-classification text detection, comprising the following steps:
[0006] S1. Collect medical bill image data by scanning physical medical bills;
[0007] S2, preprocessing the bill image data;
[0008] S3, marking key information in the bill image;
[0009] S4. Based on the DBNet framework, design and build a multi-classification text detection model for medical bills;
[0010] S5. Aiming at the interference factors in text recognition, the CRNN feature extraction network is enhanced and the Transformer structure is introduced.
[0011] As a further improvement of the technical solution, in S1, the specific steps of collecting medical bill image data by scanning the physical medical bill are:
[0012] S2.1. Use a scanner to capture images of medical bills;
[0013] S2.2, store image data and record basic information of each bill;
[0014] Among them, basic information includes date and number;
[0015] S2.3. Conduct a preliminary quality check on the collected medical bill image data and remove blurred and severely damaged images.
[0016] As a further improvement of the technical solution, in S2, the specific steps of preprocessing the bill image data are:
[0017] S3.1, adjust all images to a uniform size;
[0018] S3.2, converting the color image into a grayscale image to reduce the computational complexity;
[0019] S3.3. Use filters to remove noise from the image.
[0020] As a further improvement of the technical solution, in S3.2, the specific expression for converting the color image into a grayscale image is:
[0021] I gray =R·0.299+G·0.587+B·0.114;
[0022] Among them, I gray represents the converted grayscale image; R represents the pixel value of the red channel in the original image; G represents the pixel value of the green channel in the original image; B represents the pixel value of the blue channel in the original image.
[0023] As a further improvement of the technical solution, in S3, the specific steps of marking the key information in the bill image are:
[0024] S4.1. Define the key information types that need to be annotated;
[0025] Among them, key information types include invoice number, amount, date, and name of medical institution;
[0026] S4.2. Use the annotation tool to manually select the location of each information field;
[0027] S4.3, assign a label category to each selected area;
[0028] S4.4. Check the consistency and accuracy of the annotation results.
[0029] As a further improvement of the technical solution, in S4, based on the DBNet framework, the specific steps of designing and constructing a multi-classification text detection model suitable for medical bills are as follows:
[0030] S5.1. Design the input layer to match the size of the preprocessed image.
[0031] S5.2. Construct a feature extraction network based on DBNet;
[0032] S5.3. Design a classification layer to distinguish different categories of information fields;
[0033] S5.4. Set the cross entropy loss function to supervise the learning process;
[0034] S5.5, implement the training process;
[0035] The training process includes forward propagation, back propagation and parameter update.
[0036] As a further improvement of the technical solution, in S5.4, the specific expression of the cross entropy loss function is set as:
[0037]
[0038] Among them, L c represents the cross entropy loss; C represents the total number of categories; y i represents the true label; i represents the index variable; p i Represents the probability predicted by the model.
[0039] As a further improvement of the technical solution, in S5.5, the specific flow expression for implementing the training process is:
[0040] Forward propagation:
[0041] O = f(I; θ);
[0042] L = Loss(O,Y);
[0043] Among them, O represents the output of the model; I represents the input data; θ represents the model parameters; Y represents the true label; L represents the loss function; f represents the forward propagation function of the model;
[0044] Backward Propagation:
[0045]
[0046] in, Represents the gradient of the loss function L with respect to the model parameter θ; Represents the partial derivative of the loss function with respect to the model parameters; represents the lth layer parameter θ l The gradient of θ l Represents the lth layer parameter; O l+1 represents the output of the l+1th layer; Z l Represents the input of the lth layer; Represents the loss function L with respect to the output O of the l+1th layer l+1 The partial derivative of Indicates the output O of the l+1th layer l+1 About the l-th layer input Z l The partial derivative of Represents the input Z of the lth layer l About the lth layer parameter θ l The partial derivative of .
[0047] Parameter update:
[0048]
[0049] Among them, θ new represents the updated model parameters; η represents the learning rate.
[0050] As a further improvement of the technical solution, in S5, the specific steps of enhancing the CRNN feature extraction network and introducing the Transformer structure for interference factors in text recognition are as follows:
[0051] S6.1. Analyze interference factors to determine the characteristics that need to be enhanced;
[0052] S6.2, adding additional convolutional layers to CRNN;
[0053] S6.3, introduce the Transformer module to capture long-distance dependencies;
[0054] S6.4. Adjust the model architecture to ensure that the newly added components can be seamlessly integrated with the existing network;
[0055] S6.5. Conduct multiple experimental verifications to optimize model performance.
[0056] As a further improvement of the technical solution, in S6.3, the specific expression of introducing the Transformer module to capture the long-distance dependency is:
[0057]
[0058] Among them, Attention represents the Attention mechanism; Q represents the query matrix; K represents the key matrix; V represents the value matrix; d k represents the square root of the key vector dimension; T represents the transpose operation of the matrix; softmax represents the function that converts the attention score into a probability distribution.
[0059] Compared with the prior art, the present invention has the following beneficial effects:
[0060] 1. In this method of medical bill recognition based on multi-classification text detection, paper bills can be quickly and efficiently converted into digital images through scanning equipment, which is convenient for storage and management. Preprocessing ensures the consistency of the image, which is convenient for model training and reasoning. Labeling can accurately point out the location of key information that needs to be identified, which helps the training model to locate and identify this information more accurately.
[0061] 2. In this method of medical bill recognition based on multi-classification text detection, the advantages of text detection and recognition are combined through the DBNet framework, which can classify while detecting, improving the detection efficiency. It supports multi-classification tasks and can detect multiple different types of information on the bill at one time. The introduction of the Transformer structure can better capture the long-distance dependencies in the text, which is very important for understanding complex text structures. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 The figure is a flow chart of the overall method of the present invention. DETAILED DESCRIPTION
[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0064] Example:
[0065] See also Figure 1 As shown, this embodiment provides a method for medical bill recognition based on multi-classification text detection, comprising the following steps:
[0066] S1. Collect medical bill image data by scanning physical medical bills;
[0067] In this example, the specific steps for collecting medical bill image data by scanning physical medical bills are:
[0068] S2.1. Use a scanner to capture images of medical bills;
[0069] S2.2, store image data and record basic information of each bill;
[0070] Among them, basic information includes date and number;
[0071] S2.3. Conduct a preliminary quality check on the collected medical bill image data and remove blurred and severely damaged images.
[0072] Specifically, through strict image acquisition and quality control, the reliability and validity of the data are ensured, providing a solid foundation for subsequent processing, making the medical bill recognition method based on multi-classification text detection more reliable and efficient.
[0073] S2, preprocessing the bill image data;
[0074] In this example, the specific steps for preprocessing the bill image data are:
[0075] S3.1, adjust all images to a uniform size;
[0076] S3.2, converting the color image into a grayscale image to reduce the computational complexity;
[0077] In this example, the specific expression for converting a color image into a grayscale image is:
[0078] I gray =R·0.299+G·0.587+B·0.114;
[0079] Among them, I gray represents the converted grayscale image; R represents the pixel value of the red channel in the original image; G represents the pixel value of the green channel in the original image; B represents the pixel value of the blue channel in the original image.
[0080] S3.3. Use filters to remove noise from the image.
[0081] Specifically, filters are a common tool in image processing to improve image quality, enhance specific features, or remove unwanted components such as noise. Filters modify the pixel values of an image by applying specific mathematical operations on the image to achieve the desired effect. A filter is usually a collection of weight coefficients organized into a matrix called a kernel, mask, or convolution kernel. The working principle of a filter is to convolve the kernel with each pixel position in the image, that is, each coefficient in the kernel is multiplied by the pixel value at the corresponding position in the image, and then summed to obtain a new pixel value.
[0082] S3, marking key information in the bill image;
[0083] In this example, the specific steps for marking the key information in the bill image are:
[0084] S4.1. Define the key information types that need to be annotated;
[0085] Among them, key information types include invoice number, amount, date, and name of medical institution;
[0086] S4.2. Use the annotation tool to manually select the location of each information field;
[0087] Specifically, manual selection can ensure that the boundaries of the selected area are accurate and improve the accuracy of annotation. Manual selection can flexibly adjust the selection range according to actual conditions to adapt to different bill layouts. The results of manual selection are intuitive and easy to check and correct later.
[0088] S4.3, assign a label category to each selected area;
[0089] S4.4. Check the consistency and accuracy of the annotation results.
[0090] S4. Based on the DBNet framework, design and build a multi-classification text detection model for medical bills;
[0091] In this example, based on the DBNet framework, the specific steps for designing and building a multi-classification text detection model for medical bills are as follows:
[0092] S5.1. Design the input layer to match the size of the preprocessed image.
[0093] S5.2. Construct a feature extraction network based on DBNet;
[0094] Specifically, building a feature extraction network based on DBNet refers to designing and implementing a deep learning network that can extract useful features from input images, especially in text detection tasks. DBNet is a deep learning framework for text detection, which is characterized by its ability to detect text regions directly from images and can be combined with existing deep learning architectures.
[0095] The DBNet framework combines a variety of advanced feature extraction technologies to effectively extract key features for identification from images, such as edges and textures. DBNet takes into account the robustness of various image conditions in network design and can cope with various interference factors in images, such as noise and lighting changes. The DBNet framework can be adjusted and optimized according to actual needs to adapt to different application scenarios, such as different bill formats and content layouts.
[0096] S5.3. Design a classification layer to distinguish different categories of information fields;
[0097] Specifically, the classification layer enables the model to handle the identification of multiple information fields at the same time, improving the ability to identify multiple fields at one time and reducing the need to run the model multiple times. By designing a specialized classification layer, different categories of information fields can be distinguished more accurately, improving the accuracy of classification and ensuring that each field can be correctly identified. As new information fields are added, the classification layer can be easily expanded without having to significantly change the entire model structure, increasing the flexibility and adaptability of the model.
[0098] S5.4. Set the cross entropy loss function to supervise the learning process;
[0099] In this example, the specific expression of the cross entropy loss function is set as:
[0100]
[0101] Among them, L c represents the cross entropy loss; C represents the total number of categories; y i represents the true label; i represents the index variable; p i Represents the probability predicted by the model.
[0102] Specifically, the cross entropy loss function is a commonly used loss function that is widely used in classification tasks, especially multi-classification problems. Its main function is to measure the difference between the model prediction results and the actual labels, thereby guiding the model's learning process.
[0103] The cross entropy loss function can effectively measure the difference between the model's prediction results and the actual labels, and promote the optimization of the model. It minimizes the loss by maximizing the probability of the correct category, so that the model can focus more on the correct classification. The cross entropy loss function has good convergence characteristics, which can accelerate the model training process and improve training efficiency. It can converge quickly in the early stages of training, reducing training time and computing resources. Through cross entropy loss, the model can learn more general features on the training set, which helps to improve the generalization ability of the model, so that the model can perform well on unseen data.
[0104] S5.5, implement the training process;
[0105] The training process includes forward propagation, back propagation and parameter update.
[0106] In this example, the specific process expression for implementing the training process is:
[0107] Forward propagation:
[0108] O = f(I; θ);
[0109] L = Loss(O,Y);
[0110] Among them, O represents the output of the model; I represents the input data; θ represents the model parameters; Y represents the true label; L represents the loss function; f represents the forward propagation function of the model;
[0111] Backward Propagation:
[0112]
[0113] in, Represents the gradient of the loss function L with respect to the model parameter θ; Represents the partial derivative of the loss function with respect to the model parameters; represents the lth layer parameter θ l The gradient of θ l Represents the lth layer parameter; O l+1 represents the output of the l+1th layer; Z l Represents the input of the lth layer; Represents the loss function L with respect to the output O of the l+1th layer l+1 The partial derivative of Indicates the output O of the l+1th layer l+1 About the l-th layer input Z l The partial derivative of Represents the input Z of the lth layer l About the lth layer parameter θ l The partial derivative of .
[0114] Parameter update:
[0115]
[0116] Among them, θ new represents the updated model parameters; η represents the learning rate.
[0117] S5. Aiming at the interference factors in text recognition, the CRNN feature extraction network is enhanced and the Transformer structure is introduced.
[0118] In this example, the specific steps of enhancing the CRNN feature extraction network and introducing the Transformer structure to address the interference factors in text recognition are as follows:
[0119] S6.1. Analyze interference factors to determine the characteristics that need to be enhanced;
[0120] Specifically, by analyzing the interference factors, we can identify which features need to be strengthened, so as to improve the model in a targeted manner. Understanding and solving the interference factors can improve the robustness of the model under different conditions, so that it can perform well in a variety of environments. It provides a clear direction for subsequent steps and ensures that the selection of enhanced features is more reasonable and effective.
[0121] S6.2, adding additional convolutional layers to CRNN;
[0122] Specifically, additional convolutional layers can capture more complex features and improve the model's ability to understand image details. By adding convolutional layers, the model can better handle noise and background interference in the image and improve recognition accuracy. The number and configuration of convolutional layers can be adjusted according to actual needs to adapt to different recognition tasks.
[0123] S6.3, introduce the Transformer module to capture long-distance dependencies;
[0124] In this example, the specific expression of introducing the Transformer module to capture long-distance dependencies is:
[0125]
[0126] Among them, Attention represents the Attention mechanism; Q represents the query matrix; K represents the key matrix; V represents the value matrix; d k represents the square root of the key vector dimension; T represents the transpose operation of the matrix; softmax represents the function that converts the attention score into a probability distribution.
[0127] Specifically, the Transformer module can effectively capture long-distance dependencies in text, which is very important for understanding complex text structures. By capturing long-distance dependencies, the model can better understand the text content and improve recognition accuracy. The Transformer module supports parallel computing, which can accelerate the training process and improve computing efficiency.
[0128] S6.4. Adjust the model architecture to ensure that the newly added components can be seamlessly integrated with the existing network;
[0129] S6.5. Conduct multiple experimental verifications to optimize model performance.
[0130] Specifically, through repeated experiments, we can find the optimal model parameter configuration and improve the recognition performance of the model. A fully verified model is more reliable and can show better performance in practical applications. The process of experimental verification is also a process of continuous learning and improvement, which helps to continuously improve the accuracy and robustness of the model.
[0131] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only preferred examples of the present invention, and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.
Claims
1. A method for medical bill recognition based on multi-classification text detection, characterized in that: The following steps are involved: S1. Collect medical bill image data by scanning physical medical bills; S2, preprocessing the bill image data; S3, marking key information in the bill image; S4. Based on the DBNet framework, design and build a multi-classification text detection model for medical bills; S5. Aiming at the interference factors in text recognition, the CRNN feature extraction network is enhanced and the Transformer structure is introduced.
2. The method for medical bill recognition based on multi-classification text detection according to claim 1 is characterized in that: In S1, the specific steps of collecting medical bill image data by scanning the physical medical bill are: S2.
1. Use a scanner to capture images of medical bills; S2.2, store image data and record basic information of each bill; Among them, basic information includes date and number; S2.
3. Conduct a preliminary quality check on the collected medical bill image data and remove blurred and severely damaged images.
3. The method for medical bill recognition based on multi-classification text detection according to claim 1 is characterized in that: In S2, the specific steps of preprocessing the bill image data are: S3.1, adjust all images to a uniform size; S3.2, converting the color image into a grayscale image to reduce the computational complexity; S3.
3. Use filters to remove noise from the image.
4. The method for medical bill recognition based on multi-classification text detection according to claim 3 is characterized in that: In S3.2, the specific expression for converting a color image into a grayscale image is: I gray =R·0.299+G·0.587+B·0.114; Among them, I gray represents the converted grayscale image; R represents the pixel value of the red channel in the original image; G represents the pixel value of the green channel in the original image; B represents the pixel value of the blue channel in the original image.
5. The method for medical bill recognition based on multi-classification text detection according to claim 1 is characterized in that: In S3, the specific steps of marking the key information in the bill image are: S4.
1. Define the key information types that need to be annotated; Among them, key information types include invoice number, amount, date, and name of medical institution; S4.
2. Use the annotation tool to manually select the location of each information field; S4.3, assign a label category to each selected area; S4.
4. Check the consistency and accuracy of the annotation results.
6. The method for medical bill recognition based on multi-classification text detection according to claim 1, characterized in that: In S4, based on the DBNet framework, the specific steps of designing and building a multi-classification text detection model suitable for medical bills are as follows: S5.
1. Design the input layer to match the size of the preprocessed image. S5.
2. Construct a feature extraction network based on DBNet; S5.
3. Design a classification layer to distinguish different categories of information fields; S5.
4. Set the cross entropy loss function to supervise the learning process; S5.5, implement the training process; The training process includes forward propagation, back propagation and parameter update.
7. The method for medical bill recognition based on multi-classification text detection according to claim 6 is characterized in that: In S5.4, the specific expression of the cross entropy loss function is set as: Among them, L c represents the cross entropy loss; C represents the total number of categories; y i represents the true label; i represents the index variable; p i Represents the probability predicted by the model.
8. The method for medical bill recognition based on multi-classification text detection according to claim 6 is characterized in that: In S5.5, the specific flow expression for implementing the training process is: Forward propagation: O = f(I; θ); L = Loss(O,Y); Among them, O represents the output of the model; I represents the input data; θ represents the model parameters; Y represents the true label; L represents the loss function; f represents the forward propagation function of the model; Backward Propagation: in, Represents the gradient of the loss function L with respect to the model parameter θ; Represents the partial derivative of the loss function with respect to the model parameters; represents the lth layer parameter θ l The gradient of θ l Represents the lth layer parameter; O l+1 represents the output of the l+1th layer; Z l Represents the input of the lth layer; Represents the loss function L with respect to the output O of the l+1th layer l+1 The partial derivative of Indicates the output O of the l+1th layer l+1 About the l-th layer input Z l The partial derivative of Represents the input Z of the lth layer l About the lth layer parameter θ l The partial derivative of . Parameter update: Among them, θ new represents the updated model parameters; η represents the learning rate.
9. The method for medical bill recognition based on multi-classification text detection according to claim 1, characterized in that: In S5, the specific steps of enhancing the CRNN feature extraction network and introducing the Transformer structure to address the interference factors in text recognition are as follows: S6.
1. Analyze interference factors to determine the characteristics that need to be enhanced; S6.2, adding additional convolutional layers to CRNN; S6.3, introduce the Transformer module to capture long-distance dependencies; S6.
4. Adjust the model architecture to ensure that the newly added components can be seamlessly integrated with the existing network; S6.
5. Conduct multiple experimental verifications to optimize model performance.
10. The method for medical bill recognition based on multi-classification text detection according to claim 1, characterized in that: In S6.3, the specific expression of introducing the Transformer module to capture long-distance dependencies is: Among them, Attention represents the Attention mechanism; Q represents the query matrix; K represents the key matrix; V represents the value matrix; d k represents the square root of the key vector dimension; T represents the transpose operation of the matrix; softmax represents the function that converts the attention score into a probability distribution.