A medical bill image-oriented text detection and recognition method

Abstract

The present application relates to the technical field of text detection and recognition of images, in particular to a text detection and recognition method for medical bill images. The method comprises the following steps: S1, performing feature enhancement on the medical bill image through a feature enhancement module, and reconstructing the table structure of the medical bill image; S2, after reconstructing the table structure, further performing feature fusion on the medical bill image to form a pre-detection table image; S3, integrating the pre-detection table image with a visual language pre-training model MDETR to enhance feature expression; S4, using a connection time sequence classification (CTC) method improved through an Attention mechanism to recognize a text region from the pre-detection table image; and S5, further performing deep analysis on the detected text region through a fusion model of an SVTR network and a PP-HGNet. The present application can recognize a text region in a complex medical bill image through feature enhancement, table structure reconstruction, feature fusion and integration with a visual language pre-training model MDETR.

Description

Technical Field

[0001] This invention relates to the field of text detection and recognition technology for images, and more specifically, to a method for text detection and recognition of medical invoice images. Background Technology

[0002] The primary sources of image data for medical receipts are scanned copies from actual scenes and photos taken by mobile devices such as smartphones. Given the generally limited resolution of mobile device cameras, the captured images of medical receipts often contain various interfering factors, such as stains, scratches, handwritten notes, and stamps. These interfering factors are not only diverse in type but may also obscure or cover important textual information, posing a serious challenge to the accurate recognition of text detection algorithms, leading to decreased algorithm accuracy and increased error rates. Furthermore, some images undergo compression during network transmission, which degrades image quality and further affects subsequent image processing steps and the accuracy of text recognition. As crucial medical records, medical receipts have a compact and diverse text layout, with text content occupying over 80% of the page, encompassing a wide range of information categories and integrating multiple languages ​​and symbol systems. Specifically, Chinese is used to record patient details and medical service items; English may appear in sections such as medical institution names and doctor information; Greek letters are common in medical terminology and symbols; and mathematical formulas may appear in examination results or professional medical reports. This diversity and complexity of text formats increases the difficulty of recognizing and processing medical receipts, requiring targeted analysis and recognition strategies for various types of text. Furthermore, in specific application scenarios, such as instant settlement and online insurance verification, the image-text recognition of medical invoices needs to be highly timely. In summary, this paper provides a method for text detection and recognition of medical invoice images. Summary of the Invention

[0003] The purpose of this invention is to provide a text detection and recognition method for medical invoice images, in order to solve the problems of complex medical invoice images, poor image quality and diverse text formats mentioned in the background art.

[0004] To achieve the above objectives, the present invention aims to provide a text detection and recognition method for medical invoice images, comprising the following steps:

[0005] S1. Enhance the features of the medical invoice image through the feature enhancement module and reconstruct the table structure of the medical invoice image;

[0006] S2. After reconstructing the table structure, further feature fusion of the medical invoice image is performed to form a pre-detection table image;

[0007] S3. Integrate the pre-detected table images with the visual language pre-trained model MDETR to enhance feature representation;

[0008] S4. Use the connection-time classification (CTC) method, which is improved by the attention mechanism, to identify text regions from the pre-detected table images;

[0009] S5. The detected text regions are further analyzed in depth using a fusion model of SVTR network and PP-HGNet.

[0010] As a further improvement to this technical solution, in step S1, feature enhancement is performed on the medical invoice image through a feature enhancement module, including the following steps:

[0011] S1.1 Convert the color medical bill image to a grayscale medical bill image and adjust the histogram of the medical bill image;

[0012] S1.2. The medical bill image is smoothed by Gaussian filtering algorithm to remove high-frequency noise. The region containing the table structure is initially extracted from the processed grayscale medical bill image, and the extracted region is divided into multiple smaller region blocks.

[0013] S1.3 Enhance the contrast of medical bill images by histogram stretching and use the Laplacian operator to sharpen image edges;

[0014] S1.4 Use the dilation operation to enhance horizontal line segments in the image;

[0015] S1.5. Dynamically adjust the threshold according to the local characteristics of the medical invoice image and perform adaptive binarization processing;

[0016] S1.6 Apply Hough transform to detect and reconstruct horizontal line segments within each region block, perform K-means clustering analysis on the detected line spacing, and calculate the mean line spacing.

[0017] S1.7. Based on the average row spacing, set a threshold u, use dilation operation to further extract horizontal line segments, set a threshold h based on the average row spacing, detect vertical line segments, merge the detected horizontal and vertical line segments, and reconstruct the table structure of the medical invoice image.

[0018] As a further improvement to this technical solution, the Gaussian filtering algorithm in S1.2 is specifically as follows:

[0019]

[0020] Where p represents the offset of a pixel on the x-axis; r represents the offset of a pixel on the y-axis; σ represents the standard deviation of the Gaussian distribution; and G(p, r, σ) represents the weight of each pixel and its neighborhood in the medical invoice image when performing Gaussian filtering.

[0021] As a further improvement to this technical solution, in S1.3, the Laplace operator is specifically as follows:

[0022]

[0023] Where f(x,y) represents the gray value of the medical bill image at coordinates (x,y); x represents the x-axis in two-dimensional space; y represents the y-axis in two-dimensional space; and f represents a function in two-dimensional space. This represents the partial derivative.

[0024] As a further improvement to this technical solution, in step S1.5, the threshold is dynamically adjusted according to the local characteristics of the medical invoice image to perform adaptive binarization processing, including the following steps:

[0025] S1.51. Select a window size of 5×5. For each pixel in the medical bill image, use the selected window to calculate the local average brightness around that pixel.

[0026] S1.52. Calculate the offset value, and add the offset value to the local average brightness as the threshold b for that local area;

[0027] S1.53. Compare the gray value of each pixel with the calculated threshold of the local area. If it is greater than the threshold b, set it to white; otherwise, set it to black.

[0028] S1.54. Repeat steps S1.51 to S1.53 for the entire image until all pixels have been processed.

[0029] As a further improvement to this technical solution, in S1.6, the application of Hough transform to detect straight lines in the image includes the following steps:

[0030] S1.61. Use the standard Hough transform to detect straight lines;

[0031] S1.62. Traverse each edge point in the medical bill image and accumulate votes for the line representing that edge point in the parameter space;

[0032] S1.63. For each edge point, calculate all line parameter combinations and increment the count in the corresponding parameter space;

[0033] S1.64. Find local maxima in the parameter space;

[0034] S1.65. Based on the detected local maxima, draw a straight line in the original medical bill image.

[0035] As a further improvement to this technical solution, in step S2, after reconstructing the table structure, feature fusion is further performed, including the following steps:

[0036] S2.1. Further process the image using a custom kernel mask;

[0037] S2.2 Apply the dilation operation to the pre-extracted region block again to extract the horizontal line segments;

[0038] S2.3 Apply the standard Hough transform again within the pre-extracted region block to reconstruct the horizontal line segment;

[0039] S2.4. Perform cluster analysis on the reconstructed horizontal line segments again to calculate the mean distance between lines;

[0040] S2.5. Based on the average row spacing, set a threshold to extract vertical line segments and further improve the table structure;

[0041] S2.6. Perform region segmentation again and determine the region threshold m;

[0042] S2.7. Fuse the features of each segmented region and pre-extract the contours to form a pre-detection table image.

[0043] As a further improvement to this technical solution, in step S3, the pre-detected table image is integrated with the visual language pre-trained model MDETR to enhance feature representation, including the following steps:

[0044] S3.1. Use the generated pre-detection table image as input and perform standardization processing on the pre-detection table image;

[0045] S3.2 Input the preprocessed pre-detection table image into the MDETR model;

[0046] S3.3. Interact with image features and text features, and encode the image features;

[0047] S3.4 Decode the encoded and interacted features and output the detection results;

[0048] S3.5. Post-process the target detection results output by the MDETR model using the non-maximum suppression (NMS) method.

[0049] S3.6 Use the detection results provided by the model MDETR to further confirm and refine the table boundaries and cell positions.

[0050] As a further improvement to this technical solution, in step S4, the connection-time classification (CTC) method improved by the Attention mechanism is used to identify text regions from the pre-detected table image, including the following steps:

[0051] S4.1 Input the pre-detection table image;

[0052] S4.2 Use a convolutional neural network (CNN) to extract features from the image;

[0053] S4.3 Input the feature map into the recurrent neural network (RNN) and add an attention mechanism to the RNN.

[0054] S4.4 Generate the output sequence and calculate the CTC loss between the output sequence and the true label;

[0055] S4.5. Use a greedy strategy to decode the output sequence, determine the text region in the image based on the decoding result, and identify it.

[0056] As a further improvement to this technical solution, in step S5, the detected text region is further analyzed in depth using a fusion model of SVTR network and PP-HGNet, including the following steps:

[0057] S5.1 Input the detected text region images and preprocess each text region image;

[0058] S5.2. The feature map generated by the SVTR network is concatenated with the feature map generated by the PP-HGNet to form a new feature map;

[0059] S5.3. Use an attention mechanism to further process the concatenated feature map to generate the final feature representation;

[0060] S5.4. Use the CTC decoder to convert the final feature representation into a text sequence, and perform post-processing on the decoded text sequence.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] 1. This text detection and recognition method for medical invoice images, through feature enhancement, table structure reconstruction, feature fusion, and integration with the visual-language pre-trained model MDETR, can accurately identify text regions in complex medical invoice images. Histogram adjustment, Gaussian filtering, and Laplacian sharpening techniques in the feature enhancement step help improve image quality, making the text clearer and more legible. Furthermore, adaptive binarization and Hough transform effectively detect table structures, thereby improving the accuracy of text region detection. The improved CTC method combined with an attention mechanism further enhances the accuracy of text recognition, especially in handling blurred or handwritten text.

[0063] 2. In this text detection and recognition method for medical invoice images, noise reduction and smoothing operations in the preprocessing stage reduce unnecessary computational load, while feature fusion and the application of a multimodal model enhance the system's robustness. Adaptive binarization using a 5x5 window allows for threshold adjustment based on the local environment, thus better adapting to images under different lighting conditions. Furthermore, deep analysis using a fusion model of SVTR network and PP-HGNet enables more effective handling of diverse medical invoice formats and maintains a high recognition rate despite varying handwriting styles. These steps not only improve processing speed but also ensure stable performance when faced with diverse inputs. Attached Figure Description

[0064] Figure 1 This is a flowchart of the overall method of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0066] Example: Please refer to Figure 1 As shown, this embodiment provides a text detection and recognition method for medical invoice images, including the following steps:

[0067] S1. Enhance the features of the medical invoice image through the feature enhancement module and reconstruct the table structure of the medical invoice image;

[0068] The Feature Enhancement Module (EFEM) is a U-shaped, cascaded module. Equipped with four input and four output channels, it is specifically designed for enhancing and fusing multiple feature maps. EFEM is a core component in text detection solutions, excelling particularly in multi-scale feature enhancement. To achieve top-tier performance, EFEM is meticulously designed with three stages: first, up-scale enhancement broadens the feature field of view, enriching details; second, the Pyramid Squeezing Attention Module (PSA) uses an attention mechanism to highlight key features and filter out noise; and finally, down-scale enhancement refines and integrates features, strengthening discriminative capabilities.

[0069] In this embodiment, feature enhancement is performed on the medical invoice image using a feature enhancement module, including the following steps:

[0070] S1.1 Convert the color medical bill image to a grayscale medical bill image to reduce computational complexity, and adjust the histogram of the medical bill image to enhance contrast and make the text in the image clearer.

[0071] S1.2. The medical bill image is smoothed by Gaussian filtering algorithm to remove high-frequency noise. The region containing the table structure is initially extracted from the processed grayscale medical bill image, and the extracted region is divided into multiple smaller region blocks so that each part can be processed more finely.

[0072] Gaussian filtering can improve the overall visual effect of an image, making it appear clearer and more natural. This is very helpful for subsequent image processing steps (such as edge detection and feature extraction). Gaussian filtering is suitable for various image processing scenarios, providing good smoothing effects for both natural and document images. It is particularly effective in improving the readability and recognition rate of text-heavy medical invoices. Images processed with Gaussian filtering can better perform subsequent processing steps such as edge detection and feature extraction because important features in the image (such as text and table lines) become more prominent, while interfering factors are reduced. The specific Gaussian filtering algorithm is as follows:

[0073]

[0074] Where p represents the offset of a pixel on the x-axis; r represents the offset of a pixel on the y-axis; σ represents the standard deviation of the Gaussian distribution; and G(p, r, σ) represents the weight of each pixel and its neighborhood in the medical bill image when performing Gaussian filtering.

[0075] S1.3 Enhance the contrast of medical bill images by histogram stretching and use the Laplacian operator to sharpen image edges;

[0076] Among them, the histogram stretching method increases the contrast of the image by remapping the gray levels of the image, expanding the brightest and darkest pixel values ​​of the image to the maximum and minimum gray level ranges, respectively.

[0077] Furthermore, the Laplacian operator is a commonly used edge detection operator that can highlight edges and details in an image. By applying the Laplacian operator, the edge information of objects in an image can be enhanced, making the edges more prominent, which is beneficial for subsequent feature extraction and recognition. Sharpening image edges can increase image contrast, making the image appear clearer, which is especially important for reading and recognizing details in images, particularly in medical invoice images, where sharpening can make text and table lines more clearly distinguishable. For slight blurring that may occur during the shooting process, Laplacian sharpening can play a certain compensatory role. By sharpening the edges, the impact of blur can be reduced, improving image quality. The Laplacian operator is specifically as follows:

[0078]

[0079] Where f(x,y) represents the gray value of the medical bill image at coordinates (x,y); x represents the x-axis in two-dimensional space; y represents the y-axis in two-dimensional space; and f represents a function in two-dimensional space. Represents partial derivatives;

[0080] S1.4 Use dilation to enhance horizontal line segments in the image. This step can help remove noise and strengthen the line structure inside the table.

[0081] The dilation operation is used to expand the white areas (i.e., the foreground) in an image, making lines in the image thicker and thus enhancing horizontal line segments. The basic principle of dilation is to scan the image using a structuring element and compare the structuring element with each pixel in the image to determine the new pixel value;

[0082] S1.5. Dynamically adjust the threshold according to the local characteristics of the medical invoice image and perform adaptive binarization processing;

[0083] Furthermore, the threshold is dynamically adjusted based on the local characteristics of the medical invoice image, and adaptive binarization processing is performed, including the following steps:

[0084] S1.51 Select a window size of 5×5. This window will be used to scan every pixel in the image. For each pixel in the medical bill image, use the selected window to calculate the local average brightness around that pixel.

[0085] If I(x, y) represents the gray value of the image at coordinates (x, y), and W represents the window size (5x5), then the local average brightness M(x, y) is expressed as:

[0086]

[0087] Where w represents the side length of the window; i represents the offset on the x-axis; and i represents the offset on the y-axis.

[0088] S1.52. Calculate the offset value, which is used to prevent foreground text that is too close to the background brightness from being misjudged as background. The local average brightness is added to the offset value as the threshold b of the local area, where the offset value is a constant value.

[0089] S1.53. Compare the grayscale value of each pixel with the calculated threshold of the local area. If it is greater than the threshold b, set it to white (representing the foreground, usually text); otherwise set it to black (representing the background).

[0090] S1.54. Repeat steps S1.51 to S1.53 for the entire image until all pixels have been processed.

[0091] S1.6. Apply Hough transform within each region block to detect and reconstruct horizontal line segments to enhance the recognition of table structure. Perform K-means clustering analysis on the detected row spacing and calculate the mean row spacing to determine the average row height and provide a reference for table structure.

[0092] Furthermore, the Hough transform can still detect straight lines even in noisy images. Even if there are many noisy points in the image, as long as one set of these points approximately satisfies the straight line condition, the Hough transform can detect the straight line. This is crucial for processing real-world medical document images, as these images may be affected by uneven lighting, shadows, stains, and other factors. The Hough transform does not rely on continuous, complete straight lines, but rather on a point-cumulative voting mechanism. Even if the straight line is interrupted or partially missing, as long as enough points fall on the same straight line, the Hough transform can still detect its existence. This is particularly useful for detecting table lines in medical documents, as table lines may not be completely continuous or may be interrupted by text. By setting appropriate parameters, the Hough transform can accurately detect the position and direction of the straight line, which is extremely useful in situations requiring high-precision detection (such as fine table lines in medical documents). Applying the Hough transform to detect straight lines in an image includes the following steps:

[0093] S1.61. Use the standard Hough transform to detect straight lines;

[0094] S1.62. Traverse each edge point in the medical bill image and accumulate votes for the line representing that edge point in the parameter space;

[0095] S1.63. For each edge point, calculate all line parameter combinations and increment the count in the corresponding parameter space, where the parameter space is a two-dimensional matrix representing the multidimensional space used to represent line parameters in the Hough transform.

[0096] Furthermore, the line parameter ρ = acos(θ) + bsin(θ); (a, b) represents the coordinates of the edge point; θ represents the angle between the line and the horizontal line;

[0097] S1.64. Find local maxima in the parameter space. These points correspond to the most likely straight lines in the image. Local maxima are the points with the highest counts in a certain neighborhood.

[0098] S1.65. Based on the detected local maxima, draw a straight line in the original medical bill image;

[0099] Furthermore, the average line spacing is:

[0100]

[0101] In the formula, μe Indicates the average line spacing; C r Represents a cluster, i.e., a group of similar row spacing values; i represents the index variable; d e Indicates the specific line spacing value;

[0102] S1.7. Based on the average row spacing, set a threshold u, use dilation operation to further extract horizontal line segments, set a threshold h based on the average row spacing, detect vertical line segments to completely reconstruct the table structure, merge the detected horizontal and vertical line segments, and reconstruct the table structure of the medical invoice image.

[0103] S2. After reconstructing the table structure, further feature fusion of the medical invoice image is performed to form a pre-detection table image;

[0104] In this embodiment, after reconstructing the table structure, feature fusion is further performed, including the following steps:

[0105] S2.1. Further process the image using a custom kernel mask to remove noise or enhance specific features;

[0106] S2.2 Apply the dilation operation to the pre-extracted region block again to extract the horizontal line segments;

[0107] S2.3 Apply the standard Hough transform again within the pre-extracted region block to reconstruct the horizontal line segment;

[0108] S2.4. Perform cluster analysis on the reconstructed horizontal line segments again to calculate the mean distance between lines;

[0109] S2.5. Based on the average row spacing, set a threshold to extract vertical line segments and further improve the table structure;

[0110] S2.6. Perform region segmentation again to ensure that the features within each block are more consistent, and determine the threshold m so that further processing can be performed within each segmented region;

[0111] S2.7. Fuse the features of each segmented region and pre-extract the contours to enhance the recognition of the table structure and form a pre-detection table image;

[0112] In semantic segmentation, context is crucial, as it broadens the understanding of objects and scenes. Context and scale complement each other, providing key information for object recognition. To efficiently utilize this information, a multi-scale fusion strategy is employed.

[0113] S3. Integrate the pre-detected table images with the visual language pre-trained model MDETR to enhance feature representation;

[0114] In this embodiment, the visual language pre-trained model has undergone in-depth pre-training on a wide range of visual language understanding tasks, thereby acquiring rich visual and semantic expressive capabilities. When this pre-trained model is combined with the segmentation module, it can fully demonstrate its powerful representation capabilities, further improving the robustness and detection accuracy of the text detection model. Experimental results show that the model exhibits excellent text detection performance when processing medical invoice images with low resolution and high noise interference, providing a solution to overcome the challenges of text detection in medical invoice images. MDETR is a multimodal model that can process image and text information simultaneously. By integrating MDETR, its joint understanding of images and text can be utilized to enhance the understanding and localization of complex text information in images. This multimodal information fusion helps improve the accuracy and robustness of text detection. Through the cross-modal attention mechanism in MDETR, the interaction between image features and text features can be realized. This mechanism allows the model to focus on key parts of the image and ignore irrelevant background information, thereby generating more accurate feature representations, which is very beneficial for improving the detection accuracy of text regions. Integrating the pre-detected table image with the visual language pre-trained model MDETR to enhance feature representation includes the following steps:

[0115] S3.1. Use the generated pre-detection table images as input. These images have been processed to enhance the table structure. Standardize the pre-detection table images, such as scaling, cropping, and pixel normalization, to make them conform to the input requirements of the MDETR model.

[0116] S3.2 Input the pre-processed pre-detection table image into the MDETR model. MDETR is a multimodal model that can process image and text data simultaneously and is used for tasks such as object detection.

[0117] S3.3. Through the cross-modal attention mechanism in MDETR, image features and text features interact and the image features are encoded.

[0118] S3.4 Decode the encoded and interacted features and output the detection results;

[0119] S3.5. Post-process the target detection results output by the MDETR model using the non-maximum suppression (NMS) method.

[0120] The basic idea of ​​the NMS method is to compare the confidence scores of overlapping detection boxes and retain the detection box with the highest score, while suppressing (i.e. excluding) detection boxes with lower scores and higher degree of overlap.

[0121] S3.6 Use the detection results provided by the model MDETR to further confirm and refine the table boundaries and cell positions.

[0122] S4. Use the connection-time classification (CTC) method, which is improved by the attention mechanism, to identify text regions from the pre-detected table images;

[0123] In this embodiment, an attention mechanism is employed to optimize the training process of CTC (Connection Temporal Classification). The attention mechanism monitors and guides CTC, aiming to improve the accuracy and computational efficiency of text recognition. The attention mechanism excels at capturing the interdependencies between characters and can accurately locate characters, thereby extracting more effective features. Integrating CTC with the attention mechanism makes the model perform better in text learning, alignment, and feature representation. It can flexibly focus on key text regions in the image while effectively filtering out noise or interference factors irrelevant to the recognition task. This method cleverly combines the high speed of the CTC decoder with the attention mechanism. The high precision of the mechanism ensures that the model achieves an ideal balance between efficiency and performance. Optimizing the CTC (Connectivity-Temporal Classification) training strategy through an attention module enables the model to automatically focus on key text regions in the image, while effectively filtering out noise and interference factors irrelevant to the recognition task. This guided attention mechanism helps the model achieve more accurate text localization and recognition, thereby improving overall recognition accuracy. By exploring effective data augmentation techniques, transfer learning, or model compression, it is expected to achieve efficient text recognition under small sample conditions. The connection-temporal classification (CTC) method, improved with an attention mechanism, identifies text regions from pre-detected table images, including the following steps:

[0124] S4.1 Input pre-detection table images. These images have undergone feature enhancement, table structure reconstruction, and feature fusion processing to form relatively clear table structure images.

[0125] S4.2. Use Convolutional Neural Networks (CNNs) to extract features from images. CNNs can effectively capture local features in images and generate feature maps. The feature maps generated by CNNs contain important information in the image, and these feature maps will be used as input to the CTC model.

[0126] S4.3 Input the feature map into the recurrent neural network (RNN) and add an attention mechanism on the basis of the RNN so that the model can focus on the important parts of the feature map and ignore irrelevant information. The attention mechanism enhances the model's attention to key features by calculating the importance weight of each position in the feature map.

[0127] S4.4 Generate output sequences, which represent the predicted probability distribution at each time step, and calculate the CTC loss between the output sequences and the true labels;

[0128] Specifically, the CTC loss between the output sequence and the true label is calculated as follows:

[0129] L = -logP(Q|Z);

[0130] In the formula, Q represents the true label sequence, that is, the text sequence that actually exists in the pre-detection table image; Z represents the output sequence of the model; P(Q|Z) represents the probability of writing the true label sequence given the output sequence; L represents the loss between the output sequence and the true label.

[0131] S4.5. Use a greedy strategy to decode the output sequence, determine the text region in the image based on the decoding result, and identify it.

[0132] The basic idea of ​​the greedy strategy is to select the most likely output at each time step as the final output. This method ignores the global optimality of the entire sequence and only focuses on the best choice at the current time step.

[0133] S5. The detected text regions are further analyzed in depth using a fusion model of SVTR network and PP-HGNet;

[0134] In this embodiment, the SVTR network focuses on sparse visual marker recognition, effectively processing characters in text regions and accurately recognizing text even in complex backgrounds. PP-HGNet, on the other hand, employs a hierarchical structure to capture features at different scales, which is highly advantageous for processing text of varying sizes. The combination of these two models captures more detailed information at multiple scales, thereby improving the accuracy of text recognition. SVTR reduces computational complexity through a sparse sampling strategy, while PP-HGNet enhances its ability to process images at different resolutions through a pyramid structure. Fusing these two models better addresses medical invoice text of different formats, sizes, and handwriting styles, enhancing the model's adaptability and recognition capabilities for various complex text markers. The detected text regions are further analyzed using the fusion model of the SVTR and PP-HGNet, including the following steps:

[0135] S5.1 Input the detected text region images and preprocess each text region image;

[0136] S5.2. The feature map generated by the SVTR network is concatenated with the feature map generated by the PP-HGNet to form a new feature map;

[0137] S5.3. Use an attention mechanism to further process the concatenated feature map to generate the final feature representation;

[0138] S5.4 Use the CTC decoder to convert the final feature representation into a text sequence, and perform post-processing on the decoded text sequence, such as removing duplicate characters and correcting the format.

[0139] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for text detection and recognition of medical invoice images, characterized in that: Includes the following steps: S1. Enhance the features of the medical invoice image through the feature enhancement module and reconstruct the table structure of the medical invoice image; S2. After reconstructing the table structure, further feature fusion of the medical invoice image is performed to form a pre-detection table image; S3. Integrate the pre-detected table images with the visual language pre-trained model MDETR to enhance feature representation; S4. Use the connection-time classification (CTC) method, which is improved by the attention mechanism, to identify text regions from the pre-detected table images; S5. The detected text regions are further analyzed in depth using a fusion model of SVTR network and PP-HGNet; In step S1, the medical invoice image is enhanced using a feature enhancement module, including the following steps: S1.1 Convert the color medical bill image to a grayscale medical bill image and adjust the histogram of the medical bill image; S1.

2. The medical bill image is smoothed by Gaussian filtering algorithm to remove high-frequency noise. The region containing the table structure is initially extracted from the processed grayscale medical bill image, and the extracted region is divided into multiple small region blocks. S1.3 Enhance the contrast of medical bill images by histogram stretching and use the Laplacian operator to sharpen image edges; S1.4 Use the dilation operation to enhance horizontal line segments in the image; S1.

5. Dynamically adjust the threshold according to the local characteristics of the medical invoice image and perform adaptive binarization processing; S1.6 Apply Hough transform to detect and reconstruct horizontal line segments within each region block, perform K-means clustering analysis on the detected line spacing, and calculate the mean line spacing. S1.

7. Based on the average row spacing, set a threshold u, use dilation operation to further extract horizontal line segments, set a threshold h based on the average row spacing, detect vertical line segments, merge the detected horizontal and vertical line segments, and reconstruct the table structure of the medical bill image. In step S2, after reconstructing the table structure, feature fusion is further performed, including the following steps: S2.

1. Further process the image using a custom kernel mask; S2.2 Apply the dilation operation to the pre-extracted region block again to extract the horizontal line segments; S2.3 Apply the standard Hough transform again within the pre-extracted region block to reconstruct the horizontal line segment; S2.

4. Perform cluster analysis on the reconstructed horizontal line segments again to calculate the mean distance between lines; S2.

5. Based on the average row spacing, set a threshold to extract vertical line segments and further improve the table structure; S2.

6. Perform region segmentation again and determine the region threshold m; S2.

7. Fuse the features of each segmented region and pre-extract the contours to form a pre-detection table image; In step S3, the pre-detected table image is integrated with the visual language pre-trained model MDETR to enhance feature representation, including the following steps: S3.

1. Use the generated pre-detection table image as input and perform standardization processing on the pre-detection table image; S3.2 Input the preprocessed pre-detection table image into the MDETR model; S3.

3. Interact with image features and text features, and encode the image features; S3.4 Decode the encoded and interacted features and output the detection results; S3.

5. Post-process the target detection results output by the MDETR model using the non-maximum suppression (NMS) method. S3.6 Use the detection results provided by the model MDETR to further confirm and refine the table boundaries and cell positions.

2. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In S1.2, the Gaussian filtering algorithm is specifically as follows: ； in, Indicates the pixel point at Offset on the axis; Indicates the pixel point at Offset on the axis; This represents the standard deviation of a Gaussian distribution. This represents the weight of each pixel and its neighborhood in the medical invoice image during Gaussian filtering.

3. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In S1.3, the Laplace operator is specifically as follows: ; in, Indicates the medical bill image in coordinates The grayscale value at that location; Representing two-dimensional space axis; Representing two-dimensional space axis; Represents a function in two-dimensional space; This represents the partial derivative.

4. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In step S1.5, the threshold is dynamically adjusted based on the local characteristics of the medical invoice image to perform adaptive binarization processing, including the following steps: S1.

51. Select a window size of 5×5. For each pixel in the medical bill image, use the selected window to calculate the local average brightness around that pixel. S1.

52. Calculate the offset value, and add the offset value to the local average brightness as the threshold b of the local area; S1.

53. Compare the gray value of each pixel with the calculated threshold of the local area. If it is greater than the threshold b, set it to white; otherwise, set it to black. S1.

54. Repeat steps S1.51 to S1.53 for the entire image until all pixels have been processed.

5. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In step S1.6, the application of Hough transform to detect straight lines in an image includes the following steps: S1.

61. Use the standard Hough transform to detect straight lines; S1.

62. Traverse each edge point in the medical bill image and accumulate votes for the line representing that edge point in the parameter space; S1.

63. For each edge point, calculate all line parameter combinations and increment the count in the corresponding parameter space; S1.

64. Find local maxima in the parameter space; S1.

65. Draw a straight line in the original medical bill image based on the detected local maxima.

6. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In step S4, the text region is identified from the pre-detected table image using the connection-time classification (CTC) method improved by the attention mechanism, including the following steps: S4.1 Input the pre-detection table image; S4.2 Use a convolutional neural network (CNN) to extract features from the image; S4.3 Input the feature map into the recurrent neural network (RNN) and add an attention mechanism to the RNN. S4.4 Generate the output sequence and calculate the CTC loss between the output sequence and the true label; S4.

5. Use a greedy strategy to decode the output sequence, determine the text region in the image based on the decoding result, and identify it.

7. The text detection and recognition method for medical invoice images according to claim 1, characterized in that: In step S5, the detected text regions are further analyzed using a fusion model of the SVTR network and PP-HGNet, including the following steps: S5.1 Input the detected text region images and preprocess each text region image; S5.

2. The feature map generated by the SVTR network is concatenated with the feature map generated by the PP-HGNet to form a new feature map; S5.

3. Use an attention mechanism to further process the concatenated feature map to generate the final feature representation; S5.

4. Use the CTC decoder to convert the final feature representation into a text sequence, and perform post-processing on the decoded text sequence.

Images