An intelligent parsing method and system for medical test reports

Abstract

This application discloses an intelligent parsing method and system for medical test reports, which deeply integrates table structure detection and optical character recognition technology. To solve the problem of data scarcity in the medical field, this application systematically analyzes and enhances the CMDD dataset to expand the scale of training samples; in terms of table detection, based on the TableNet architecture, the performance of various feature extraction networks is explored, and the Transformer attention mechanism and feature pyramid network FPN module are innovatively introduced, thereby improving the detection accuracy (IoU reaches 0.8423) and environmental adaptability; in the OCR stage, through systematic evaluation, it is found that the combination of TableNet‑ResNet34+Transformer and PaddleOCR performs best in the structured information extraction task of medical test reports.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent parsing method and system for medical test reports. Background Art

[0002] With the rapid development of medical informatization, the efficient acquisition and intelligent processing of medical data have become key factors in improving the quality and efficiency of medical services. Medical test reports, as an important basis for clinical diagnosis, contain a wealth of patient health data and physiological indicators, which are of great value to clinical decision-making, medical record management, and health monitoring. However, the complex format of medical test reports, including professional terminology, symbols, and handwritten content, poses a significant challenge to automated processing. Existing optical character recognition (OCR) methods struggle to accurately recognize structured information (such as basic patient information and test data) in medical test reports, limiting their application in medical scenarios. Specifically, medical test reports typically have highly irregular table structures with complex nesting and unstable rows and columns, making it difficult for traditional OCR technology to accurately locate table elements and extract text. Furthermore, medical documents often contain medical terminology and special symbols, which further complicate recognition for OCR systems.

[0003] For example, the medical document table detection method based on convolutional neural networks disclosed in CN2020108163977 has been successfully applied to table recognition in medical documents and data extraction of table contents. However, this method is mostly targeted at standardized documents and still has the problem of insufficient accuracy for the complex table structures and terminology in medical test orders. In addition, Li Ming et al. proposed a method for table recognition in electronic medical record documents based on deep learning, and proposed a solution that combines convolutional neural networks and RNNs to improve the efficiency of table extraction. Although this method has achieved good results in table detection in electronic medical records, it still needs further optimization for medical test orders containing complex terminology and irregular formats.

[0004] Therefore, there is an urgent need to develop an efficient parsing method for table structure detection and text recognition suitable for medical test reports. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide an intelligent parsing method and system for medical test reports.

[0006] To solve the above technical problems, the present invention provides an intelligent parsing method for medical test reports, comprising:

[0007] S1: Perform image preprocessing and feature analysis on medical diagnostic datasets to generate segmented images;

[0008] S2: Using the trained multi-frame deep learning model to predict the table structure of the segmented image, generating a table structure prediction result and extracting the table cell position according to the table structure prediction result;

[0009] S3: extracting text from the segmented image area that matches the table cell position using a preset OCR method to generate table data;

[0010] S4: The extracted table data is structured and output in a preset format.

[0011] In one aspect, in S1, the method comprises:

[0012] S11: extracting a preset medical test report image as a medical diagnosis data set, converting the original RGB image of the medical diagnosis data set into a grayscale image, and performing binarization processing to generate a binarized image, so as to form a clear contrast between the table lines and the background;

[0013] S12: Extract horizontal and vertical lines from the binary image based on the structural elements of the design direction: , ,in and represent the extracted vertical and horizontal lines, respectively, represents the morphological opening operation, and Represent vertical and horizontal structural elements respectively, Represents a binary image;

[0014] S13: Extract the y coordinates of all horizontal lines and merge the lines with close distances using a clustering algorithm:

[0015] ,in, Indicates the position of the merged horizontal line, Represents the y coordinates of all detected horizontal lines, represents the merge threshold;

[0016] S14: Determine key segmentation points according to the positions of the merged horizontal lines, divide the original image into multiple table areas, and generate a segmented image.

[0017] On the one hand, in S2, the multi-framework deep learning models include TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models;

[0018] The calculation formula of TableNet-ResNet34+Transformer is:

[0019] , Q, K, V represent query, key and value matrices respectively, is the scaling factor;

[0020] Among them, the TableNet-ResNet34+FPN introduces FPN fusion feature pyramid network to improve table detection capabilities. The feature fusion process is as follows: ,in Represents the first Layer feature map, Is FPN in the first The pyramid feature map generated by the layer, Represents upsampling of the previous pyramid feature map. Pass 1 1Convolution adjusts the number of channels.

[0021] In one aspect, in S2, when training a multi-framework deep learning model, the method includes:

[0022] S21: Perform image filling and enhancement processing according to a preset data set;

[0023] S22: Use the custom TableDataset class to identify and divide the rows, columns, and cells of the real image into the enhanced preset dataset;

[0024] S23: Divide the preset data set into a training set, a test set, and a validation set according to a preset ratio and set a loss function and an optimizer;

[0025] S24: traversing all images of the training set for each sub-model in the multi-frame deep learning model and calculating the error between the predicted result and the true label using a loss function and updating the weight using an optimizer;

[0026] S25: Calculate evaluation metrics on the validation set;

[0027] S26: Loop S24 to S25 until the preset requirements are met.

[0028] On the one hand, in S22, the original image and the corresponding row, column and cell labels are read and the original images of different sizes are uniformly scaled to a preset size.

[0029] In one aspect, in S25, the method further includes:

[0030] S251: comparing the performance of each sub-model in the multi-framework deep learning model according to the evaluation indicators and scoring the performance of each sub-model in extracting different table feature types;

[0031] S252: Matching the sub-model with the best performance with the corresponding table feature type so that the multi-frame deep learning model can predict the locally optimal segmentation effect during prediction.

[0032] In one aspect, in S2, when the multi-framework deep learning model makes a prediction, the method includes:

[0033] S201: Based on the segmented image, the multi-frame deep learning model automatically calls the sub-model with the best performance in terms of features according to a pre-established model feature matching relationship;

[0034] S202: Each sub-model collaborates to complete table prediction, partitioning, and structured feature extraction and fuses the extracted features to generate a table structure prediction result.

[0035] In one aspect, the preset OCR method includes one or more of TesseractOCR, EasyOCR and PaddleOCR.

[0036] The present application also provides an intelligent parsing system for medical test reports using the method described, comprising:

[0037] The data processing module is used to perform image preprocessing and feature analysis on the medical diagnosis data set to generate segmented images;

[0038] a training prediction module, configured to train a multi-frame deep learning model, and use the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generate a table structure prediction result, and extract table cell positions based on the table structure prediction result;

[0039] A text extraction module is used to extract text from the segmented image area that matches the position of the table cells using a preset OCR method to generate table data;

[0040] The data output module is used to output the extracted table data in a structured manner in a preset format.

[0041] The above technical solution of the present application has the following advantages over the prior art:

[0042] 1. This application explicitly states that a dedicated multi-framework TableNet model has been constructed for medical test reports through a multi-framework fusion design. This includes improvements and integration of traditional networks (ResNet, MobileNet, and EfficientNet), while incorporating Transformer and FPN modules to implement a new method for automatic detection and segmentation of table structures.

[0043] 2. Design a preprocessing process and data enhancement strategy specifically for medical images, which effectively solves the problems of data scarcity and image quality and ensures the robustness of subsequent recognition.

[0044] 3. The OCR module (open source engine) is organically combined with table structure detection, and structural information is used to guide text recognition, which significantly improves the accuracy of medical document information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0046] Figure 1 This is a schematic diagram of a sample medical test report provided in an embodiment of the present application.

[0047] Figure 2 This is a schematic diagram of the grayscale binarization results of a medical test sample provided in an embodiment of the present application.

[0048] Figure 3 This is a schematic diagram of the extracted horizontal line layout provided in an embodiment of the present application.

[0049] Figure 4 This is a schematic diagram of an example of the segmentation table T1 provided in an embodiment of the present application.

[0050] Figure 5 This is a schematic diagram of an example of the segmentation table T2 provided in an embodiment of the present application.

[0051] Figure 6 Schematic diagram of an example of T1 horizontal line positioning provided in an embodiment of the present application.

[0052] Figure 7 Schematic diagram of an example of T2 horizontal line positioning provided in an embodiment of the present application.

[0053] Figure 8 This is a schematic diagram of the design framework of the multi-framework deep learning model provided in the embodiment of the present application.

[0054] Figure 9 This is a schematic diagram of the architecture of the ResNet34&Transformer model provided in an embodiment of the present application.

[0055] Figure 10 This is a schematic diagram of the architecture of the ResNet34&FPN model provided in an embodiment of the present application.

[0056] Figure 11 3 is a schematic diagram comparing the sizes of data sets before and after expansion provided in an embodiment of the present application.

[0057] Figure 12 Schematic diagram of the image and true segmentation label provided in the embodiment of the present application.

[0058] Figure 13 This is a schematic diagram of the TableNet-ResNet18 model training process provided in an embodiment of the present application.

[0059] Figure 14 This is a schematic diagram of the TableNet-ResNet34 model training process provided in an embodiment of the present application.

[0060] Figure 15 2 is a schematic diagram for comparing the test results provided in the embodiments of the present application.

[0061] Figure 16 This is a schematic diagram comparing the detection effects of the four model architectures provided in the embodiments of this application.

[0062] Figure 17 This is a schematic diagram of the optimization training process of the multi-framework deep learning model provided in an embodiment of the present application.

[0063] Figure 18 This is a schematic diagram of a simple sample real label provided in an embodiment of the present application.

[0064] Figure 19 This is a schematic diagram of a simple sample structure prediction comparison provided in an embodiment of the present application.

[0065] Figure 20 This is a schematic diagram of the real label of the medium sample provided in the embodiment of the present application.

[0066] Figure 21 This is a schematic diagram of the comparison of medium sample structure prediction provided in the embodiment of the present application.

[0067] Figure 22 This is a schematic diagram for comparing cell divisions provided in an embodiment of the present application.

[0068] Figure 23 2 is a schematic diagram comparing the cell OCR results provided in the examples of this application.

[0069] Figure 24This is a schematic diagram of content structuring and proofreading provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0071] With the rapid development of medical informatization, the efficient acquisition and intelligent processing of medical data have become key factors in improving the quality and efficiency of medical services. Medical test reports, as an important basis for clinical diagnosis, contain a wealth of patient health data and physiological indicators, which are of great value to clinical decision-making, medical record management, and health monitoring. However, the complex structure of medical test reports, which often involve nested tables and irregular formats, and contain medical terminology, symbols, and handwritten content, poses a significant challenge to their automated processing.

[0072] Existing optical character recognition (OCR) methods struggle to accurately recognize structured information (such as patient information and test data) found in medical test reports, limiting their application in healthcare settings. Specifically, medical test reports often feature highly irregular table structures with complex nesting and variable rows and columns. This makes it difficult for traditional OCR technologies to accurately locate table elements and extract text. Furthermore, medical documents often contain highly complex medical terminology and special symbols, which further complicate recognition for OCR systems and lead to low information extraction accuracy. Specifically, complex table structures, such as those found in medical test reports with nested, variable rows and columns, and irregular layouts, can make traditional OCR models prone to errors when locating table boundaries and distinguishing adjacent cells. Medical terminology and special symbols, such as the abundance of specialized terminology, symbols, and handwritten text in medical documents, can interfere with recognition systems. Image noise and low-quality scans, such as blur, noise, and uneven lighting in raw images, make it difficult to generate accurate segmentation information during preprocessing (such as binarization and edge detection).

[0073] To address these challenges, deep learning technology has made significant progress in table structure recognition and optical character recognition (OCR) tasks in recent years. In particular, the successful application of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in image and text processing has provided new solutions for table recognition and text extraction in medical documents. For example, the TableNet model, combining the advantages of deep learning, has significantly improved the accuracy of table structure detection. However, existing research has largely focused on table recognition for standard forms and general documents, lacking in-depth research on documents such as medical test reports, which have complex table structures, rich terminology, and irregular formats. Consequently, current research has yet to effectively address the challenges of automated table structure detection and efficient text extraction in medical test reports.

[0074] Therefore, in some embodiments, the present application relates to an intelligent parsing method for medical test reports, comprising:

[0075] S1: Perform image preprocessing and feature analysis on medical diagnostic datasets to generate segmented images.

[0076] Specifically, in S1, the method includes:

[0077] S11: Extracting a preset medical test report image as a medical diagnostic dataset and converting the original RGB image of the medical diagnostic dataset into a grayscale image, and then applying adaptive threshold binarization to create a clear contrast between the table lines and the background. This process can be described as follows: ,in, represents a binary image, represents a grayscale image, represents the adaptive threshold function, and Respectively represent the window size and constant offset for threshold calculation. Figure 1 and Figure 2 As shown, Figure 2 Shows the Figure 1 The conversion to its binarized version.

[0078] The preset medical examination report image is formed by the user inputting the medical examination report image to be analyzed, and is specifically selected by the user according to actual needs.

[0079] S12: Extract horizontal and vertical lines from the binary image based on the structural elements of the design direction:

[0080] , ,in and represent the extracted vertical and horizontal lines, respectively, represents the morphological opening operation, and Represents vertical and horizontal structural elements respectively. Figure 3 As shown, Figure 3 Shows the Figure 2 The horizontal lines extracted from the binarized image in

[15] clearly reveal the table features of the image: the first and fourth lines represent the upper and lower boundaries of the entire table, while the second and third lines divide the image into two upper and lower tables. Based on this information, the image can be preliminarily segmented according to the sequential positioning of the image lines.

[0081] S13: The table segmentation algorithm is implemented based on the detected horizontal line positions. First, the y coordinates of all horizontal lines are extracted and the lines with close distances are merged using a clustering algorithm: ,in, Indicates the position of the merged horizontal line, Represents the y coordinates of all detected horizontal lines, represents the merge threshold;

[0082] S14: Determine key segmentation points based on the merged horizontal line positions, divide the original image into multiple table areas, and generate segmented images. For example: Determine key segmentation points based on the merged horizontal line positions, and divide the original image into table areas T1 and T2: , ,in, and Represents the first and second partition tables respectively, 、 、 Represents the detected key horizontal line position, reference Figure 4 and Figure 5 The example of the segmentation table shown in the figure shows that the table segmentation algorithm shows a higher accuracy compared to the original image. It is applied to two different source images to verify the effectiveness of the algorithm in line detection and positioning to obtain two T2 tables, which accurately locate all horizontal lines in these tables. Figure 6 and Figure 7 shown.

[0083] Thus, the image preprocessing process is completed. Further, the complete data preprocessing workflow integrates all the above functional modules into an end-to-end automated system. The entire processing process framework can be expressed as:

[0084] ,in Represents the final text recognition result, represents the original input image, 、 、 They represent preprocessing, table segmentation, table detection and text recognition respectively; segmentation tables with consistent format but different sizes are generated through the image preprocessing stage.

[0085] S2: Use the trained multi-frame deep learning model to predict the table structure of the segmented image, generate a table structure prediction result, and extract the table cell position based on the table structure prediction result.

[0086] Specifically, in order to improve the accuracy of table detection and recognition in medical test report images, this application designs six improved TableNet architectures. The TableNet model is a deep learning architecture.

[30] , which is used to extract neat table structure content from scanned document images. Its functions include detecting table structure and extracting data information of rows and columns in the table. Due to its outstanding performance in detecting table structure, this application uses the TableNet model as the baseline model. Based on the segmented medical image obtained after preprocessing, the original network architecture is optimized. The highlight of this study is the realization of the gradual evolution of the model with different backbone network combinations, ranging from simpler to more complex architectures. These models are based on different backbone networks and enhancement modules to adapt to the complexity of various table structures. Reference Figure 8 As shown in the figure, the combination of TableNet and different feature extraction networks is demonstrated, and the overall model design framework and implementation process are summarized.

[0087] In S2, the multi-framework deep learning models include TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models.

[0088] The TableNet-MobileNet model uses MobileNetV2 as its backbone network, making it suitable for real-time applications due to its lightweight and moderate resource consumption. Its streamlined architecture enables efficient table detection even with limited computing resources. The model extracts features from MobileNetV2 and passes them to a segmentation head for processing, accurately predicting table boundaries and cell locations. The segmentation head consists of multiple convolutional and upsampling layers, ultimately generating a segmentation map of the same size as the input image.

[0089] The TableNet-EfficientNet model is based on the EfficientNet-B0 network. This architecture enables the model to achieve higher accuracy with fewer parameters. Its segmentation head structure is similar to that of the MobileNet network, but it is adjusted according to the feature map size of EfficientNet to optimize its feature extraction capabilities.

[0090] TableNet-ResNet18. This solution uses ResNet18 as the backbone network. ResNet18 is deeper and wider than MobileNetV2, and its residual connections enhance feature extraction capabilities, making it better able to handle complex table structures. The segmentation head consists of convolutional layers designed to generate table segmentation masks from the features extracted by ResNet18. The model's training strategy includes the SGD optimizer and the ReduceLROnPlateau learning rate, which is dynamically adjusted based on validation set performance.

[0091] Compared with ResNet18, TableNet-ResNet34 provides stronger feature extraction capabilities through a deeper network architecture. This model provides stronger expressive power but has higher computational requirements, making it suitable for tasks involving complex table structures.

[0092] TableNet-ResNet34+Transformer: This variant of the TableNet model combines a ResNet34 network with a Transformer module. This module introduces a self-attention mechanism to enhance the modeling of complex table layouts. Its calculation formula is as follows:

[0093] , Q, K, V represent query, key and value matrices respectively, is the scaling factor; this mechanism can effectively focus on the global information in the image and improve the model's ability to handle complex table layouts, especially for tables with large row and column spans or complex formats. The Transformer module function mechanism built on the ResNet34 feature extraction framework is as follows Figure 9 shown.

[0094] TableNet-ResNet34+FPN: This solution introduces the FPN feature pyramid network to improve table detection capabilities. FPN achieves multi-scale representation through a top-down feature fusion strategy. The feature fusion process can be expressed as:

[0095] ,in Represents the first Layer feature map, Is FPN in the first The pyramid feature map generated by the layer, Represents upsampling of the previous pyramid feature map. Pass 1 1. Convolution adjusts the number of channels. This method effectively improves the accuracy of table structure detection. The FPN architecture function mechanism built based on the ResNet34 feature extraction framework is as follows: Figure 10 shown.

[0096] S3: Using a preset OCR method to extract text from the segmented image area that matches the table cell position to generate medical text information.

[0097] Specifically, in the final stage of automated processing of medical test reports, optical character recognition (OCR) technology is required to extract text from detected table structures. Such reports contain a variety of character types, including Chinese, English, numbers, and special symbols, and are often complex, including handwritten text, printed text, and noise interference. With breakthroughs in deep learning technology, numerous representative open-source projects have emerged in the field of OCR, providing important technical support for text detection and recognition in medical images. Based on the characteristics of medical test report images, this application selects three of the most representative OCR methods for text content: the traditional Tesseract OCR engine offers excellent cross-platform compatibility but performs poorly in complex scenarios; the lightweight EasyOCR engine supports multiple languages ​​and is more suitable for rapid deployment; and PaddleOCR offers superior Chinese recognition and a comprehensive toolchain. Table detection and cell information extraction in medical documents are crucial steps in the digitization of medical information. This application compares the performance of multiple TableNet models in recognizing table structures in medical segmentation images and selects the optimal architecture for training and evaluation. This application then applies the OCR engine to perform text recognition in designated cell areas based on the recognition results.

[0098] S4: The extracted medical text information is structured and output in a preset format.

[0099] Specifically, the extracted medical text information is saved in CSV and JSON formats and output to the specified target.

[0100] In some embodiments of the present application, when training a multi-framework deep learning model, the method includes:

[0101] S21: Perform image filling and enhancement processing according to a preset data set.

[0102] Specifically, the preset training set used in this application can be the Chinese Medical Diagnosis Dataset (CMDD) provided by the Alibaba Cloud Tianchi platform. CMDD is specifically designed for medical document analysis research and contains images of Chinese medical test reports. These images have a unique dual-table structure: the upper half is the patient information table (5 rows 4 columns), recording basic patient information; the lower part is a test data table (usually 6 columns, 1-25 rows), which stores medical test data. In addition, in order to solve the problem of limited size of CMDD datasets, this application implements medical image enhancement processing. Taking into account the professionalism of medical test reports, subtle transformations are applied at this stage to expand the dataset to 30 times its original size while retaining the readability of the text. This method not only retains key diagnostic information but also increases the diversity of the data. A statistical analysis was performed on the enhanced dataset to compare the distribution of key features such as image width, height, and aspect ratio between the original data and the enhanced data. Figure 11 As shown in Figure 3, the enhancement process ultimately produced a dataset of 14,756 medical images, including 14,280 enhanced images and 476 original samples.

[0103] S22: Use the custom TableDataset class to identify and divide the rows, columns, and cells of the real image of the enhanced preset dataset.

[0104] S23: Divide the preset data set into a training set, a test set, and a validation set according to a preset ratio and set a loss function and an optimizer.

[0105] Specifically, in this stage, the 14,756 medical image samples in the enhanced dataset are divided into training set, test set and validation set in a ratio of 7:2:1; and the cross entropy loss function and SGD optimizer are set.

[0106] S24: Each sub-model in the multi-frame deep learning model traverses all images in the training set and uses a loss function to calculate the error between the prediction result and the true label and uses an optimizer to update the weight.

[0107] S25: Calculate evaluation metrics on the validation set; to evaluate model performance, this application uses the following metrics: Intersection over Union (IoU) evaluates the overlap between the predicted and true regions, the F1 score comprehensively measures precision and recall, accuracy evaluates the overall classification accuracy, and the loss function reflects the difference between the model prediction and the true label. Thus, based on the evaluation metrics, the performance of each sub-model in the multi-framework deep learning model is compared and each sub-model's performance in extracting different table feature types is scored. The sub-model with the best performance is then matched to the corresponding table feature type, so that the multi-framework deep learning model can predict the locally optimal segmentation effect.

[0108] S26: Loop S24 to S25 until the preset requirements are met.

[0109] Specifically, this application adopts a two-stage approach: first, the table structure, including table areas, rows, columns, and cells, is detected using an optimized TableNet model; then, based on the detection results, OCR technology is applied to extract the cell text content. For the TableNet model, this application designs six variants to explore the optimal feature extraction strategy:

[0110] (1) TableNet-ResNet18 (baseline model)

[0111] (2) TableNet-ResNet34 (depth enhancement)

[0112] (3) TableNet-MobileNet (lightweight)

[0113] (4) TableNet-EfficientNet (High-efficiency)

[0114] (5) TableNet-ResNet34+Transformer (hybrid)

[0115] (6) CapableNet-ResNet34+FPN (multi-scale)

[0116] To solve the data loading and standardization problems, this application designs the TableDataset class, whose core functions include three points: unified loading of images and segmentation labels, uniform scaling of original images of different sizes to 1024×1024, and generation of row labels, column labels, and cell labels. The image and its label loading results are shown in the following figure. Figure 12 As shown, based on Figure 12The benchmark TableNet model first uses ResNet18 as a feature extractor and implements table region segmentation by constructing a custom segmentation head. During training, the cross-entropy loss function (CrossEntropyLoss) is used to calculate the error between the predicted results and the true labels, and the SGD optimizer is introduced to iteratively update the model weights. Specifically, the following steps are performed: Data loading and preprocessing: Using a custom TableDataset class, data is uniformly loaded, image resizing (1024*1024), and label generation (construction of row, column, and cell labels) is performed for the unique storage format and annotation files (pkl) of medical test reports; Network initialization and parameter setting: The initialization method, parameter count, and use of pre-trained models used by each network; Optimizer and loss function, training cycle and batch size, dynamic monitoring and model preservation strategies, and visualization of training process diagrams for different model schemes.

[0117] The baseline model exhibited significant instability in the early stages of training. The validation set accuracy (Val Accuracy), validation set loss (Val Loss), and training set loss (Train Loss) all showed large fluctuations, especially in the first 10 training batches. The specific training process is as follows: Figure 13 shown. Figure 13 This phenomenon indicates that the ResNet18 architecture may not be able to fully capture the complex tabular features in the CMDD dataset after data augmentation. To this end, this application has designed multiple optimization paths:

[0118] (1) Deep enhancement path: ResNet18 is replaced with a deeper ResNet34 architecture to improve feature extraction capabilities;

[0119] (2) Lightweight path: Using MobileNet and EfficientNet architectures to optimize computing resources and improve the ability to recognize simple structures;

[0120] (3) Hybrid architecture path: Based on ResNet34, the Transformer module and FPN feature pyramid are introduced to enhance the long-distance dependency modeling and multi-scale feature expression capabilities.

[0121] Therefore, after replacing ResNet18 with a deep ResNet34 network, the model training process showed higher stability. The initial loss value of the training set was reduced by about 15% compared with the baseline model, and the initial loss value of the validation set was reduced by about 10%, indicating that the deep enhanced model can extract table features more effectively. Figure 14The above training dynamics show that deep optimization is an improvement idea. In order to truly test the strong feature acquisition capability of the ResNet34 architecture, the segmentation effects of the two architectures on the same test image are compared, as shown in the figure below. Figure 15 shown.

[0122] The TableNet-ResNet34 architecture demonstrates more accurate table boundary recognition capabilities, but it still has shortcomings in recognizing some subtle cells, and its recognition effect on some subtle cells is even worse than that of the TableNet-ResNet18 architecture.

[0123] The deep enhancement path and the baseline model path have poor recognition capabilities for subtle cells in medical images. Therefore, this experiment introduces the MobileNet and EfficientNet architectures to improve the model's recognition capabilities for subtle features.

[0124] The segmentation effects of four architectures (ResNet18, ResNet34, MobileNet and EfficientNet) on the same medical image are as follows Figure 16 shown.

[0125] Through comparative analysis, we found that the MobileNet architecture performs best in identifying subtle cells, especially for the cell structures in the second and third columns; EfficientNet performs best in identifying complex cell structures, especially for the complex cells in the first column; and the ResNet series architectures do not show a significant advantage in comprehensive recognition capabilities. The performance indicators of the four architectures during training are as follows: Figure 17 shown. Figure 17 Numbers 1 to 4 represent the MobileNet, EfficientNet, ResNet18, and ResNet34 network architectures, respectively. The respective training curves show that the MobileNet architecture exhibits the greatest fluctuations and the worst initial performance on the validation set; the ResNet18 network exhibits instability both early and late in training; and EfficientNet and ResNet34 perform relatively stably, but neither exhibits a loss above 0.16 on the validation set, with accuracy hovering around 93%. This suggests that a single feature extractor is unable to fully address the diverse table structures found in medical images, prompting further exploration of hybrid architecture solutions.

[0126] Therefore, in order to enhance the model's ability to recognize tables of different specifications, this paper introduces the Transformer module and FPN feature pyramid based on ResNet34, and constructs two hybrid architectures: TableNet-ResNet34+Transformer and TableNet-ResNet34+FPN. TableNet-ResNet34+Transformer subsequently adds the input processing flowchart of Transformer (ResNet34 feature map → Patch Embedding → position encoding), and TableNet-ResNet34+FPN specifies the number of FPN layers (4 layers) and feature map size (1024×1024→64×64). At the same time, it is also necessary to reflect how to adjust the number of feature pyramid layers in medical scenarios.

[0127] The Transformer module enhances global feature modeling capabilities through a self-attention mechanism, improving the recognition of complex table structures. The FPN architecture, on the other hand, improves the recognition of small objects (such as tiny cells and thin table lines) through multi-scale feature representation. Table 1 shows a performance comparison of the four advanced architectures on key metrics:

[0128] Table 1 Evaluation index statistics

[0129]

[0130] Quantitative evaluation results show that both hybrid architectures outperform the lightweight architecture across all metrics. The Transformer hybrid architecture achieves the best performance, primarily due to the Transformer structure's enhanced global information modeling capabilities, which improves the accuracy of structural recognition of complex tables. It achieves the highest IoU (0.8423) and F1 score (0.9307), as well as the lowest loss (0.1628). The FPN architecture also performs very well, reaching a high IoU (0.8351) and approaching the Transformer architecture in terms of F1 score and accuracy.

[0131] To deeply evaluate the performance differences of the hybrid architecture, experiments were conducted on medical tables of different complexities for visual comparison. Figure 18 As shown in Figure 1, the true label of a simple medical table is shown (row structure is red, column structure is green, and cell structure is blue). The prediction results of the two hybrid architectures in this simple scenario are shown in Figure 1. Figure 19 As shown in the figure, in the simple table scenario, the two hybrid architectures perform similarly, with only a slight difference in the recognition of the first column "ID number". However, the FPN architecture learns more irrelevant features and is slightly less accurate than the Transformer architecture. The real label of a medium-complexity medical table is shown in the figure. Figure 20 The prediction results of the two hybrid architectures in this scenario are shown as follows. Figure 21 In medium-complexity scenarios, the differences between the architectures are more obvious: the FPN architecture misclassifies non-text areas as a mixture of row and cell structures; the Transformer architecture performs better overall, but also misses detections in the simple structure recognition in the second column.

[0132] Comprehensive evaluation shows that the two hybrid architectures outperform the other four optimization frameworks in predicting the table structure of medical images, providing a high-quality table segmentation foundation for subsequent OCR.

[0133] In some embodiments, to verify the impact of table structure detection on OCR recognition performance, three solutions are designed at this stage:

[0134] (1) Direct OCR: It does not rely on table structure detection and performs OCR directly on the original image;

[0135] (2) FPN+OCR: Perform OCR based on the table detection results of the TableNet-ResNet34+FPN model;

[0136] (3) Transformer+OCR: Perform OCR based on the table detection results of the TableNet-ResNet34+Transformer model.

[0137] At the same time, the experiment compared the recognition effects of three OCR engines (TesseractOCR, EasyOCR and PaddleOCR).

[0138] Three schemes for the same medical form (e.g. Figure 20 The cell division result is as shown in Figure 22 Analysis Figure 22 It was found that the direct OCR method lacks structural information, making it difficult to accurately restore the table logic, and the phenomenon of cell mixing is frequent; the cell division based on the FPN architecture and the Transformer architecture has similar division areas, but the former's division area is not complete, and gaps appear in the division of large cells, making it slightly less accurate than the latter; in addition, the data in medical test reports is highly rigorous in order, so a strong distinction needs to be made between cells. The table detection function applied by the TableNet model under the Transformer architecture has the best division performance among the three solutions, correctly distinguishing between project numbers and project names, and effectively avoiding data interference between cells. In summary, the table detection function implemented by the TableNet model will directly affect the recognition area of ​​the OCR engine, thereby indirectly affecting the accuracy of subsequent OCR.

[0139] The performance of the three OCR engines in recognizing the text content of the first specified cell is as follows: Figure 23 As shown, A, B, and C correspond to Figure 21 Three OCR effects for the first cell under the three table detection schemes:

[0140] The comparative analysis found that TesseractOCR has low accuracy in English character recognition and the weakest overall performance; EasyOCR performs well in mixed Chinese and English text recognition; and PaddleOCR performs best in recognizing complex Chinese characters and Chinese and English combinations.

[0141] Based on the comprehensive table structure detection and OCR engine evaluation results, the TableNet-ResNet34+Transformer+PaddleOCR solution achieved the best results in cell division and text content recognition. Figure 24 The results of structured extraction of medical forms using this solution are shown.

[0142] The medical text information is ultimately saved in CSV and JSON formats, facilitating subsequent integration into medical information systems. Comparison with the original data shows that this solution achieves highly accurate information extraction.

[0143] By comprehensively comparing three OCR engine solutions and analyzing the final text recognition results, it can be seen that the table structure-based OCR method significantly improves the accuracy of information extraction from medical test form sheets; at the same time, the OCR system for specific medical test form sheets considers the coordinated optimization of table structure recognition and text recognition.

[0144] Therefore, through in-depth analysis of the CMDD dataset and targeted data augmentation strategies, the bottleneck problem of insufficient data in the medical field was successfully resolved. Based on the TableNet architecture, the performance of multiple feature extraction networks was systematically evaluated and the Transformer attention mechanism and FPN module were integrated, significantly improving the accuracy and robustness of table detection. It was further confirmed that by prioritizing the construction of a high-performance model framework in the table detection stage and combining it with targeted OCR methods, the accuracy and reliability of overall text extraction can be significantly improved. Especially in professional document scenarios such as medical test reports, PaddleOCR demonstrated excellent recognition capabilities and adaptability.

[0145] In some embodiments, when the multi-frame deep learning model is predicting, the method includes: based on the segmented image, the multi-frame deep learning model automatically calls the sub-model with the best feature performance according to a pre-established model feature matching relationship; each sub-model collaborates to complete table prediction, division, structured feature extraction and fuses the extracted features to generate a table structure prediction result.

[0146] In some embodiments, the present application also relates to an intelligent parsing system for medical test reports using the method described, comprising:

[0147] The data processing module is used to perform image preprocessing and feature analysis on the medical diagnosis data set to generate segmented images;

[0148] a training prediction module, configured to train a multi-frame deep learning model, and use the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generate a table structure prediction result, and extract table cell positions based on the table structure prediction result;

[0149] A text extraction module is used to extract text from the segmented image area that matches the position of the table cells using a preset OCR method to generate table data;

[0150] The data output module is used to output the extracted table data in a structured manner in a preset format.

[0151] In some embodiments, the present application also relates to a computer medium having a computer program stored thereon, and the computer program is executed by a processor to implement any of the above-mentioned intelligent parsing methods for medical test reports.

[0152] In some embodiments, the present application also relates to a computer, comprising the above-mentioned computer medium.

[0153] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0154] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An intelligent parsing method for medical test reports, characterized in that: The following steps are involved: S1: Perform image preprocessing and feature analysis on medical diagnostic datasets to generate segmented images; S2: Using the trained multi-frame deep learning model to predict the table structure of the segmented image, generating a table structure prediction result and extracting the table cell position according to the table structure prediction result; S3: extracting text from the segmented image area that matches the table cell position using a preset OCR method to generate medical text information; S4: Output the extracted medical text information in a structured manner in a preset format; In S1, the method comprises: S11: extracting a preset medical test report image as a medical diagnosis data set, converting the original RGB image of the medical diagnosis data set into a grayscale image, and performing binarization processing to generate a binarized image, so as to form a clear contrast between the table lines and the background; S12: Extract horizontal and vertical lines from the binary image based on the structural elements of the design direction: , ,in and represent the extracted vertical and horizontal lines, respectively, represents the morphological opening operation, and Represent vertical and horizontal structural elements, Represents a binary image; S13: Extract the y coordinates of all horizontal lines and merge the lines with close distances using a clustering algorithm: ,in, Indicates the position of the merged horizontal line, Represents the y coordinates of all detected horizontal lines, represents the merge threshold; S14: determining key segmentation points according to the positions of the merged horizontal lines, dividing the original image into multiple table areas, and generating a segmented image; In S2, when training a multi-framework deep learning model, the method includes: S21: Perform image filling and enhancement processing according to a preset data set; S22: Use the custom TableDataset class to identify and divide the rows, columns, and cells of the real image into the enhanced preset dataset; S23: Divide the preset data set into a training set, a test set, and a validation set according to a preset ratio and set a loss function and an optimizer; S24: traversing all images of the training set for each sub-model in the multi-frame deep learning model and calculating the error between the predicted result and the true label using a loss function and updating the weight using an optimizer; S25: Calculate evaluation metrics on the validation set; S26: Loop S24 to S25 until the preset requirements are met.

2. The intelligent parsing method for medical test reports according to claim 1, characterized in that: In S2, the multi-framework deep learning models include TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models; The calculation formula of TableNet-ResNet34+Transformer is: , Q, K, V represent query, key and value matrices respectively, is the scaling factor; Among them, the TableNet-ResNet34+FPN introduces FPN fusion feature pyramid network to improve table detection capabilities. The feature fusion process is as follows: ,in Represents the first Layer feature map, Is FPN in the first The pyramid feature map generated by the layer, Represents upsampling of the previous pyramid feature map. Pass 1 1Convolution adjusts the number of channels.

3. The intelligent parsing method for medical test reports according to claim 1, characterized in that: In S22 , the original image and the corresponding row, column, and cell labels are read and the original images of different sizes are uniformly scaled to a preset size.

4. The intelligent parsing method for medical test reports according to claim 1, characterized in that: In S25, the method further includes: S251: comparing the performance of each sub-model in the multi-framework deep learning model according to the evaluation indicators and scoring the performance of each sub-model in extracting different table feature types; S252: Matching the sub-model with the best performance with the corresponding table feature type so that the multi-frame deep learning model can predict the locally optimal segmentation effect during prediction.

5. The intelligent parsing method for medical test reports according to claim 1 or 4, characterized in that: The evaluation indicators include one or more of intersection-over-union ratio, F1 score, accuracy, and loss function value.

6. The intelligent parsing method for medical test reports according to claim 4, characterized in that: In S2, when the multi-framework deep learning model makes a prediction, the method includes: S201: Based on the segmented image, the multi-frame deep learning model automatically calls the sub-model with the best performance in terms of features according to a pre-established model feature matching relationship; S202: Each sub-model collaborates to complete table prediction, partitioning, and structured feature extraction and fuses the extracted features to generate a table structure prediction result.

7. The intelligent parsing method for medical test reports according to claim 1, characterized in that: in, The preset OCR method includes one or more of TesseractOCR, EasyOCR and PaddleOCR.

8. An intelligent analysis system for medical test reports using the method according to any one of claims 1 to 7, characterized in that: include: The data processing module is used to perform image preprocessing and feature analysis on the medical diagnosis data set to generate segmented images; a training prediction module, configured to train a multi-frame deep learning model, and use the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generate a table structure prediction result, and extract table cell positions based on the table structure prediction result; A text extraction module is used to extract text from the segmented image area that matches the position of the table cells using a preset OCR method to generate table data; The data output module is used to output the extracted table data in a structured manner in a preset format.

Images