Intelligent analysis method and system for medical laboratory sheet

Abstract

The invention discloses a medical laboratory sheet-oriented intelligent analysis method and system, which deeply integrate table structure detection and optical character recognition technologies, and in order to solve the problem of data scarcity in the medical field, systematic analysis and data enhancement are performed on a CMDD data set so as to expand the scale of a training sample; in the aspect of table detection, on the basis of a TableNet architecture, the performance of various feature extraction networks is explored, and a Transform attention mechanism and a feature pyramid network FPN module are innovatively introduced, so that the detection precision (IoU reaches 0.8423) and the environmental adaptability are improved; in the OCR stage, it is found through systematic evaluation that the combined scheme of TableNet-ResNet34 + Transform combined with PaddleOCR is optimal in the structured information extraction task of the medical laboratory sheet.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to an intelligent parsing method and system for medical test reports. Background Art

[0002] With the rapid development of medical informatization, the efficient acquisition and intelligent processing of medical data have become one of the key factors in improving the quality and efficiency of medical services. As an important basis for clinical diagnosis, medical test reports contain a large amount of patient health data and physiological index information, which are of great value for clinical decision-making, medical record management, and health monitoring. However, the format of medical test reports is complex, including professional terms, symbols, and handwritten content, which poses great challenges to automated processing. Existing OCR methods are difficult to accurately identify the structured information (such as patient basic information and test data) in medical test reports, limiting their application in medical scenarios. Specifically, medical test reports usually have a highly irregular table structure, with complex nesting, and the rows and columns of the table are not fixed, making it difficult for traditional OCR technologies to accurately locate table elements and extract text. In addition, medical documents often contain medical terms and special symbols, and the presence of these elements further increases the recognition difficulty of OCR systems.

[0003] For example, in the medical document table detection method based on convolutional neural network disclosed in CN2020108163977, although it has been successfully applied to the table recognition in medical documents and data extraction has been performed on the table content, however, this method is mostly for standardized documents, and there are still problems of insufficient accuracy for the complex table structure and terms in medical test reports. In addition, Li Ming et al. proposed a method for table recognition in electronic medical record documents based on deep learning, which combines convolutional neural network and RNN to improve the efficiency of table extraction. Although this method has achieved good results in the table detection of electronic medical records, it still needs to be further optimized for medical test reports with complex terms and irregular formats.

[0004] Therefore, there is an urgent need to develop an analytical method for efficient table structure detection and text recognition applicable to medical test reports. Summary of the Invention

[0005] Object of the Invention: To overcome the above deficiencies, the object of the present invention is to provide an intelligent parsing method and system for medical test reports.

[0006] To solve the above technical problems, the present invention provides an intelligent parsing method for medical test reports, including: S1: Perform image preprocessing and feature analysis on the medical diagnosis data set to generate a segmented image; S2: Use the trained multi-frame deep learning model to predict the table structure of the segmented image, generate the table structure prediction result, and extract the table cell positions according to the table structure prediction result; S3: Use a preset OCR method to extract text from the segmented image area matching the table cell positions to generate table data; S4: Structurally output the extracted table data in a preset format.

[0007] On the one hand, in S1, the method includes: S11: Extract a preset medical test report image as a medical diagnosis data set, convert the original RGB image of the medical diagnosis data set into a grayscale image, and perform binarization processing to generate a binarized image, so as to form a clear contrast between the table lines and the background; S12: Extract horizontal and vertical lines from the binarized image according to the structural elements in the design direction: , , where and respectively represent the extracted vertical and horizontal lines, represents the morphological opening operation, and respectively represent the vertical and horizontal structural elements, represents the binarized image; S13: Extract the y coordinates of all horizontal lines and merge the lines with relatively close distances through a clustering algorithm: , where, represents the position of the merged horizontal line, represents the y coordinates of all detected horizontal lines, represents the merging threshold; S14: Determine the key segmentation points according to the position of the merged horizontal lines, divide the original image into multiple table areas, and generate a segmented image.

[0008] On the one hand, in S2, the multi-frame deep learning model includes TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models; Among them, the calculation formula of the TableNet-ResNet34+Transformer is: , where Q, K, and V respectively represent the query, key, and value matrices, is the scaling factor; Among them, the TableNet-ResNet34+FPN introduces the FPN (Feature Pyramid Network) to fuse features to enhance the table detection ability. The feature fusion process is as follows: , where represents the -th layer feature map from the ResNet34 backbone network, is the pyramid feature map generated by the FPN at the -th layer, represents upsampling the previous layer's pyramid feature map, and adjusting the number of channels through a 1 ×1 convolution.

[0009] On the one hand, in S2, when training the multi-frame deep learning model, the method includes: S21: Perform image padding and enhancement processing according to a preset dataset; S22: Use the custom TableDataset class to identify and divide the rows, columns, and cells of the real images in the enhanced preset dataset; S23: Divide the preset dataset into a training set, a test set, and a validation set according to a preset ratio and set the loss function and optimizer; S24: Traverse all the images in the training set for each sub-model in the multi-frame deep learning model, calculate the error between the prediction result and the true label using the loss function, and update the weights using the optimizer; S25: Calculate the evaluation metrics on the validation set; S26: Loop from S24 to S25 until the preset requirements are met.

[0010] On the one hand, in S22, read the original images and the corresponding row, column, and cell labels, and uniformly scale the original images of different sizes to a preset size.

[0011] On the one hand, in S25, the method further includes: S251: Compare the performance of each sub-model in the multi-frame deep learning model according to the evaluation metrics and score the performance of each sub-model in extracting different table feature types; S252: Match the sub-model with the best performance with the corresponding table feature type, so that the multi-frame deep learning model can predict the locally optimal segmentation effect during prediction.

[0012] On the one hand, in S2, when the multi-frame deep learning model is predicting, the method includes: S201: Based on the segmented image, the multi-frame deep learning model automatically invokes the sub-model that performs best in terms of features according to the pre-established model feature matching relationship; S202: Each sub-model collaborates to complete table prediction, division, structured feature extraction, and fuse the extracted features to generate a table structure prediction result.

[0013] On the one hand, the preset OCR method includes one or more of TesseractOCR, EasyOCR, and PaddleOCR.

[0014] This application also provides an intelligent parsing system for medical test reports using the above method, including: A data processing module for performing image preprocessing and feature analysis on a medical diagnosis data set to generate a segmented image; A training and prediction module for training a multi-frame deep learning model, and using the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generating a table structure prediction result and extracting table cell positions according to the table structure prediction result; A text extraction module for performing text extraction on the segmented image area matching the table cell positions using a preset OCR method to generate table data; A data output module for structuring and outputting the extracted table data in a preset format.

[0015] The above technical solution of this application has the following advantages compared with the prior art: 1. Through the multi-frame fusion design, this application clearly points out the construction of a dedicated multi-frame TableNet model for medical test reports, including the improvement and integration of traditional networks (ResNet, MobileNet, EfficientNet), and at the same time adding Transformer and FPN modules, realizing a new method for automatic detection and segmentation of table structures.

[0016] 2. Design a dedicated preprocessing process and data augmentation strategy for medical images, effectively solving the problems of data scarcity and image quality, and ensuring the robustness of subsequent recognition.

[0017] 3. The organic combination of the OCR module (open-source engine) and table structure detection, using structural information to guide text recognition, significantly improves the accuracy of medical document information extraction. Description of the Drawings

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0019] Figure 1 It is a schematic diagram of a medical test report sample provided by an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram of the grayscale binaryzation result of the medical test report sample provided by an embodiment of the present application.

[0021] Figure 3 It is a schematic diagram of the extracted horizontal line layout provided by an embodiment of the present application.

[0022] Figure 4 It is a schematic diagram of an example of the segmentation table T1 provided by an embodiment of the present application.

[0023] Figure 5 It is a schematic diagram of an example of the segmentation table T2 provided by an embodiment of the present application.

[0024] Figure 6 It is a schematic diagram of an example of the T1 horizontal line positioning provided by an embodiment of the present application.

[0025] Figure 7 It is a schematic diagram of an example of the T2 horizontal line positioning provided by an embodiment of the present application.

[0026] Figure 8 It is a schematic diagram of the design framework of the multi-frame deep learning model provided by an embodiment of the present application.

[0027] Figure 9 It is a schematic diagram of the architecture of the ResNet34&Transformer model provided by an embodiment of the present application.

[0028] Figure 10 It is a schematic diagram of the architecture of the ResNet34&FPN model provided by an embodiment of the present application.

[0029] Figure 11 It is a schematic diagram of the comparison of the dataset sizes before and after expansion provided by an embodiment of the present application.

[0030] Figure 12 It is a schematic diagram of the image and the true segmentation label provided by an embodiment of the present application.

[0031] Figure 13 It is a schematic diagram of the training process of the TableNet-ResNet18 model provided by an embodiment of the present application.

[0032] Figure 14 It is a schematic diagram of the training process of the TableNet-ResNet34 model provided by an embodiment of the present application.

[0033] Figure 15 It is a schematic diagram of the comparison of detection results provided by an embodiment of the present application.

[0034] Figure 16 It is a schematic diagram of the comparison of the detection effects of four model architectures provided by an embodiment of the present application.

[0035] Figure 17 It is a schematic diagram of the optimization training process of a multi-frame deep learning model provided by an embodiment of the present application.

[0036] Figure 18 It is a schematic diagram of the true label of a simple sample provided by an embodiment of the present application.

[0037] Figure 19 It is a schematic diagram of the comparison of the structure prediction of a simple sample provided by an embodiment of the present application.

[0038] Figure 20 It is a schematic diagram of the true label of a medium sample provided by an embodiment of the present application.

[0039] Figure 21 It is a schematic diagram of the comparison of the structure prediction of a medium sample provided by an embodiment of the present application.

[0040] Figure 22 It is a schematic diagram of the comparison of cell division provided by an embodiment of the present application.

[0041] Figure 23 It is a schematic diagram of the comparison of the OCR results of cells provided by an embodiment of the present application.

[0042] Figure 24 It is a schematic diagram of the structuring and proofreading of the recognized content provided by an embodiment of the present application. Detailed implementation manners

[0043] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0044] With the rapid development of medical informatization, the efficient acquisition and intelligent processing of medical data have become one of the key factors in improving the quality and efficiency of medical services. As an important basis for clinical diagnosis, medical test reports contain a large amount of patient health data and physiological index information, which are of great value for clinical decision-making, medical record management, and health monitoring. However, the table structure of medical test reports is complex, usually involving nested tables and irregular formats, and contains medical terms, symbols, and handwritten content, which poses a huge challenge to its automated processing.

[0045] Existing optical character recognition (OCR) methods are difficult to accurately identify the structured information (such as patient basic information and test data) in medical test reports, restricting their application in medical scenarios. Specifically, medical test reports usually have a highly irregular table structure, with complex nesting, and the rows and columns of the table are not fixed, which makes it difficult for traditional OCR technologies to accurately locate table elements and extract text. In addition, medical documents often contain medical terms and special symbols with high complexity, and the existence of these elements further increases the recognition difficulty of the OCR system, resulting in low accuracy of information extraction. Specifically: the complex table structure, for example, medical test reports generally have nesting, unfixed rows and columns, and irregular layouts, and traditional OCR models are prone to errors when locating table boundaries and distinguishing adjacent cells; medical terms and special symbols, for example, there are a large number of professional terms, symbols, and handwritten contents in medical documents, which interfere with the recognition system; image noise and low-quality scans, for example, the original images often have problems such as blurring, noise, and uneven illumination, making it difficult to generate accurate segmentation information in the preprocessing stage (such as binarization and edge detection).

[0046] To solve the above problems, in recent years, deep learning technologies have made remarkable progress in table structure recognition and OCR tasks. In particular, the successful application of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in the fields of image and text processing has provided new solutions for table recognition and text extraction in medical documents. For example, the TableNet model, combining the advantages of deep learning, effectively improves the detection accuracy of table structures. However, most existing studies focus on the table recognition of standard tables and general documents, lacking in-depth research on medical test reports with complex table structures, rich terms, and irregular formats. Therefore, the current research still fails to effectively solve the problems of automated detection of table structures and efficient text extraction in medical test reports.

[0047] Therefore, in some embodiments, the present application relates to an intelligent parsing method for medical test reports, including: S1: Perform image preprocessing and feature analysis on a medical diagnosis dataset to generate a segmented image.

[0048] Specifically, in S1, the method includes: S11: Extract a preset medical test report image as a medical diagnosis data set and convert the original RGB image of the medical diagnosis data set into a grayscale image, and then apply adaptive threshold binarization to form a clear contrast between the table lines and the background. This process can be expressed as follows: , where represents the binarized image, represents the grayscale image, represents the adaptive threshold function, and respectively represent the window size and constant offset for threshold calculation. Refer to Figure 1 and Figure 2 as shown, Figure 2 shows the conversion from Figure 1 to its binarized version.

[0049] Among them, the preset medical test report image is formed by the user inputting the medical test report image to be parsed, specifically selected by the user according to actual needs.

[0050] S12: According to the structural elements in the design direction, extract horizontal and vertical lines from the binarized image respectively: , , where and respectively represent the extracted vertical and horizontal lines, represents the morphological opening operation, and respectively represent the vertical and horizontal structural elements. Refer to Figure 3 as shown, Figure 3 shows the horizontal lines extracted from the binarized image in Figure 2 . The detected horizontal line positions clearly reveal the image table features: the first and fourth lines represent the upper and lower boundaries of the entire table, while the second and third lines divide the image into upper and lower tables. Based on this information, the image can be initially segmented according to the order positioning of the image lines in the follow-up.

[0051] S13: The table segmentation algorithm is implemented based on the detected horizontal line positions. First, extract the y coordinates of all horizontal lines and merge the lines with relatively close distances through a clustering algorithm: , where represents the merged horizontal line positions, represents the y coordinates of all detected horizontal lines, represents the merging threshold; S14: Determine the key segmentation points according to the merged horizontal line positions, divide the original image into multiple table areas, and generate a segmented image. For example: Determine the key segmentation points according to the merged horizontal line positions, and divide the original image into table areas T1 and T2: , , where and respectively represent the first and second segmented tables, 、 、 represent the detected key horizontal line positions. Refer to the examples of the segmented tables shown in Figure 4 and Figure 5 . Compared with the original image, this table segmentation algorithm shows high precision. Apply it to two different source images to verify the effectiveness of the algorithm in line detection and positioning, and obtain two T2 tables, which accurately locate all the horizontal lines in these tables. Refer to Figure 6 and Figure 7 shown.

[0052] Thus, the image preprocessing process is completed. Further, the complete data preprocessing workflow integrates all the above functional modules into an end-to-end automated system. The entire processing flow framework can be expressed as: , where represents the final text recognition result, represents the original input image, 、 、 respectively represent preprocessing, table segmentation, table detection, and text recognition; segmented tables with consistent formats but different sizes are generated through the image preprocessing stage.

[0053] S2: Use the trained multi-frame deep learning model to predict the table structure of the segmented image, generate the table structure prediction result, and extract the table cell positions according to the table structure prediction result.

[0054] Specifically, in order to improve the accuracy of table detection and recognition in medical test report images, this application designs six improved TableNet architectures. The TableNet model is a deep learning architecture

[30] , which is used to extract neat table structure content from scanned document images, and its functions include detecting table structures and extracting data information of rows and columns in the table. Due to its excellent performance in detecting table structures, this application uses the TableNet model as the baseline model and optimizes the original network architecture based on the segmented medical images obtained after preprocessing. The highlight of this research is the realization of the progressive evolution of the combination of the model and different backbone networks, ranging from relatively simple to relatively complex architectures. These models are based on different backbone networks and enhancement modules to adapt to the complexity of various table structures. Refer to Figure 8 As shown, it shows the combination of TableNet and different feature extraction networks, and summarizes the overall model design framework and implementation process.

[0055] In S2, the multi-frame deep learning model includes TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models. Among them: The TableNet-MobileNet model uses MobileNetV2 as the backbone network, which is suitable for real-time applications, has the advantage of being lightweight and moderate resource consumption. Its streamlined structure can achieve efficient table detection even under limited computing resources. This model extracts features from MobileNetV2 and passes them to the segmentation head for processing, accurately predicting the table boundary and cell positions. The segmentation head consists of multiple convolutional layers and upsampling layers, and finally generates a segmentation map of the same size as the input image.

[0056] The TableNet-EfficientNet model is based on the EfficientNet-B0 network. This architecture enables this model to obtain a higher accuracy with fewer parameters. Its segmentation head structure is similar to that of the MobileNet network, but is adjusted according to the feature map size of EfficientNet to optimize its feature extraction ability.

[0057] For TableNet-ResNet18, this solution uses ResNet18 as the backbone network. ResNet18 is deeper than MobileNetV2 and enhances the feature extraction ability through residual connections, and can better handle complex table structures. The segmentation head consists of convolutional layers and is designed to generate a table segmentation mask from the features extracted by ResNet18. The training strategy of this model includes using the SGD optimizer and the ReduceLROnPlateau learning rate, and the learning rate can be dynamically adjusted according to the performance on the validation set.

[0058] Compared with ResNet18, TableNet-ResNet34 provides stronger feature extraction capabilities through a deeper network architecture. This model offers stronger expressive power but has higher computational requirements, so it is suitable for tasks involving complex table structures.

[0059] TableNet-ResNet34+Transformer, this variant of the TableNet model combines the ResNet34 network with the Transformer module. This module introduces the self-attention mechanism to enhance the ability to model complex table layouts. Its calculation formula is as follows: , where Q, K, and V represent the query, key, and value matrices respectively, is the scaling factor; this mechanism can effectively focus on the global information in the image and improve the model's ability to process complex table layouts, especially suitable for tables with large row and column spans or complex formats. The functional mechanism of the Transformer module constructed based on the ResNet34 feature extraction framework is as Figure 9 shown.

[0060] TableNet-ResNet34+FPN: This solution introduces the FPN (Feature Pyramid Network) to enhance the table detection ability. FPN achieves multi-scale representation through a top-down feature fusion strategy. This feature fusion process can be expressed as: , where represents the feature map of the th layer from the ResNet34 backbone network, is the pyramid feature map generated by FPN at the th layer, represents upsampling the previous layer's pyramid feature map, adjusts the number of channels through a 1 1 convolution. This method effectively improves the accuracy of table structure detection. The functional mechanism of the FPN architecture constructed based on the ResNet34 feature extraction framework is as Figure 10 shown.

[0061] S3: Apply a preset OCR method to the segmented image region matching the table cell position for text extraction to generate medical text information.

[0062] Specifically, in the final stage of the automated processing of medical test reports, OCR technology is needed to extract text information from the detected table structure. Such reports contain various character types, including Chinese, English, numbers, special symbols, etc., and there are complex situations such as handwritten, printed text, and noise interference. With the breakthrough of deep learning technology, many representative open-source projects have emerged in the OCR field, providing important technical support for the text detection and recognition of medical images. According to the characteristics of medical test report images, this application selects three of the most representative OCR processing methods for text content: The traditional Tesseract OCR engine performs well in cross-platform compatibility but is average in complex scenarios; the lightweight EasyOCR engine supports multi-language recognition and is more suitable for rapid deployment; PaddleOCR performs better in Chinese recognition and has a complete tool chain. The table detection and cell information extraction of medical documents are important links in the digitization of medical information. This application compares the performance of the multi-frame TableNet model in recognizing the table structure of medical segmentation images and screens the optimal architecture model for training and evaluation to recognize the table structure in medical segmentation images; based on the recognition results, the OCR engine is applied to perform text recognition on the specified cell area.

[0063] S4: Structurally output the extracted medical text information in a preset format.

[0064] Specifically, the extracted medical text information is saved and output in CSV and JSON formats to a set object.

[0065] In some embodiments of this application, when training a multi-frame deep learning model, the method includes: S21: Perform image padding and enhancement processing according to a preset data set.

[0066] Specifically, the preset training set adopted in this application can select the Chinese Medical Diagnosis Dataset (CMDD) provided by the Alibaba Cloud Tianchi platform. CMDD is designed specifically for medical document analysis research and contains Chinese medical test report images. These images have a unique double-table structure: the upper part is the patient information table (5 rows 4 columns), recording the basic information of the patient; the lower part is the test data table (usually 6 columns, 1 - 25 rows), storing medical test data. Moreover, to solve the problem of the limited scale of the CMDD data set, this application implements medical image enhancement processing. Considering the professionalism of medical test reports, subtle transformations are applied in this stage, expanding the data set to 30 times the original size while retaining the readability of the text. This method not only retains the key diagnostic information but also increases the data diversity. Statistical analysis is performed on the enhanced data set to compare the distribution of key features such as image width, height, and aspect ratio between the original data and the enhanced data.Figure 11 As shown, the enhancement process finally produced a dataset of 14,756 medical images, including 14,280 enhanced images and 476 original samples.

[0067] S22: Identify and divide the rows, columns, and cells of real images in the enhanced preset dataset through a custom TableDataset class.

[0068] S23: Divide the preset dataset into a training set, a test set, and a validation set according to a preset ratio and set a loss function and an optimizer.

[0069] Specifically, in this stage, the 14,756 medical image samples in the enhanced dataset are divided into a training set, a test set, and a validation set according to a ratio of 7:2:1; and a cross-entropy loss function and an SGD optimizer are set.

[0070] S24: Traverse all the images in the training set for each sub-model in the multi-frame deep learning model, calculate the error between the prediction result and the real label using the loss function, and update the weights using the optimizer.

[0071] S25: Calculate evaluation metrics on the validation set; among them, to evaluate the performance of the model, the following metrics are used in this application: the intersection over union (IoU) to evaluate the overlap between the predicted region and the real region, the F1 score to comprehensively measure the precision and recall rate, the accuracy (Accuracy) to evaluate the overall correct classification degree, and the loss function value (Loss) to reflect the difference between the model prediction and the real label. Thus, the performance of each sub-model in the multi-frame deep learning model is compared according to the evaluation metrics, and each sub-model is scored on its performance in extracting different table feature types. Furthermore, the sub-model with the best performance is matched with the corresponding table feature type to facilitate the multi-frame deep learning model to obtain a locally optimal segmentation effect during prediction.

[0072] S26: Loop through S24 to S25 until the preset requirements are met.

[0073] Thus, specifically, this application adopts a two-stage method: first, detect the table structure, including the table region, rows, columns, and cells, through an optimized TableNet model; then, based on the detection results, apply OCR technology to extract the cell text content. For the TableNet model, this application designs six variants to explore the best feature extraction strategy: (1), TableNet-ResNet18 (benchmark model) (2), TableNet-ResNet34 (deep enhanced type) (3), TableNet-MobileNet (lightweight) (4), TableNet-EfficientNet (Efficient Type) (5), TableNet-ResNet34+Transformer (Hybrid Type) (6), ableNet-ResNet34+FPN (Multi-Scale Type) To solve the problems of data loading and normalization, this application designs the TableDataset class, whose core functions include three points: realizing the unified loading of images and segmentation labels, uniformly scaling the original images of different sizes to 1024×1024, and generating row labels, column labels, and cell labels. The loading results of the images and their labels are as Figure 12 shown. Based on Figure 12 the label visualization results, the benchmark TableNet model can first adopt ResNet18 as the feature extractor and realize the segmentation of the table area by constructing a custom segmentation head. During the training process, the cross-entropy loss function (CrossEntropyLoss) is used to calculate the error between the prediction result and the true label, and the SGD optimizer is introduced to iteratively update the model weights. Specifically: Data loading and preprocessing: Use the custom TableDataset class to perform unified data loading, uniform scaling of image sizes (1024*1024), and label generation (construction of row, column, and cell labels) for the specific storage form and annotation files (pkl) of medical test reports; Network initialization and parameter settings: The initialization methods, the number of parameters, and the usage of pre-trained models adopted by each network; Optimizer and loss function, training cycle and batch size, dynamic monitoring and model saving strategy, visualization of the training process schematic diagram under different model schemes.

[0074] The benchmark model showed significant instability in the initial stage of training. The validation set accuracy (Val Accuracy), validation set loss (Val Loss), and training set loss (Train Loss) all showed large fluctuations, especially in the first 10 training batches. The specific training process is as Figure 13 shown. Figure 13 This phenomenon indicates that the ResNet18 architecture may not be able to fully capture the complex table features in the CMDD dataset after data augmentation. Therefore, this application designs multiple optimization paths: (1), Deep Enhancement Path: Replace ResNet18 with a deeper ResNet34 architecture to improve the feature extraction ability; (2), Lightweight Path: Adopt the MobileNet and EfficientNet architectures to optimize computing resources and improve the recognition ability for simple structures; (3) Hybrid architecture path: On the basis of ResNet34, the Transformer module and FPN feature pyramid are introduced respectively to enhance the long-distance dependence modeling and multi-scale feature expression capabilities.

[0075] Thus, after replacing ResNet18 with the deeper ResNet34 network, the model training process shows higher stability. The initial loss value of the training set is reduced by about 15% compared with the baseline model, and the initial loss value of the validation set is reduced by about 10%, indicating that the depth-enhanced model can extract table features more effectively. Its training process is as Figure 14 shown. The above training dynamics indicate that depth optimization is an improvement idea. To truly test the strong feature acquisition ability of the ResNet34 architecture, the segmentation effects of the two architectures on the same test images are compared, as Figure 15 shown.

[0076] The TableNet-ResNet34 architecture shows more accurate table boundary recognition ability, but there are still deficiencies in the recognition of some fine cells, and even the recognition effect on some fine cells is not as good as that of the TableNet-ResNet18 architecture.

[0077] The depth-enhanced path and the baseline model path have poor recognition ability on the fine cells of medical images. Therefore, the MobileNet and EfficientNet architectures are introduced in this experiment to improve the recognition ability of the model on fine features.

[0078] The segmentation effects of the four architectures (ResNet18, ResNet34, MobileNet, and EfficientNet) on the same medical image are as Figure 16 shown.

[0079] Through comparative analysis, it is found that: The MobileNet architecture performs best in the recognition of fine cells, especially for the cell structures in columns 2 and 3; The EfficientNet performs best in the recognition of complex cell structures, especially for the complex cells in column 1; While the ResNet series architectures do not show obvious advantages in the comprehensive recognition ability. The changes in the performance indicators of the four architectures during the training process are as Figure 17 shown. Figure 17Serial numbers 1 to 4 represent the MobileNet, EfficientNet, ResNet18, and ResNet34 network architectures respectively. It can be observed from their respective training curves that: the MobileNet architecture has the largest fluctuations and the worst initial performance on the validation set; the ResNet18 network shows unstable characteristics in both the early and late stages of training; the EfficientNet and ResNet34 perform relatively stably, but the losses on the validation set cannot break through 0.16, and the accuracy hovers around 93%. This indicates that a single feature extractor is difficult to comprehensively handle the diverse tabular structures in medical images, prompting further exploration of hybrid architecture solutions.

[0080] Therefore, in order to enhance the model's ability to recognize tables of different specifications, this paper introduces the Transformer module and the FPN feature pyramid based on ResNet34 respectively to construct two hybrid architectures: TableNet-ResNet34+Transformer and TableNet-ResNet34+FPN. Among them, TableNet-ResNet34+Transformer subsequently adds the input processing flowchart of the Transformer (ResNet34 feature map → Patch Embedding → positional encoding), and TableNet-ResNet34+FPN clarifies the number of layers of the FPN (4 layers) and the size of the feature map (1024×1024 → 64×64). At the same time, it is also necessary to show how to adjust the selection of the number of layers of the feature pyramid in the medical scenario.

[0081] The Transformer module enhances the global feature modeling ability through the self-attention mechanism and improves the recognition ability of complex tabular structures; while the FPN architecture improves the recognition ability of small targets (such as fine cells, thin table lines) through multi-scale feature representation. Table 1 shows the performance comparison of the four advanced architectures on key metrics: Table 1 Statistical Evaluation Metrics From the quantitative evaluation results, it can be seen that the two hybrid architectures outperform the lightweight architectures in all metrics. Among them, the Transformer hybrid architecture achieves the best performance, mainly due to the fact that the Transformer structure enhances the global information modeling ability, improves the recognition accuracy of complex tabular structures, and has the highest IoU (0.8423) and F1 score (0.9307), as well as the lowest loss value (0.1628); the performance of the FPN architecture is also very impressive, reaching a relatively high level in IoU (0.8351), and the F1 score and accuracy are close to those of the Transformer architecture.

[0082] To deeply evaluate the performance differences of the hybrid architectures, visual comparisons were made in the experiments on medical forms of different complexities. Refer to Figure 18 As shown, the ground truth of a medical form with a simple structure is presented (the row structure is represented in red, the column structure in green, and the cell structure in blue). The prediction results of the two hybrid architectures in this simple scenario are as Figure 19 shown. In the simple form scenario, the two hybrid architectures perform similarly, with only minor differences in the recognition of the "ID number" in the first column. However, the FPN architecture learns more irrelevant features and has a slightly lower accuracy than the Transformer architecture. The ground truth of a medical form of medium complexity is as Figure 20 shown. The prediction results of the two hybrid architectures in this scenario are as Figure 21 shown. In the medium complexity scenario, the differences between the architectures are more obvious: the FPN architecture makes misjudgments and recognizes non-text areas as a mixture of row and cell structures; the Transformer architecture performs better overall, but there are also missed detections in the recognition of the simple structure in the second column.

[0083] Comprehensive evaluation shows that the two hybrid architectures perform better than the other four optimized frameworks in the prediction of the medical image table structure, providing a high-quality table segmentation basis for the subsequent OCR process.

[0084] In some embodiments, to verify the impact of table structure detection on the OCR recognition effect, three schemes were designed in this stage: (1) Direct OCR: directly perform OCR on the original image without relying on table structure detection; (2) FPN + OCR: perform OCR based on the table detection results of the TableNet-ResNet34 + FPN model; (3) Transformer + OCR: perform OCR based on the table detection results of the TableNet-ResNet34 + Transformer model.

[0085] Meanwhile, the recognition effects of three OCR engines (Tesseract OCR, EasyOCR, and PaddleOCR) were compared in the experiments.

[0086] The cell division results of the three schemes for the same medical form (as Figure 20 shown) are as Figure 22 shown. Analyze Figure 22It is known that the direct OCR method lacks structural information, making it difficult to accurately restore the table logic, and the phenomenon of cell hybridization is frequent; for the cell division based on the FPN architecture and the Transformer architecture, the divided regions of the two are similar, but the divided region of the former is not complete enough, with gaps appearing in the division of large cells, and the accuracy is slightly inferior to the latter; in addition, the data in medical test reports has strong rigor in order, so strong differentiation is required between cells, and the table detection function applied by the TableNet model under the Transformer architecture has the best division performance among the three solutions, correctly differentiating the item number and the item name, effectively avoiding data interference between cells. In summary, the table detection function implemented by the TableNet model will directly affect the recognition area of the OCR engine, thereby indirectly affecting the subsequent OCR accuracy.

[0087] The performance of the three OCR engines in recognizing the text content of the first specified cell is as Figure 23 shown, where A, B, and C respectively correspond to Figure 21 the three OCR effects on the first cell under the three table detection solutions in Through comparative analysis, it is found that TesseractOCR has a low accuracy in English character recognition and the weakest overall performance; EasyOCR performs better in the recognition of mixed Chinese and English text; while PaddleOCR performs best in the recognition of complex Chinese characters and the combination of Chinese and English.

[0088] Based on the comprehensive results of table structure detection and OCR engine evaluation, the TableNet-ResNet34+Transformer+PaddleOCR solution achieves the best results in cell division and text content recognition. Figure 24 The results of the structured extraction of the medical form by this solution are shown.

[0089] The medical text information is finally saved in CSV and JSON formats for easy subsequent integration into the medical information system. Comparing with the original data shows that this solution realizes high-accuracy information extraction.

[0090] By comprehensively comparing the three OCR engine solutions and analyzing the final text recognition effect, it can be seen that the OCR method based on the table structure significantly improves the accuracy of the extraction of the table information in medical test reports; at the same time, the OCR system for specific medical test report forms considers the collaborative optimization of table structure recognition and text recognition.

[0091] Thus, through in-depth analysis of the CMDD dataset and targeted data augmentation strategies, the bottleneck problem of insufficient data in the medical field has been successfully solved. Based on the TableNet architecture, the performance of multiple feature extraction networks has been systematically evaluated, and the Transformer attention mechanism and FPN module have been integrated, significantly improving the accuracy and robustness of table detection. It is further confirmed that by preferentially constructing a high-performance model framework in the table detection link and then combining it with targeted OCR methods, the accuracy and reliability of overall text extraction can be significantly improved. Especially in the scenario of professional documents such as medical test reports, PaddleOCR demonstrates excellent recognition ability and adaptability.

[0092] In some embodiments, when the multi-frame deep learning model is making a prediction, the method includes: based on the segmented image, the multi-frame deep learning model automatically invokes the sub-model that performs best in terms of features according to the pre-established model feature matching relationship; each sub-model collaborates to complete table prediction, division, structured feature extraction, and fuse the extracted features to generate a table structure prediction result.

[0093] In some embodiments, the present application also relates to an intelligent parsing system for medical test reports using the above method, including: A data processing module, configured to perform image preprocessing and feature analysis on a medical diagnosis dataset to generate a segmented image; A training and prediction module, configured to train a multi-frame deep learning model, and use the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generate a table structure prediction result, and extract the table cell positions according to the table structure prediction result; A text extraction module, configured to perform text extraction on the segmented image region matching the table cell positions using a preset OCR method to generate table data; A data output module, configured to structurally output the extracted table data in a preset format.

[0094] In some embodiments, the present application also relates to a computer medium, on which a computer program is stored, and the computer program is executed by a processor to implement an intelligent parsing method for medical test reports according to any one of the above.

[0095] In some embodiments, the present application also relates to a computer, including the above computer medium.

[0096] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0097] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An intelligent parsing method for medical test reports, characterized in that It includes the following steps: S1: Perform image preprocessing and feature analysis on the medical diagnosis dataset to generate a segmented image; S2: Use the trained multi-frame deep learning model to predict the table structure of the segmented image, generate the table structure prediction result, and extract the table cell positions according to the table structure prediction result; S3: Extract text from the segmented image area matching the table cell positions using a preset OCR method to generate medical text information; S4: Structurally output the extracted medical text information in a preset format.

2. The intelligent parsing method for medical test reports according to claim 1, wherein, In S1, the method includes: S11: Extract the preset medical test report image as the medical diagnosis dataset, convert the original RGB image of the medical diagnosis dataset into a grayscale image, and perform binarization processing to generate a binarized image, thereby forming a clear contrast between the table lines and the background; S12: Extract horizontal and vertical lines from the binarized image according to the structural elements in the design direction; , , where and respectively represent the extracted vertical and horizontal lines, represents morphological opening operation, and respectively represent vertical and horizontal structuring elements, represents the binary image; S13: Extract the y coordinates of all horizontal lines and merge the lines with close distances through a clustering algorithm; , where represents the position of the merged horizontal line, represents the y - coordinates of all detected horizontal lines, represents the merging threshold; S14: Determine the key segmentation points according to the positions of the merged horizontal lines, divide the original image into multiple table areas, and generate a segmented image.

3. An intelligent parsing method for medical test reports according to claim 1 or 2, characterized in that In S2, the multi-frame deep learning model includes TableNet-MobileNet, TableNet-EfficientNet, TableNet-ResNet18, TableNet-ResNet34, TableNet-ResNet34+Transformer, and TableNet-ResNet34+FPN models; Among them, the calculation formula of the TableNet-ResNet34+Transformer is: , where Q, K, and V represent the query, key, and value matrices respectively, is the scaling factor; Among them, the TableNet-ResNet34+FPN introduces the FPN (Feature Pyramid Network) to fuse features and improve the table detection ability. The feature fusion process is as follows: , where represents the -th layer feature map from the ResNet34 backbone network, is the pyramid feature map generated by the FPN at the -th layer, represents upsampling the previous layer of the pyramid feature map, adjusts the number of channels through a 1 ×1 convolution.

4. The intelligent parsing method for medical test reports according to claim 1, wherein In S2, when training the multi-frame deep learning model, the method includes: S21: Perform image filling enhancement processing according to the preset dataset; S22: Identify and divide the rows, columns, and cells of the real images in the enhanced preset dataset through the custom TableDataset class; S23: Divide the preset dataset into a training set, a test set, and a validation set according to a preset ratio and set the loss function and optimizer; S24: Traverse all the images in the training set for each sub-model in the multi-frame deep learning model, calculate the error between the prediction result and the real label using the loss function, and update the weights using the optimizer; S25: Calculate the evaluation metrics on the validation set; S26: Loop S24 to S25 until the preset requirements are met.

5. The intelligent parsing method for medical test reports according to claim 4, wherein, In S22, read the original image and the corresponding row, column, and cell labels, and uniformly scale the original images of different sizes to the preset size.

6. The intelligent parsing method for medical test reports according to claim 4, characterized in that In S25, the method further includes: S251: Compare the performance of each sub-model in the multi-frame deep learning model according to the evaluation metrics and score the performance of each sub-model in extracting different table feature types; S252: Match the sub-model with the best performance with the corresponding table feature type, so that when predicting, the multi-frame deep learning model can predict a locally optimal segmentation effect.

7. An intelligent parsing method for medical test reports according to claim 4 or 6, characterized in that The evaluation metrics include one or more of intersection over union, F1 score, accuracy, and loss function value.

8. The intelligent parsing method for medical test reports according to claim 6, characterized in that In S2, when the multi-frame deep learning model is predicting, the method includes: S201: Based on the segmented image, the multi-frame deep learning model automatically invokes the sub-model with the best performance in terms of features according to the pre-established model feature matching relationship; S202: Each sub-model collaborates to complete table prediction, division, structured feature extraction, and fuse the extracted features to generate a table structure prediction result.

9. The intelligent parsing method for medical test reports according to claim 1, characterized in that Wherein, The preset OCR method includes one or more of TesseractOCR, EasyOCR, and PaddleOCR.

10. An intelligent parsing system for medical test reports using the method according to any one of claims 1-9, characterized in that, Includes: A data processing module for performing image preprocessing and feature analysis on a medical diagnosis data set to generate a segmented image; A training and prediction module for training a multi-frame deep learning model, and using the trained multi-frame deep learning model to perform table structure prediction on the segmented image, generate a table structure prediction result, and extract table cell positions according to the table structure prediction result; A text extraction module for performing text extraction on the segmented image area matching the table cell position using a preset OCR method to generate table data; A data output module for structuring and outputting the extracted table data in a preset format.

Images