Medical test sheet image text detection method based on SFE-DBnet

Abstract

The application discloses a medical test report image text detection method based on SFE-DBnet, and the process is as follows: after obtaining the image of a medical test report and inputting the SFE-DBnet network in a computer, a prediction candidate box is obtained to mark the text in the test report; the SFE-DBnet network is constructed based on a DBnet network and comprises a backbone network of ResNext50 combined with a CBAM (CBAM) and an aggregation residual block to extract features, then multi-scale feature extraction and fusion are carried out through an FPEM_FFM module, and then horizontal direction spatial sequence feature extraction is carried out through a BLSTM network to obtain a feature map. The SFE-DBnet network can effectively detect and identify the text such as serial numbers, item names, English names, measured values, marks, units, reference intervals and test methods in the medical test report, and the detection capacity is improved.

Description

Technical Field

[0001] This invention relates to the field of laboratory report text detection technology, specifically a method for detecting text in medical laboratory report images based on SFE-DBnet. Background Technology

[0002] In modern medical diagnosis and treatment, examinations and tests have become indispensable tools. Past test reports are frequently used in situations such as patient transfers and insurance applications, and these reports are usually archived as photographic images. However, images are an unstructured data format, which is not conducive to further utilization of information. Therefore, intelligent detection and recognition technology for test reports has emerged. However, medical test reports often present various problems, such as mixed text (Chinese, English, numbers), uneven horizontal text arrangement, and the influence of photograph angle and lighting, making detection difficult. At the same time, the accuracy requirements for text detection on test reports are extremely high; a single misidentification could potentially lead to a serious medical accident. Therefore, designing a high-precision algorithm for detecting text information on medical test reports is both necessary and important.

[0003] Traditional text region detection primarily leverages the characteristics of digital images themselves. The Canny edge detection algorithm can be used to detect all edges, and the Hough transform line detection can be used to identify straight lines in the image, thus detecting text regions. In recent years, with the rapid development of deep learning, research on deep learning-based text detection has also surged. For example, the text detection network DBnet proposed a differentiable binarization (DB) method. In segmentation-based text detection networks, DB implements binarization processing. Segmentation networks based on the DB module can adaptively set the binarization threshold, simplifying post-processing and improving text detection performance. However, the challenge of scene-based text detection lies in the complexity of the background, making it difficult to distinguish between text and background. Therefore, most text detection algorithms, including DBnet, focus on features that differentiate text from non-text backgrounds. The difficulty of medical lab report detection lies in the diverse character types and uneven horizontal spacing of blank lines. Therefore, existing text detection networks (such as DBnet) are difficult to directly apply to lab report text detection tasks. Meanwhile, due to the scarcity of publicly available image datasets of medical test reports, there are very few studies on text detection for medical test reports. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for detecting text in medical test reports based on SFE-DBnet, which can effectively detect and identify text such as serial number, item name, English name, measured value, symbol, unit, reference range and test method in medical test reports.

[0005] To address the aforementioned technical problems, this invention provides a method for text detection in medical test report images based on SFE-DBnet. The specific process includes: acquiring an image of a medical test report and inputting it into the SFE-DBnet network in a computer, and finally obtaining prediction candidate boxes to mark the text in the test report.

[0006] The SFE-DBnet network is built on the DBnet network. It includes extracting features through a ResNext50 backbone network that combines the attention mechanism module CBAM and aggregated residual blocks, then performing multi-scale feature extraction and fusion through the FPEM_FFM module, and finally extracting horizontal spatial sequence features through the BLSTM network to obtain feature maps.

[0007] As an improvement to the SFE-DBnet-based medical laboratory report image text detection method of the present invention:

[0008] The backbone network comprises a convolutional layer, a max pooling layer, three aggregated residual blocks (ResNextblock), an attention mechanism module (CBAM), an aggregated residual block (ResNextblock), and an attention mechanism module (CBAM) connected in sequence.

[0009] As a further improvement to the SFE-DBnet-based method for text detection in medical laboratory report images of this invention:

[0010] The FPEM_FFM module, as the neck part, includes the Feature Pyramid Enhancement Module (FPEM) and the Feature Fusion Module (FFM). The output of the Feature Pyramid Enhancement Module (FPEM) serves as the input of the Feature Fusion Module (FFM).

[0011] As a further improvement to the SFE-DBnet-based medical laboratory report image text detection method of the present invention:

[0012] The training and testing process of the SFE-DBnet network is as follows: Images from laboratory reports are collected, and training and testing sets are established. Training parameters are set, including learning rate, num_work, batch_size, and epoch. The training set is input into the SFE-DBnet network. During training, the loss function value is calculated, and backpropagation is used to iteratively optimize the model parameters until the preset epoch is reached, at which point training ends. Then, the test set is input into the trained model to obtain the intersection-over-union ratio (IoU). Precision, recall, and harmonic mean are then used as evaluation metrics to obtain an online-usable SFE-DBnet network.

[0013] As a further improvement to the SFE-DBnet-based method for text detection in medical laboratory report images of this invention:

[0014] The training set and test set are obtained by collecting medical test reports, taking pictures and saving them as JPG images, including images with different lighting conditions and shooting angles, and then labeling them using the labelimg tool.

[0015] The beneficial effects of this invention are mainly reflected in:

[0016] 1. The SFE-DBnet network of this invention adds CBAM attention mechanism, ResNext50, FPEM_FFM module and BLSTM module, which can effectively detect and recognize text such as serial number, item name, English name, measured value, symbol, unit, reference range and test method in medical test reports, thus improving the detection capability.

[0017] 2. The SFE-DBnet network of this invention uses ResNext50, which has strong feature extraction capabilities and low computational cost, as the backbone network, and introduces the CBAM (Convolutional Block Attention Module) attention mechanism to improve the weight distribution, thereby assisting the network to focus more effectively on the target's location information, enhancing robustness, improving detection efficiency and effect, and improving the problem of difficult text region feature recognition caused by complex lighting environments in medical test report photos;

[0018] 2. The Head part of the SFE-DBnet network of the present invention adopts the FPEM_FFM module. FPEM has a cascaded structure and low computational cost, which can make features of different sizes deeper and more representative. The introduction of FFM can enhance the connection between high-level semantic features and low-level semantic features, and improve the recognition problem of the variety of characters and different font sizes on the test report.

[0019] 3. The SFE-DBnet network of this invention introduces a bidirectional long short-term memory (BLSTM) module, which can help the network learn horizontal text features and effectively solve the problem of uneven spacing of blank lines in the horizontal direction of medical test report text. Attached Figure Description

[0020] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the SFE-DBnet network model architecture of the present invention;

[0022] Figure 2 for Figure 1A schematic diagram of the ResNext50 network architecture that integrates the CBAM attention mechanism;

[0023] Figure 3 The diagram shows the aggregated residual structure ((a) is a schematic diagram of the internal structure of the residual structure, and (b) is a schematic diagram of the overall residual structure).

[0024] Figure 4 A schematic diagram of the CBAM attention mechanism structure;

[0025] Figure 5 This is a schematic diagram of the Feature Pyramid Enhancement Module (FPEM) structure.

[0026] Figure 6 This is a schematic diagram of the feature fusion module FFM structure;

[0027] Figure 7 This is a schematic diagram of the LSTM model structure;

[0028] Figure 8 This is a schematic diagram of the BLSTM model structure;

[0029] Figure 9 A flowchart of the model training process of the SFE-DBnet network of the present invention;

[0030] Figure 10 This is an example of one of the medical laboratory report dataset images of the present invention;

[0031] Figure 11 A comparison of the actual results detected by the EAST model, the baseline model DBNet network, and the SFE-DBNet network ((a) and (e) are the actual results detected by the EAST model, (b) and (f) are the actual results detected by the baseline model DBNet network, and (c) and (g) are the actual results detected by the SFE-DBNet network). Detailed Implementation

[0032] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto:

[0033] Example 1: A method for text detection in medical laboratory report images based on SFE-DBnet, such as... Figure 1 As shown, the details are as follows:

[0034] 1. Establish the SFE-DBnet network

[0035] The SFE-DBnet network of this invention is built based on the DBnet network, such as... Figure 1As shown, ResNext50, which has strong feature extraction capabilities and low computational cost, is used as the backbone network. A convolutional block attention module (CBAM) is introduced into the backbone network to improve weight distribution, thereby helping the network to focus more effectively on the target's location information and improve detection efficiency and performance. The FPEM_FFM module is used for multi-scale feature extraction and fusion. The FPEM_FFM module includes a Feature Pyramid Enhancement Module (FPEM) and a Feature Fusion Module (FFM). The Feature Pyramid Enhancement Module (FPEM) enhances the backbone's output features. Compared to the ordinary Feature Pyramid structure FPN, the Feature Pyramid Enhancement Module (FPEM) has the advantages of low computational cost and greater representational power. The Feature Fusion Module (FFM) fuses the obtained features at different scales. Then, the extracted feature maps are processed by a Bi-directional Long Short-Term Memory (BLSTM) module for horizontal spatial sequence feature extraction, thus obtaining the final feature map. The introduction of a Feature Fusion Module (FFM) enhances the connection between high-level and low-level semantic features. Further, the introduction of a Bidirectional Long Short-Term Memory (BLSTM) module helps the network learn horizontal text features, effectively addressing the issue of uneven spacing between blank lines in the horizontal direction of medical lab report text. The feature maps processed by the BLSTM module are used to predict a probability map (P) and a threshold map (T). Finally, differentiable binarization of P and T is performed to calculate an approximate binary map (B), ultimately obtaining predicted candidate boxes to detect the text in the medical lab report image. The specific details are as follows:

[0036] 1.1 ResNext50 network with CBAM integration and improvement

[0037] The backbone network is a ResNext50 network that incorporates the attention mechanism module CBAM, such as... Figure 2 As shown, its structure includes a ResNext block and a CBAM attention mechanism. The input feature map passes through a convolutional layer and a max pooling layer, and then through three ResNext blocks for initial feature extraction. The resulting feature map then passes through a CBAM attention mechanism module to obtain a new feature map, which then passes through another ResNext block and finally through a CBAM attention mechanism module to output the feature map.

[0038] ResNext is based on the ResNet network and makes some adjustments to its structure. The main change is that the previous ordinary convolution is replaced with group convolution, which reduces the number of parameters. Figure 3 Here is a structural example of a block in ResNext. Figure 3 (a) is a diagram of the internal structure of the aggregated residual structure, which employs a multi-branch strategy based on cross-layer connections. Figure 3 (b) shows an overall schematic diagram. Each aggregated residual block contains modules with many identical branches. Each branch prunes the input channels and finally merges the results. The number of branches is called cardinality, which is another dimension in the ResNext network besides depth and width. Each branch includes the same number of input channels, convolutional kernel size, and output channels, thus ensuring that the parameter size does not increase despite the large number of branches. Each branch prunes the input channels and finally merges the results.

[0039] CBAM is a simple and effective attention mechanism module for convolutional neural networks that can be seamlessly integrated into any CNN at virtually zero cost and can be trained end-to-end with the CNN. It involves concatenating the channel attention module and the spatial attention module, with the channel attention module placed first, such as... Figure 5 As shown, the attention weights of the feature map are calculated from the above two dimensions, and then multiplied with the original feature map to perform adaptive learning of the features.

[0040] The channel attention module works as follows: After inputting the H×W×C feature map F, it first undergoes max pooling and average pooling to obtain two 1×C channel descriptions. These two descriptions are then input into the same two-layer neural network. Finally, the two obtained features are processed through a Leaky ReLU activation function to obtain the weight coefficients M. c The calculation formula is:

[0041]

[0042] In the formula: σ represents the Leaky_ReLU function; MLP is a multilayer perceptron, Avgpool is average pooling, Maxppol is max pooling, W is the weight, and F is the input feature. The calculated channel attention weight coefficients M c Multiplying the feature F by the input feature yields the scaled new feature:

[0043] The newly generated feature F' serves as the input to the spatial attention module, retaining its size of H×W×C. Within the spatial attention module, average pooling and max pooling are first applied to the input feature map to obtain two H×W×1 channel description maps. Then, a concave attention operation is used to merge their feature channels into a single effective feature layer. Finally, a 7×7 convolutional layer and a LeakyReLU activation function are applied to generate an H×W×1 spatial attention map M. s The specific calculation formula is as follows:

[0044]

[0045] In the formula: and Size is H×W×1; f 7×7 This is represented as a convolution operation with a filter size of 7×7. The M obtained from the formula... s A new feature map can be obtained by multiplying it with the input features.

[0046] 1.2. Establish the FPEM_FFM module

[0047] The FPEM_FFM module is based on FPN. It uses the Feature Pyramid Enhancement Module (FPEM) to fuse low-level and high-level information to enhance the connection between features at different scales. Then, the Feature Fusion Module (FFM) is used to fuse the features generated by the Feature Pyramid Enhancement Module (FPEM) at different depths.

[0048] FPEM such as Figure 6 As shown, it has a U-shaped structure, consisting of an up-scale enhancement stage and a down-scale enhancement stage. The up-scale enhancement stage acts on the input feature pyramid, iteratively enhancing the feature map with strides of 32, 16, 8, and 4 pixels. In the down-scale stage, the input is the feature pyramid generated by the up-scale enhancement, with enhancement strides ranging from 4 to 32. The output feature pyramid of the down-scale enhancement stage is the final output of the FPEM module. FPEM also has two advantages. First, FPEM is a cascaded module; with n_c cascades, feature maps of different scales can be better integrated, increasing the receptive field of the features. Second, FPEM consists of separate convolutions requiring minimal computation, resulting in low computational cost, approximately 1 / 5 that of FPN.

[0049] The Feature Fusion Module (FFM) first combines feature maps of corresponding scales by element-wise addition, then downsamples the feature maps and concatenates them into a final feature map with only 4×128 channels. The FFM module is used to fuse feature pyramids of different depths because both low-level and high-level semantic information are important for semantic segmentation. (The process is repeated in the original text.)

[0050] 1.3 Introduction of Bidirectional Long Short-Term Memory (BLSTM)

[0051] After feature extraction, a bidirectional long short-term memory (BLSTM) module is introduced to enhance the extraction of horizontal spatial sequence features from the horizontal text in the extracted feature map.

[0052] BLSTM is a variant of Long Short-Term Memory (LSTM). The LSTM structure is as follows: Figure 8 As shown, it mainly consists of a three-gate structure (forget gate, input gate, and output gate), corresponding to Figure 7 The forgetting gate formula for the structures within boxes 1, 2, and 3 is:

[0053] f t =σ(W f ·[h t-1 ,x t ]+b f (3)

[0054] Where σ represents the sigmoid function, W is the weight, b is the bias, h is the time series, and h t-1 t-1 is the state parameter value of the sequence number, x is the input. This step mainly acts on the memory cell state c-1 from the previous unit, with the aim of selectively forgetting the information in the memory cell.

[0055] The formula for the input gate is:

[0056] i t =σ(W i ·[h t-1 ,x t ]+b i (4)

[0057]

[0058]

[0059] The function of the input gate is to update the state of the memory cells and then selectively transmit the updated information to the next level.

[0060] The formula for the output gate is:

[0061] o t =σ(W o ·[h t-1 ,x t ]+b o (7)

[0062] h t =o t *tanh(C t (8)

[0063] The output gate acts on both the input and the hidden layer output. After passing through the output gate, the final output includes both the cell state and the input, and the result is then passed to the next layer.

[0064] BLSTM neural network structure model as follows Figure 9 As shown, the algorithm consists of two independent LSTMs. The input sequence is fed into the two LSTM neural networks in both forward and reverse order for feature extraction. The word vector formed by concatenating the two output vectors (i.e., the extracted feature vectors) is used as the final feature representation of the word. The design philosophy of BLSTM is to ensure that the feature data obtained at time t contains information from both the past and the future. This neural network structure model has better efficiency and performance in extracting text sequence features than a single LSTM structure model.

[0065] Then, the probability map (P) and threshold map (T) are predicted using the feature map. Finally, differentiable binarization of P and T is used to calculate the approximate binary map (B), and the predicted candidate boxes are finally obtained.

[0066] 2. Training and testing of the SFE-DBnet network

[0067] 2.1 Construction of Training and Test Sets

[0068] Because medical test reports involve personal privacy and are difficult to collect and create, there are currently no publicly available medical test reports. Therefore, with the consent of hospitals and patients, we collected hundreds of medical test reports and used them to create a dataset of medical test report images and text. An example of one of the dataset's images is shown below. Figure 11As shown, all images in the dataset were taken with a mobile phone and saved in JPG format. The images included different lighting conditions (e.g., sunlight, cloudy days, or indoor lighting) and shooting angles to simulate the storage conditions of real-life medical lab reports. The images were then labeled using the labelimg tool, and the label format was finally converted to ICDA format. The dataset was then divided into training and testing sets, with 303 samples in the training set and 163 samples in the testing set.

[0069] 2.2 Training and Testing Process

[0070] The labeled images are input into the SFE-DBnet network for training, such as... Figure 9 As shown, features are first extracted using the ResNext50 backbone network, which combines the CBAM attention mechanism and aggregated residual blocks. Then, the FPEM_FFM module is used for multi-scale feature extraction and fusion. The extracted feature maps are then processed by a BLSTM network for horizontal spatial sequence feature extraction, resulting in the final feature map. This feature map is then used to predict a probability map (P) and a threshold map (T). Finally, P and T are differentiable and binarized to calculate an approximate binary map (B), ultimately yielding the predicted candidate boxes. The loss function used is:

[0071] L = L S +α×L b +γ×L t (9)

[0072] Among them, L S For the loss of the probabilistic graph, L b It is the loss of the binary image, L t This represents the loss of the threshold plot, with α and γ set to 1 and 10, respectively.

[0073] The training parameters are preset with a learning rate of 0.01, num_work set to 2, batch_size set to 1, and epoch set to 500. During training, the loss function value is calculated and the model parameters are iteratively optimized through backpropagation. Training ends when the 500th epoch is completed.

[0074] The testing process involves feeding the labeled test set into the trained model to form candidate boxes. A positive sample is defined as an Intersection over Union (IoU) greater than 0.7, and a negative sample is defined as an IoU less than 0.7. Precision, Recall, and F-score, commonly used in text detection algorithms, are employed as evaluation metrics. Their formulas are shown in equations (10) to (12), respectively.

[0075]

[0076]

[0077]

[0078] In the above formula, TP represents the number of correctly predicted positive samples, FP represents the number of positive samples incorrectly predicted as negative samples, TN represents the number of correctly predicted negative samples, and TF represents the number of negative samples incorrectly predicted as positive samples. Precision is the proportion of correctly detected text regions, recall is the proportion of detected regions out of all labeled text regions, and the harmonic mean is used to comprehensively consider and evaluate the performance of the detection algorithm, avoiding the inability to effectively evaluate based on only one criterion when precision and recall conflict. After testing, the SFE-DBnet network achieved a recall of 90.1%, a precision of 96.4%, and an F-score of 93.1%, significantly outperforming similar algorithms. An online-usable SFE-DBnet network was obtained through training and testing.

[0079] 3. Online use

[0080] The image of the medical test report to be identified is acquired and input into the online SFE-DBnet network obtained in step 2. Features are extracted through the ResNext50 backbone network, which combines the CBAM attention mechanism and aggregated residual blocks. Then, multi-scale feature extraction and fusion are performed through the FPEM_FFM module, followed by horizontal spatial sequence feature extraction through the BLSTM network to obtain a feature map. The feature map is then used to predict a probability map (P) and a threshold map (T). Finally, differentiable binarization of the probability map P and the threshold map T is performed to calculate an approximate binary map (B), ultimately obtaining predicted candidate boxes. This allows for the detection and recognition of text such as serial number, item name, English name, measured value, symbol, unit, reference range, and test method in the test report. The output results are as follows: Figure 11 As shown in (c) and (g).

[0081] By adding the CBAM (Convolutional Block Attention Module) attention mechanism, ResNext50, FPEM_FFM, and BLSTM modules to the DBnet network, the SFE-DBnet network was obtained, which improved its detection capability for medical test reports. First, addressing the challenge of complex lighting conditions in medical test report images that make text region feature recognition difficult, this invention uses ResNext50, which has strong feature extraction capabilities and low computational cost, as the backbone network and introduces the CBAM (Convolutional Block Attention Module) attention mechanism to improve weight distribution. This helps the network focus more effectively on the target's location information, enhancing robustness and improving detection efficiency and effectiveness. Second, addressing the issue of diverse character types and varying font sizes, the Head part uses the FPEM_FFM module. FPEM has a cascaded structure and low computational cost, allowing features of different sizes to be deeper and more representative. The introduction of FFM enhances the connection between high-level and low-level semantic features. Finally, a Bi-directional Long Short-Term Memory (BLSTM) module is introduced. This module helps the network learn horizontal text features, effectively solving the problem of uneven spacing of blank lines in the horizontal direction of medical test report text.

[0082] experiment

[0083] Ablation and comparison experiments were conducted on the SFE-DBnet network of this invention to verify the feasibility and effectiveness of the scheme. The experimental data were the training set and test set data established in step 2.1 of Example 1. The experimental method involved importing the training set into the networks used in the ablation and comparison experiments for training, and then importing the trained network into the test set for prediction. The prediction results were statistically analyzed for Precision, Recall, and F-score, and the results were compared as follows:

[0084] (1) Ablation test

[0085] Based on the DBNet network, DBNet0, DBNet1, and DBNet2 were created by correspondingly changing the module types of the backbone network or neck part of the DBNet network. Together with the SFE-DBnet network of this invention, they served as the comparison network for ablation experiments. Table 1 shows the comparison results of the ablation experiments. Comparing the results of DBNet0 and DBNet1, we can see that after changing the backbone network, precision improved by 2.2%, F-score improved by 0.7%, and although recall decreased slightly, the overall detection performance was better than the original backbone network. Comparing DBNet0 and DBNet2, replacing the neck part with the FPEM_FFM module improved precision by 1.7%, recall by 3.4%, and F-score by 2.6%, further confirming that the FPEM_FFM module has stronger multi-scale feature extraction and fusion capabilities than FPN on medical test reports, and can detect text regions more completely and accurately. Comparing DBNet1, DBNet2, and SFE-DBNet, we can see that the introduction of BLSTM further improved all three metrics, proving the effectiveness of BLSTM's semantic sequence feature extraction in the task of detecting text on medical test reports. Therefore, the three improvements complement each other, effectively improving the overall performance of the algorithm. The SFE-DBNet proposed in this invention shows significant improvements in various aspects of the model's performance when applied to medical laboratory report text detection, demonstrating the feasibility of the improved method presented in this paper.

[0086] Table 1. Experimental results of SFE-DBNet ablation

[0087]

[0088] (2) Comparative Experiment

[0089] The CTPN text detection algorithm, SSD algorithm, EAST algorithm, and DBNet text detection network were used as comparison networks in the experiments. The CTPN text detection model, proposed by TianZhi et al., borrows ideas from Faster R-CNN, designs a text line construction algorithm, proposes a vertical anchor method to generate predicted candidate boxes in the horizontal direction, and uses BLSTM to strengthen the semantic connections in the horizontal direction, achieving superior performance in detecting text in horizontal scenes. The EAST algorithm (Efficient and Accuracy Scene Text), proposed by Zhou et al., builds upon CTPN. It adopts the approach of segmenting, detecting, and then merging complete text lines, directly generating text regions using FCN, eliminating redundant and time-consuming intermediate steps. Locality-Aware NMS is then used to filter the generated geometry, effectively improving detection accuracy and speed. The SSD algorithm, proposed by Wei Liu at ECCV in 2016, borrows the anchor box idea from Faster R-CNN and uses multi-scale feature map detection. The text detection network DBNet, designed by Liao et al., proposed a differentiable binarization (DB) method. DB implements binarization processing in segmentation-based text detection networks. Segmentation networks based on the DB module can adaptively set the binarization threshold, simplifying post-processing and improving text detection performance.

[0090] The SFE-DBNet model of this invention was compared with the CTTN text detection model, the SSD algorithm, and the EAST algorithm using the same test samples. Table 2 shows the comparative test results of this invention with the CTTN, SSD, and EAST text detection algorithms under the same test set. The comparison results show that the SFE-DBNet model proposed in this paper has significant advantages over similar models in text detection of medical laboratory report images, with substantial improvements in all three evaluation indicators. It exhibits good detection performance and strong competitiveness.

[0091] Table 2. Comparison results with mainstream text detection algorithms

[0092]

[0093] (3) Comparison of actual test results

[0094] To more intuitively demonstrate the actual detection performance of the SFE-DBNet algorithm in this invention compared to other algorithms, Figure 11Examples of the best detection results using the EAST model, the baseline model (i.e., the text detection network DBNet), and the SFE-DBNet model of this invention are presented on two test reports. Figure 11 It can be seen that EAST has a higher false negative rate (e.g., Figure 11 (e) shows that there is text that is not fully framed (such as...). Figure 11 (a) and (e) are examples of cases where the baseline model DBNet has a significant number of missed detections in the detection of medical test report text (e.g., Figure 11 (b) and more text box distortion and text box overlap (e.g.) Figure 11 (f) In this case, the SFE-DBNet of the present invention performs better than the baseline model and EAST in detecting text regions (such as...) on medical test reports, and can detect text regions more completely and accurately. Figure 11 (c)(g) is shown).

[0095] Finally, it should be noted that the above examples are merely some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for text detection in medical laboratory report images based on SFE-DBnet, characterized in that... The specific process is as follows: acquire the image of the medical test report and input it into the SFE-DBnet network in the computer, and finally obtain the prediction candidate box to mark the text in the test report; The SFE-DBnet network is built on the DBnet network. It includes extracting features through a ResNext50 backbone network that combines the attention mechanism module CBAM and aggregated residual blocks, then performing multi-scale feature extraction and fusion through the FPEM_FFM module, and finally extracting horizontal spatial sequence features through the BLSTM network to obtain feature maps. The attention mechanism module CBAM takes the H×W×C feature map F as input, performs max pooling and average pooling to obtain two 1×C channel descriptions, and then inputs them separately into the same two-layer neural network. Finally, the two features are processed by a Leaky ReLU activation function to obtain the weight coefficients. The calculation formula is: （1） In equation (1): σ represents the Leaky_ReLU function; MLP is a multilayer perceptron, Avgpool is average pooling, Maxppol is max pooling, W is the weight, and F is the input feature; Calculated channel attention weight coefficients Multiplying the input feature F by the scaled feature yields the new, scaled feature: ; New features generated As input to the spatial attention module, its size remains H×W×C. Within the spatial attention module, average pooling and max pooling are first performed on the input feature map to obtain two H×W×1 channel description maps. Then, a concave attention operation is used to merge their feature channels into a single effective feature layer. Finally, a 7×7 convolutional layer and a LeakyReLU activation function are used to generate an H×W×1 spatial attention map. The calculation formula is: （2） In formula (2): and The size is H×W×1; This is represented as a convolution operation with a filter size of 7×7; the result obtained in equation (2) Multiplying the feature map by the input feature map yields a new feature map. The backbone network comprises, in sequence, convolutional layers, max pooling layers, three aggregated residual blocks (ResNext blocks), an attention mechanism module (CBAM), an aggregated residual block (ResNext block), and an attention mechanism module (CBAM); The FPEM_FFM module serves as the neck part. It includes a Feature Pyramid Enhancement Module (FPEM) and a Feature Fusion Module (FFM), with the output of the FPEM serving as the input to the FFM. The training and testing process of the SFE-DBnet network is as follows: Images from laboratory reports are collected, and training and testing sets are established. Training parameters are set, including learning rate, num_work, batch_size, and epoch. The training set is input into the SFE-DBnet network. During training, the loss function value is calculated, and backpropagation is used to iteratively optimize the model parameters until the preset epoch is reached, at which point training ends. Then, the test set is input into the trained model to obtain the intersection-over-union ratio (IoU). Precision, recall, and harmonic mean are then used as evaluation metrics to obtain an online-usable SFE-DBnet network. The loss function used is: (9)； Among them, L s For the loss of the probabilistic graph, L b It is the loss of the binary image, L t This is the loss of the threshold map, with α and β set to 1 and 10, respectively.

2. The method for detecting text in medical laboratory report images based on SFE-DBnet according to claim 1, characterized in that: The training set and test set are obtained by collecting medical test reports, taking pictures and saving them as JPG images, including images with different lighting conditions and shooting angles, and then labeling them using the labelimg tool.

Images