Lightweight OCR method based on medical semantic enhancement

Abstract

The invention provides a medical scene optimization-oriented OCR neural network based on a lightweight CRNN architecture, and aims to significantly improve the medical text recognition precision while reducing the calculation overhead. The core innovation of the method is that on one hand, a lightweight convolution feature extraction module is introduced, and the model scale and reasoning energy consumption are effectively compressed; on the other hand, a dictionary constraint mechanism in the medical field is fused, medical terms and semantic structure priori knowledge are utilized, and the recognition capacity for professional symbols, abbreviations and context sensitive content is enhanced. According to the architecture, high accuracy is kept, meanwhile, the requirement for computing power is remarkably reduced, efficient deployment can be achieved on medical edge devices (such as portable terminals and basic diagnosis and treatment devices) with limited resources, feasible technical support is provided for large-scale application of medical OCR in scenes such as mobile medical treatment, basic health services and intelligent diagnosis and treatment terminals, and the application prospect is wide. Important clinical and industrialization values are realized.

Description

TECHNICAL FIELD

[0001] The present application designs the application of Artificial Intelligence (AI), Deep Learning (DL), Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM) in Optical Character Recognition (OCR). BACKGROUND

[0002] Since the Optical Character Recognition (OCR) technology was proposed in the 1950s, it was initially applied to the recognition of postal codes and the automatic processing of bank checks. With the rapid development of computer vision, pattern recognition and deep learning technologies, the OCR technology has gradually developed from a system that can only recognize fixed fonts and structured forms in the early stage to a general intelligent recognition engine that can process multiple languages, multiple fonts, complex layouts and even handwritten texts. For example, using the method of combining convolutional neural networks with sequence modeling (such as connection time classification or attention mechanism), the identity card recognition and license plate recognition have achieved relatively mature applications in general scenarios. In recent years, multi-modal large models (MLLMs) have fused visual and language features through neural networks with a large number of parameters, so that OCR can not only recognize text, but also understand semantics and layout structure. Multi-modal OCR can simultaneously complete text detection, recognition and semantic analysis.

[0003] In the field of medical health, OCR technology has important potential application value and can be widely used in automatic entry of electronic medical records, recognition of prescription information, structured analysis of test reports, extraction of pathological section labels and intelligent processing of medical insurance documents. Through OCR technology, massive paper or imaged medical documents can be quickly converted into structured, searchable, analyzable and integrable digital data, thereby significantly improving the level of medical informatization, reducing the cost of manual entry, and providing high-quality data basis for clinical decision support systems, real-world research and artificial intelligence assisted diagnosis.

[0004] However, the practical application of existing OCR technology in medical scenarios still has the following problems:

[0005] Medical semantic understanding is insufficient. Medical text has high professionalism, contains a large number of non-standardized abbreviations (such as 'DM' representing diabetes mellitus and 'QD' representing once a day), complex medical terms (such as 'amlodipine' and 'troponin I'), and semantically associated expressions (such as 'blood pressure 130 / 80 mmHg' which needs to identify the value and its unit). The existing OCR system lacks the ability to model the context in the medical field, and often has problems such as misidentification of terms, missing units or semantic breakage, which makes it difficult to meet clinical needs.

[0006] Model lightening and deployment difficulty. Large models with large parameter quantities consume large computing resources and are difficult to deploy in medical edge devices (such as mobile ward rounds, primary clinic terminals, and portable scanners) or embedded systems. Therefore, there are higher requirements for the lightening design of the model, the inference acceleration ability and the cross-platform adaptation, because the medical scene generally requires real-time response, low-power operation and offline use.

[0007] In summary, the application of existing OCR technology in the medical field has the technical bottlenecks of insufficient professional semantic recognition ability and difficulty in lightening deployment, and there is an urgent need to propose a medical OCR method that takes into account the recognition accuracy and deployment feasibility.

[0008] Based on the above technical background, the present application proposes an OCR neural network architecture based on the CRNN architecture and optimized for medical scenarios. The core innovation is to fuse a light-weight convolutional feature extraction module with a medical field dictionary constraint mechanism, which significantly reduces the computational load while greatly improving the recognition accuracy of medical symbols, terms and semantic structures. Experimental results show that this method can still maintain excellent performance on medical edge devices with limited computing power, providing a practical technical path for the large-scale landing of medical OCR in primary medical care, mobile diagnosis and treatment, and intelligent terminals. SUMMARY

[0009] The purpose of the present application is to address the problems of insufficient semantic understanding and difficulty in lightening deployment of existing medical OCR technology, and to propose an OCR neural network architecture and method optimized for medical scenarios. The present application can effectively improve the recognition accuracy of medical professional terms, symbols and semantic structures while reducing the consumption of computing resources, thereby significantly improving the efficiency and reliability of medical document digitization and intelligent processing.

[0010] Currently, mainstream large models and Transformer architectures have strong modeling capabilities in character recognition tasks, but their parameter sizes are large, and both training and inference require high computing power, resulting in high running costs and poor device adaptability. In contrast, traditional CRNN architectures have relatively fewer parameters, but still have certain deployment difficulties in computing power limited environments, and perform poorly in medical semantic recognition.

[0011] To solve the above problems, the application adopts the following technical solutions:

[0012] Lightweight convolutional feature extraction module. By designing an efficient convolutional neural network structure, combining depth separable convolution and optimizing the activation function, while ensuring the feature expression ability, the network parameter quantity and the calculation overhead are significantly reduced, thereby realizing efficient feature extraction of the medical image text region. At the same time, the SE attention (Squeeze-and-Excitation attention) mechanism is introduced, which greatly improves the feature representation ability and recognition accuracy with only a small amount of calculation.

[0013] Medical field dictionary constraint mechanism. In the character sequence decoding process, a medical special dictionary is introduced to constrain and correct the OCR output result based on CTC (Connectionist Temporal Classification) decoding, which can effectively avoid professional term misrecognition, abbreviation ambiguity and unit omission, etc., thereby improving the recognition accuracy of medical abbreviations, drug names, disease names and numerical units.

[0014] The beneficial effects of the application are that by combining lightweight convolutional feature extraction and medical dictionary constraint mechanism, the deficiencies of existing medical OCR in semantic recognition and model deployment are effectively solved. Experimental results show that the method of the application can still maintain high recognition accuracy and running efficiency on medical edge devices with limited computing power, has good cross-platform adaptability and real-time performance, and can be widely applied to electronic medical record input, prescription recognition, test report analysis, pathological section label extraction and other medical scenarios, providing a feasible technical path for the large-scale application of medical OCR in primary medical care, mobile diagnosis and treatment and intelligent terminals.

[0015] In summary, the main innovations of the application include:

[0016] (1) A lightweight convolutional feature extraction network is proposed. The network uses depth separable convolution to reduce the parameter quantity of convolution operation, combines an optimized activation function to improve the calculation efficiency, and introduces an SE attention mechanism in the last convolution layer to achieve significant feature expression enhancement and recognition accuracy improvement with small calculation overhead.

[0017] (2) A medical special dictionary is constructed and integrated into the CTC decoding process. By semantically constraining the decoding results, the recognition accuracy of key terms such as disease names, drug names and medical numerical units is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The overall flowchart of the application is shown

[0019] Figure 2A structure diagram of a deep separable convolutional layer used in the present application is shown

[0020] Figure 3 A structure diagram of a CNN convolutional neural network for feature extraction is shown

[0021] Figure 4 A structure diagram of a complete neural network of the present application is shown DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described in detail below in combination with the drawings and specific examples.

[0023] As Figure 1 shown, the present application mainly has six modules.

[0024] First, construction of a medical dictionary

[0025] The present application constructs a medical-scene-oriented dictionary for subsequent constraint and correction of OCR decoding in view of the characteristics of professional terms in medical documents and image data. The specific implementation process is as follows.

[0026] First, medical-related corpus is collected, including but not limited to clinical medical records, medical order texts, imaging reports, pharmacopoeias and guidelines, etc.

[0027] Then, the text is segmented and term extracted through a natural language processing tool, and disease names, drug names, test indicators, unit symbols, medical abbreviations, etc. are extracted. The extracted terms are uniformly standardized, for example: uniform case (such as all converted to lowercase or the first letter capitalized); symbol standardization (such as μ and mu, II and 2, etc.); removing extra spaces, punctuation marks; generating a mapping relationship for common abbreviations or synonyms for uniform output during decoding.

[0028] Finally, the dictionary is constructed into a hash table data structure for efficient query, facilitating subsequent fast constraint search in the CTC decoding stage. The dictionary allows dynamic expansion and update, and as new medical literature or clinical data is added, the dictionary can be updated to ensure coverage of the latest professional terms. The newly added terms are standardized as described above, and the hash table is updated.

[0029] The constructed medical dictionary can be directly used for constraint search and post-processing correction in the OCR decoding stage, ensuring that the recognition result conforms to the medical context, reducing recognition errors of professional terms, and improving the overall recognition accuracy.

[0030] Second, image preprocessing

[0031] First, collect the dataset containing medical report sheets, drug lists, and uniformly crop these images to 512x512 size. Then, in a random arrangement, according to the 4:1 ratio, divide the dataset into training and test sets to ensure randomness and rationality of data division.

[0032] During training and testing, for the input medical image, first perform grayscale and normalization, and use the GaussianBlur function of the OpenCV library for denoising.

[0033] Third, text detection (EAST network)

[0034] The open source network EAST convolutional neural network is used to detect candidate text regions in the input image, and the output is a set of candidate boxes. According to the coordinates output by EAST, the text region is cropped using the OpenCV library in python, and the cropped image is scaled to a fixed size of 64x256.

[0035] Fourth, text recognition

[0036] The text recognition module is divided into the following two parts.

[0037] (1) Feature extraction part

[0038] The cropped image is input into the lightweight CNN convolutional feature extraction network to obtain a two-dimensional feature map:

[0039]

[0040] where I n is the normalized input medical image obtained after the previous steps (grayscale, normalization). I n |B represents the input image cropped to the candidate box region output by the text detection module EAST

[0041] f CNN represents the feature extraction function, and F represents the output two-dimensional feature.

[0042] As Figure 2 shown, the convolutional layer uses depth separable convolution, which can greatly reduce the parameters used for convolution.

[0043] The depth separable convolution can be represented as a whole:

[0044]

[0045] where, represents the depth convolution, and represents the pointwise convolution (i.e., 1x1 convolution).

[0046] The convolutional layer uses an optimized activation function: the h-swish function.

[0047] The formula is as follows:

[0048]

[0049] Where ReLU6(x) = min(max(0, x), 6).

[0050] In order to improve the feature expression enhancement and recognition accuracy, an SE attention mechanism is introduced in the last convolutional layer.

[0051] First, the feature map of each channel is subjected to GAP (Global Average Pooling) to obtain a channel descriptor.

[0052] For the input feature map The statistical quantity Z of the cth channel c is:

[0053]

[0054] Then, a lightweight two-layer fully connected network is used to sequentially reduce and increase the dimension, learn the nonlinear relationship between channels, and output the weight of each channel.

[0055] s = σ(W2·δ(W1·z))

[0056] Where z is the output after GAP; W1 represents a reduction matrix, r is a reduction ratio, and in the present application, r = 4; δ represents a ReLU activation function; W2 represents an increase matrix; and σ represents a Sigmoid activation function. The final channel attention weight vector is represented by s = {s1, s2, … sn}.

[0057] ReLU(x) = max(0, x)

[0058]

[0059] Finally, the learned channel weight s = {s1, s2, … sn} is multiplied to the original feature map U channel by channel c

[0060]

[0061] The complete CNN convolutional neural network structure diagram is shown in Figure 3 .

[0062] (2) Sequence modeling

[0063] ​The two-dimensional convolution feature map is flattened into a one-dimensional sequence.

[0064]

[0065] The flattened sequence is input into a bidirectional LSTM network:

[0066]

[0067] Fifth, CTC decoding

[0068] The context feature sequence {h t} output by the BiLSTM is input into a fully connected layer to map to a character class space:

[0069]

[0070] The final character sequence is obtained by using CTC (Connectionist Temporal Classification) decoding. CTC allows repeated characters and introduces a special blank symbol (denoted as "-") during training, and the probability sum of all possible alignment paths is calculated to perform end-to-end training.

[0071]

[0072] where X is the input sequence, Y is the target label sequence

[0073] π is the path sequence, is a compression mapping function, and T is the input sequence length (number of time steps).

[0074] At time step t, the model outputs the probability of path character π t .

[0075] All path sets after compression are equal to the target label sequence Y.

[0076] To adapt to the medical scene, the application introduces a hierarchical medical dictionary constraint mechanism in the CTC decoding process. Specifically, during the generation and selection of candidate paths, the output confidence is divided into two regions: low confidence region: when the probability of the candidate path is low, only the results that completely match the medical dictionary are retained to avoid serious misrecognition caused by low-quality prediction; high confidence region: when the probability of a candidate path is significantly higher than all candidates in the dictionary (more than threshold τ = 0.8), even if the result is not in the dictionary, it is also retained, thereby ensuring the open adaptability of the method to new terms, rare abbreviations, or uncollected drug names.

[0077] The confidence of the final output sequence s is defined as:

[0078]

[0079] where S denotes the set of all candidate sequences, and P(s|x) is the sum of all path probabilities that map to s.

[0080] This formula measures the normalized confidence of the decoding result relative to all candidates.

[0081] When the output confidence is lower than 0.8, the constraint formula is

[0082]

[0083] Sixth, the output

[0084] Through the CTC decoding under the above dictionary constraint, the text recognition result conforming to the medical semantic specification is finally obtained, and accurate recognition and structured extraction of medical documents such as prescription information, test indicators, medical records, etc. are realized.

Claims

1. A lightweight OCR (Optical Character Recognition) method based on medical semantic enhancement, characterized in that, Includes the following steps: Medical dictionary construction: Collect medical corpora, including clinical medical records, test reports, pharmacopoeia and guideline texts, perform word segmentation and term extraction on the corpora, and generate a medical dictionary containing disease names, drug names, test indicators, unit symbols and abbreviations; Image preprocessing: Perform grayscale, normalization and Gaussian denoising on the input medical document images, and crop them to a preset size; Text detection: Utilizes the EAST convolutional neural network to detect text regions in an image, outputs a set of candidate boxes, and crops the text region image based on the coordinates of the candidate boxes; Feature extraction: The cropped text image is input into a lightweight convolutional neural network, and a two-dimensional feature map is extracted through depthwise separable convolution and h-swish activation function. The Squeeze-and-Excitation (SE) attention mechanism is introduced in the last convolutional layer to enhance the channel feature representation. Sequence modeling: Flatten the two-dimensional feature map into a one-dimensional sequence and input it into a bidirectional long short-term memory network (BiLSTM) to obtain context-related features; CTC Decoding and Semantic Constraints: The BiLSTM output is decoded based on the Connectionist Temporal Classification (CTC) algorithm, and semantic constraints and result corrections are performed in conjunction with the medical dictionary to output character recognition results that conform to medical semantics.

2. The method according to claim 1, characterized in that, The medical dictionary construction process includes standardizing entries, including: unifying capitalization, standardizing symbols, establishing a mapping relationship between synonyms and abbreviations, and storing entries based on a hash structure to support fast lookups.

3. The method according to claim 1, characterized in that, Feature extraction employs depthwise separable convolution, which consists of two parts: depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, while pointwise convolution is used to fuse channel information, thereby significantly reducing the number of model parameters.

4. The method according to claim 1, characterized in that, The SE attention mechanism includes: performing global average pooling on each channel to obtain channel descriptors, performing dimensionality reduction and dimensionality increase through a two-layer fully connected network, generating channel weights through the Sigmoid function and multiplying them channel by channel with the original feature map to achieve feature enhancement.

5. The method according to claim 1, characterized in that, The activation function is the h-swish function, and its expression is: ReLU6(x)=min(max(0,x),6) Used to enhance nonlinear expressive power in lightweight networks.

6. The method according to claim 1, characterized in that, The semantic constraints of the CTC decoding adopt a hierarchical medical dictionary constraint mechanism. The process includes: during the CTC decoding process, the candidate paths are divided into high-confidence regions and low-confidence regions based on their output confidence. In the low-confidence regions, only candidate paths that completely match the entries in the medical dictionary are retained to avoid serious misidentification caused by low-quality predictions. In the high-confidence regions, when the probability of a candidate path is significantly higher than that of all candidates in the medical dictionary (exceeding the threshold τ = 0.8), it is retained even if the candidate result is not included in the medical dictionary, thereby improving the ability to identify and adapt to new terms, rare abbreviations, and drug names not included in the dictionary.

7. The method according to claim 1, characterized in that, The BiLSTM network has 256 hidden layer nodes to balance computational efficiency and semantic modeling capability.

Images