Method, system and computer readable medium for extracting and classifying indicators of inspection report based on OCR recognition

Abstract

The application provides an OCR recognition-based test report index extraction and classification method, system and computer readable medium, which comprises the following steps: obtaining an image of a test report and pre-processing the image to obtain a feature image; identifying the feature image based on an identification model constructed based on a deep convolutional neural network to generate identification data; calculating the edit distance between the index name of the test report and the index name in the dictionary library by using a first edit distance algorithm; selecting the index name in the dictionary library with the minimum edit distance and taking the index name as a correction word to correct and match the index name of the test report; if the matching is successful, extracting the ID, numerical range, unit and type of the index in the dictionary library of the correction word, replacing the information of the index name of the test report, combining data post-processing, and generating text data of the classified test index. The application solves the problem of inaccurate extraction and low efficiency of the index data of the current medical institutions due to complexity.

Description

Technical Field

[0001] This invention relates to the field of text recognition and data sharing technology, and more specifically to a method and system for extracting and classifying indicators from inspection reports based on OCR recognition. Background Technology

[0002] In medical laboratory technology, laboratory reports are an important source of data, recording various test results and indicators. Currently, most laboratory reports exist in paper or image form, such as blood test, urine test, biochemical test, and pathology reports. The clinical data contained in these reports is crucial for the diagnosis and treatment of diseases. However, manually inputting and processing this data is very time-consuming and error-prone. The formats, structures, and names of laboratory indicators archived by different medical institutions and hospitals also vary, which increases the complexity of indicator data extraction and classification, making it very difficult to accurately extract and classify indicators.

[0003] CN114417798A discloses an OCR document extraction method that extracts text data through OCR recognition technology, performs anomaly detection on the OCR recognition results based on data feature rules, and corrects the detection results to make the data extraction results more accurate. It aims to solve the common extraction problem of documents of different formats. However, when processing inspection reports containing complex structures and scenarios such as tables, graphics, and mobile phone screenshots, it cannot accurately extract and classify inspection indicators, and the recognition accuracy and efficiency are not ideal.

[0004] Currently, OCR (Optical Character Recognition) technology can only process reports in specific formats. For other formats, the recognition effect is poor. Even though OCR technology can convert reports into text, accurately extracting various test indicators from the text and classifying them remains a challenge. Furthermore, after OCR recognition, existing technologies still require manual data cleaning, formatting, and error correction, which not only increases workload and time costs but also introduces errors that reduce recognition accuracy. Consequently, it cannot provide convenient, reliable, accurate, and efficient services for medical testing technologies. Summary of the Invention

[0005] According to a first aspect of the present invention, a method for extracting and classifying indicators from test reports based on OCR recognition is provided, comprising:

[0006] Step 1: Acquire the image of the test report and preprocess it to obtain the feature image;

[0007] Step 2: The recognition model built based on the deep convolutional neural network recognizes the feature image and generates recognition data. The recognition data includes at least the name, ID, numerical range, unit and type of the test report indicator.

[0008] Step 3: Calculate the edit distance between the indicator names on the inspection report and the indicator names in the dictionary using the first edit distance algorithm;

[0009] Step 4: Select the indicator name from the dictionary containing the minimum edit distance, and use it as the correction word to perform error correction matching on the indicator name of the test report:

[0010] a. If the match is successful, extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining the data post-processing.

[0011] b. If the match fails, the cosine similarity algorithm is invoked to calculate the cosine value between the test report indicator name and the error correction word. The cosine value determines the cosine similarity based on its range, which is at least [-1, 1], where:

[0012] b1. If 0 < cosine value ≤ 1, the higher the similarity, the higher the similarity. Extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate text data of the classified test indicators by combining data post-processing.

[0013] b2. If -1 ≤ cosine value < 0, then the lower the similarity, the more likely the error correction words will be ignored.

[0014] Further, in step 1, obtaining the feature image includes:

[0015] Salt-and-pepper noise and Gaussian noise in the image of the test report were removed using median filtering, Gaussian filtering, and bilateral filtering, respectively.

[0016] The image is denoised by grayscale conversion, and the edges of objects in the denoised image are identified by pixel edge detection.

[0017] Based on the object's edge, a convolutional neural network algorithm is used to calculate the ratio of the number of pixels in the region formed by adjacent pixels in the image to the area of ​​the image's bounding box, thus obtaining the pixel density.

[0018] The pixel density is integrated to generate the feature image.

[0019] Further, in step 2, generating the identification data includes:

[0020] A recognition model is built based on a deep convolutional neural network;

[0021] The training samples are input into the recognition model in the form of text sequences for deep learning training;

[0022] Training ends when the character sequence of the training sample is output.

[0023] The feature image is substituted into the trained recognition model for recognition and detection, and the output tensor of shape (batchsize, num classes) is the recognition data.

[0024] Where batch size represents the batch size of the input feature images, and num classes represents the number of output classes.

[0025] Furthermore, the deep learning training includes:

[0026] Collect structured data of medical test report text and OCR image, and preprocess them using the preprocessing method implemented in step 1 to obtain training samples;

[0027] Initialize the word segmenter and build the recognition model;

[0028] The training text is converted into a text sequence and input into the recognition model for training.

[0029] Training stops after N iterations.

[0030] Save the trained model and use it as the recognition model after deep learning training is complete.

[0031] Furthermore, in step 3, the calculation of the first edit distance algorithm is based on the number of character-level operations. The number of character-level operations includes at least the number of insertions, deletions, and replacements. The fewer the number of operations, the smaller the edit distance.

[0032] Further, in step 4, the cosine similarity algorithm is invoked to calculate the cosine value between the test report indicator name and the error correction word, including:

[0033] The test report indicator names and the error correction words are subjected to word embedding processing and converted into vector A and vector B, respectively;

[0034] Calculate the dot product and length of vector A and vector B respectively. The cosine value is obtained from the ratio of the dot product to the length, and its mathematical expression is as follows:

[0035] Vector A·Vector B = A1B1 + A2B2 + ... + An*Bn

[0036]

[0037]

[0038]

[0039] Where vector A·vector B represents the dot product of vectors A and B, ||vector A|| represents the length of vector A, ||vector B|| represents the length of vector B, which is the square root of the sum of the squares of each element, An represents the nth element of vector A, Bn represents the nth element of vector B, and n is a constant.

[0040] Furthermore, the data post-processing involves cleaning and standardizing the identified data to generate structured data, namely, text data of the classified test indicators.

[0041] According to a second aspect of the present invention, a system for extracting and classifying indicators from inspection reports based on OCR recognition is provided, comprising a data acquisition module, a preprocessing module, a recognition and detection model, an error correction module, a data processing module, a classification module, and a data transmission module; wherein:

[0042] The data acquisition module is used to collect, capture, and photograph image data from medical test reports;

[0043] The preprocessing module is used to perform denoising, grayscale conversion, binarization, edge detection, and feature extraction on the images acquired by the data acquisition module;

[0044] A recognition and detection model is used to identify indicators in preprocessed images;

[0045] The error correction module is used to correct and match the identified data;

[0046] The data processing module is used to clean and format the identification data to generate structured data.

[0047] The classification module is used to classify the identified indicators and test report types;

[0048] The data transmission module is used to provide data transmission services for the above modules.

[0049] In a third aspect of the present invention, a computer system is also provided, comprising:

[0050] One or more processors;

[0051] The memory stores operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the aforementioned process of performing the OCR-based inspection report indicator extraction and classification method.

[0052] In a fourth aspect of the present invention, a computer-readable medium for storing software is also provided, the software including instructions executable by one or more computers, the instructions, when executed by the one or more computers, performing the aforementioned process of the OCR-based inspection report indicator extraction and classification method.

[0053] Compared with existing technologies, the OCR-based method for extracting and classifying test report indicators proposed in this invention has the following advantages:

[0054] 1. This invention performs single-structure analysis on basic OCR images and preprocessing, establishes a deep learning model for text recognition and processing sequence data for data cleaning and formatting, extracts and returns test indicators and corresponding results, and identifies and classifies various indicators through matching degree, achieving good practicality and versatility, reducing manual intervention, improving data processing efficiency, providing important data support for medical decision-making and scientific research analysis, and helping users make more accurate judgments and decisions.

[0055] 2. This invention is applied to processing test reports of various scenarios, formats and structures, accurately extracting various test indicator data, and effectively solving the problems of inaccurate extraction and low efficiency caused by the complexity of the indicator data archived by medical institutions.

[0056] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail below may be considered part of the inventive subject matter of this disclosure, provided that such concepts do not contradict each other. Furthermore, all combinations of the claimed subject matter are considered part of the inventive subject matter of this disclosure.

[0057] The foregoing and other aspects, embodiments, and features of the teachings of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and / or beneficial effects of exemplary embodiments, will become apparent from the following description or may be learned through practice of specific embodiments according to the teachings of the present invention. Attached Figure Description

[0058] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be denoted by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings.

[0059] Figure 1 This is a flowchart illustrating the OCR-based method for extracting and classifying indicators from inspection reports, as shown in this invention.

[0060] Figure 2 This is a schematic diagram of the similarity matching process shown in this invention.

[0061] Figure 3 This is a schematic diagram of the text data after classifying a single blood test indicator, as shown in this invention.

[0062] Figure 4 This is a trend chart of squamous epithelial cell indicators in the urine test report shown in this invention.

[0063] Figure 5 This is a schematic diagram of the inspection form management page in the WeChat mini program shown in this invention. Detailed Implementation

[0064] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0065] Various aspects of the invention are described in this disclosure with reference to the accompanying drawings, which illustrate numerous illustrative embodiments. The embodiments of this disclosure are not necessarily intended to encompass all aspects of the invention. It should be understood that the various concepts and embodiments described above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.

[0066] Existing medical testing technologies cannot comprehensively and practically extract and classify the indicator data from test reports. Even with intelligent recognition algorithms, they can only perform single-format recognition and classification for test reports, which cannot be applied to all types of medical test reports. This fails to provide convenient, reliable, accurate, and efficient services for medical testing technologies. This embodiment provides a test report indicator extraction and classification method based on OCR recognition, which accurately extracts various test indicator data and effectively solves the problem of inaccurate and inefficient extraction caused by the complexity of the indicator data archived by medical institutions.

[0067] Combination Figure 1 The method for extracting and classifying indicators from test reports based on OCR recognition according to an exemplary embodiment of the present invention includes the following steps:

[0068] Step 1: Acquire the image of the test report and preprocess it to obtain the feature image;

[0069] Step 2: The recognition model built based on the deep convolutional neural network recognizes the feature images and generates recognition data. The recognition data shall at least include the name, ID, numerical range, unit and type of the indicators in the test report.

[0070] Step 3: Calculate the edit distance between the indicator names on the inspection report and the indicator names in the dictionary using the first edit distance algorithm;

[0071] Step 4: Select the indicator name from the dictionary containing the minimum edit distance, and use it as the correction term to perform error correction matching on the indicator names in the test report:

[0072] a. If the match is successful, extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining the data post-processing.

[0073] b. If the match fails, the cosine similarity algorithm is invoked to calculate the cosine value between the indicator name and the error correction word in the inspection report. The cosine value is used to determine the cosine similarity based on its range, which is at least [-1, 1].

[0074] b1. If 0 < cosine value ≤ 1, the higher the similarity, the better. Extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining data post-processing.

[0075] b2. If -1 ≤ cosine value < 0, then the lower the similarity, the more likely the error correction words will be ignored.

[0076] It should be noted that the dictionary database in this embodiment of the invention is adaptively generated by a combination of intelligent algorithms and manual analysis techniques. This dictionary database contains at least the name, type, range, unit and type of various test indicators in medical testing technology. Among them, the types of each test indicator include at least blood, urine and tumor.

[0077] In embodiments of the present invention, the extraction and classification of indicator data in medical testing reports can be improved by using a first edit distance algorithm and a cosine similarity algorithm. A recognition model based on a deep convolutional neural network is used for deep learning to obtain a network model with fine-tuned parameters, which further improves the recognition accuracy of test reports and avoids the influence of errors from complex features and computational efficiency.

[0078] The following is combined Figures 2-5 The accompanying drawings and some preferred or optional examples of the present invention are used to describe more specifically the implementation process and / or effects of certain embodiments of the present invention.

[0079]

Obtain the feature image

[0080] Images of the test reports are captured using cameras and webcams and uploaded to a WeChat mini-program.

[0081] The background program runs a noise reduction algorithm, using median filtering, Gaussian filtering, and bilateral filtering to remove salt-and-pepper noise and Gaussian noise from the image of the test report;

[0082] The image is denoised by grayscale conversion, and the edges of objects in the denoised image are identified by pixel edge detection.

[0083] Based on the object edges, the pixel density is obtained by using a convolutional neural network algorithm to calculate the ratio of the number of pixels in the region formed by adjacent pixels to the area of ​​the image bounding box.

[0084] Integrate pixel density to generate a feature image.

[0085] As an example, this embodiment removes salt-and-pepper noise by replacing the value of the center pixel with the median value of the pixels within the area covered by the median filter, thus preserving the edge details of the image; it also removes Gaussian noise by replacing the value of the center pixel with the weighted average value of the pixels within the area covered by the Gaussian filter, thus smoothing the image; and finally, it replaces the value of the center pixel with the weighted average value of the pixels within the area covered by the bilateral filter, thus preserving the edge sharpness.

[0086] As an example, grayscale processing uses a weighted method to assign different weights to the red, green, and blue channels in the denoised image to obtain the grayscale value of each pixel. A global threshold is then set, and all pixels are binarized. Pixel settings are then determined based on the grayscale values. Wherein:

[0087] (1) If the gray value of a pixel is greater than or equal to the global threshold, then set the pixel to white;

[0088] (2) If the gray value of a pixel is less than the global threshold, then the pixel is set to black.

[0089] In an optional embodiment, the weights are assigned to red (0.299), green (0.587), and blue (0.114).

[0090] In an optional embodiment, the global threshold is set to 128.

[0091] As an example, pixel edge detection uses the Sobel discrete differential operator to measure the intensity change in the neighborhood of each pixel in a grayscale image to find object edges. Wherein:

[0092] The Sobel discrete differential operator consists of two 3x3 matrices, used to detect changes in the horizontal X-axis and the vertical Y-axis, respectively.

[0093] By performing convolution calculations on the neighborhood of each pixel using two 3x3 matrices, the square root of the sum of the squares of each pixel is obtained, which is the edge width of that pixel.

[0094] As an example, the pixel density is calculated by dividing the number of pixels in a region formed by adjacent pixels in an image by the area of ​​the image bounding box. This density is then used to compare the pixel distribution between different cells. The formula is as follows:

[0095] cosine_similarity=dot_product(m,n) / (norm(m)*norm(n))

[0096] Where dot_product(m,n) represents the dot product of vectors m and n, and norm(m) and norm(n) represent the norms of vectors m and n. If the dot product (i.e. the ratio) is large, it means that the pixel density of the cell is high, and vice versa.

[0097] It should be noted that this embodiment obtains clear and complete images, avoids interference from light reflection and shadows, and reduces the computational burden by adjusting the image size to unify the image dimensions, thereby improving the processing speed. Then, the image data is checked for data quality, and low-quality or erroneous image data is deleted to ensure the high quality of the dataset and improve the accuracy of subsequent OCR recognition.

[0098] [Generate recognition data]

[0099] A recognition model is built based on a deep convolutional neural network;

[0100] The training samples are input into the recognition model in the form of text sequences for deep learning training.

[0101] Training ends when the character sequence of the training samples is output.

[0102] The feature image is fed into the trained recognition model for recognition and detection, and the output tensor of shape (batch size, num classes) is the recognition data.

[0103] Where batch size represents the batch size of the input feature images, and num classes represents the number of output classes.

[0104] In an embodiment of the present invention, the recognition model is constructed based on a deep convolutional neural network, including the following steps:

[0105] (1) Define a model class named TextCNN, which inherits from nn.Module (the parent class of all network layers).

[0106] The program initializes the various components of TextCNN, which include at least word embedding layers, convolutional layers (conv1 and conv2), and fully connected layers (fc).

[0107] (2) Create line features

[0108] The Hough transform technique was used to detect horizontal and vertical lines in a table (a randomly sampled medical laboratory test report image), and the number, length, and angle of the detected lines were used as line features.

[0109] (3) Set up a word embedding layer to input line features into a deep convolutional neural network ((TextCNN)+Faster R-CNN) for recognition and classification tasks.

[0110] (4) Set up a convolutional layer, slide it over the input line features, and perform convolution operations through the convolution kernel to generate a feature image.

[0111] The convolution operation performed by the convolution kernel in this embodiment of the invention is expressed in the following form:

[0112] output[index,process]=Σ_{dx,dy}input[index+dx,process+dy]*kernel[dx,dy]

[0113] Where Σ_{dx,dy} represents the summation of dx (horizontal lines) and dy (vertical lines), input is the input data (line features), output is the output data (convolution values), and kernel is the convolution kernel.

[0114] As an example, embodiments of the present invention pad the input data to maintain the spatial size of the data and set a stride to control the stride of the convolution kernel sliding.

[0115] (5) Add an activation function (ReLU (Rectified Linear Unit)) to add nonlinearity to the model.

[0116] (6) Set up pooling layers to reduce the spatial size of feature images, reduce model parameters and computational cost, and prevent overfitting.

[0117] For example, candidate regions are selected progressively on the feature image using a sliding window approach, with the maximum value within each candidate region taken as the output. For instance, a 2x2 pooling window can have its corresponding max pooling operation represented as:

[0118] output[index,process]=max_{dx in{0,1},dy in{0,1}}input[2*index+dx,2*process+dy]

[0119] (7) Set up a fully connected layer to connect all output nodes of the previous layer to each input node.

[0120] In embodiments of the present invention, the operational form of the fully connected layer can be expressed as:

[0121] output[index]=Σ_process input[process]*weight[index,process]+bias[index]

[0122] Among them, weight and bias are parameters of the fully connected layer, which need to be learned through training.

[0123] (8) Through the processing of the convolutional layer in step (4) and the pooling layer in step (6), the local features of the image are gradually fused. The features are then stitched together and output through the fully connected layer in step (7) to form high-level semantic features.

[0124] (9) After extracting the high-level semantic features formed in step (8), use Faster R-CNN (deep learning object detection framework) to perform object detection and output recognition and classification data.

[0125] The Faster R-CNN consists of a CNN at the bottom layer for feature extraction and an RPN (Region Generation Network) + Fast R-CNN at the top layer. In the object detection stage, the RPN is used to generate candidate regions (proposals), and then the Fast R-CNN is used to perform fine classification, recognition and localization of the candidate regions.

[0126] (10) Complete the network structure settings and save the network structure, which is the recognition model.

[0127] Since the network structure parameters of the initially constructed recognition model have errors, in order to improve the accuracy of recognition and classification of input data, it is necessary to further perform deep learning on the recognition model and fine-tune the parameters to perform target detection tasks with accurate data.

[0128] In an embodiment of the present invention, deep learning training includes the following steps:

[0129] (1) Collect structured data of medical test report text and OCR image, and preprocess them using the aforementioned preprocessing method of obtaining feature images to obtain training samples;

[0130] (2) Define the text data and corresponding labels of the training samples, which shall at least include the name, range, unit and type of the indicator;

[0131] (3) Define the vocabulary size, word embedding dimension, number of convolution kernels, list of convolution kernel sizes, and number of output categories;

[0132] (4) Update the model parameters using the Adam optimizer;

[0133] (5) Repeat N training cycles

[0134] In each training cycle, the model is set to training mode, the optimizer's gradient is cleared to zero, forward propagation is performed, the model's output, i.e. the value of the loss function, is calculated, and then back propagation is performed to calculate the gradient and update the model's parameters.

[0135] (6) Model Prediction

[0136] After N iterations, training is complete. The model is set to task execution mode, gradient calculation is disabled using the context manager, and forward propagation is performed to calculate the model's output.

[0137] (7) Model output

[0138] The output is a tensor (containing character sequences and number sequences) with shape (batch_size, num_classes), where batch_size represents the batch size of the input data and num_classes represents the number of output classes.

[0139] (8) Save the model and model parameters after training is completed, and use it as the recognition model for target recognition and detection in the later stage.

[0140] It should be noted that the goal of model training in this embodiment of the invention is to iteratively adjust the model parameters through optimization algorithms (such as gradient descent) to minimize the value of the loss function, that is, to make the model's prediction result as close as possible to the true label.

[0141] Preferably, the present invention trains the recognition model with a large amount of labeled data through deep learning, enabling the recognition model to learn how to extract and fuse features from the original input image to improve the accuracy of classification. In the embodiments of the present invention, the training process adopts the backpropagation algorithm and gradient descent method. By continuously iterating and optimizing the parameters of the network, the difference between the output of the recognition model and the real label (i.e., the value of the loss function) is minimized, that is, the accuracy is maximized.

[0142] It should be further explained that when using the above recognition model to recognize the test report image in medical testing technology, the aforementioned preprocessing methods are used to obtain the feature image, which is then converted into a text sequence and input into the word embedding layer of the recognition model for vector processing. After processing by convolutional and pooling layers, the obtained features are concatenated and output as high-level semantic features through a fully connected layer. Object detection is then performed through Faster R-CNN (deep learning object detection framework) to output the corresponding character sequence.

[0143] As an example, the input text sequence is in the form shown below:

[0144]

[0145] The output character sequence is as follows:

[0146]

[0147] In an optional embodiment, the output character sequence (identification data) includes at least the name, ID, numerical range, unit, and type of the test report indicator.

[0148] [Perform error correction and matching on the identified data to generate text data of the classified test indicators]

[0149] The first edit distance algorithm is used to calculate the edit distance between the indicator names on the test report and the indicator names in the dictionary.

[0150] Select the index name from the dictionary containing the minimum edit distance and use it as the error correction word.

[0151] As an example, let the lengths of strings s1 and s2 be q and w respectively, and let dp[i][j] represent the minimum number of operations required to transform the first i characters of string s1 into the first j characters of string s2:

[0152] When i equals 0, i.e., s1 is an empty string, the number of operations to transform s1 into s2 is j, i.e., dp[0][j] = j.

[0153] When j equals 0, meaning s2 is an empty string, the number of operations to transform s1 into s2 is i, i.e., dp[i][0] = i.

[0154] When both i and j are greater than 0, there are two cases:

[0155] If the i-th character of s1 is equal to the j-th character of s2, then no operation is needed, i.e., dp[i][j] = dp[i-1][j-1]

[0156] If the i-th character of s1 is not equal to the j-th character of s2, then insertion, deletion, or replacement operations can be performed. The minimum number of operations is taken, i.e., dp[i][j] = min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) + 1

[0157] By filling the dynamic programming table dp, we can obtain the minimum number of operations required to transform s1 into s2, which is the minimum edit distance.

[0158] In this embodiment of the invention, the input test report indicator name is taken as s1, and the indicator name in the dictionary is taken as s2. The edit distance is calculated using the first edit distance algorithm, and the indicator name in the dictionary with the smallest edit distance is selected as the error correction word. The mathematical expression is as follows:

[0159] Levenshtein distance (s1, s2) = dp[q][w]

[0160] Where q is the length of the input index name s1, w is the length of the candidate word s2 in the dictionary, and dp is the dynamic programming table.

[0161] Furthermore, error correction and matching are performed on the indicator names in the test report, including two cases:

[0162] a. If the match is successful, extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining the data post-processing.

[0163] b. If the match fails, the cosine similarity algorithm is invoked to calculate the cosine value between the indicator name and the error correction word in the inspection report. The cosine value is used to determine the cosine similarity based on its range, which is at least [-1, 1].

[0164] (1) If 0 < cosine value ≤ 1, the higher the similarity, the better. Extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining the data post-processing.

[0165] (2) If -1≤cosine value<0, the lower the similarity, the more likely the error correction words will be ignored.

[0166] Furthermore, the cosine similarity algorithm is used to calculate the cosine value between the indicator name and the error correction word in the inspection report, including the following steps:

[0167] The names of the indicators and the correction words in the test report are processed by word embedding, and converted into vector A and vector B respectively;

[0168] Calculate the dot product and length of vectors A and B respectively. The cosine value is obtained from the ratio of the dot product to the length. The mathematical formula for this is as follows:

[0169] Vector A·Vector B = A1B1 + A2B2 + ... + An*Bn

[0170]

[0171]

[0172]

[0173] Where vector A·vector B represents the dot product of vectors A and B, ||vector A|| represents the length of vector A, ||vector B|| represents the length of vector B, which is the square root of the sum of the squares of each element, An represents the nth element of vector A, Bn represents the nth element of vector B, and n is a constant.

[0174] In an embodiment of the present invention, structured data is generated by post-processing the error-corrected and matched identification data, including the following steps:

[0175] Remove redundant information and delete useless information from the identification data, such as headers, footers, copyright information, and watermarks;

[0176] The cleaned data is then standardized by unifying indicator names, date values, and indicator ranges, ensuring that the identified data meets the standardized structure and format.

[0177] Reference Figures 3-5 The identification results are generated using structured data, which is text data of classified and tested indicators. This is beneficial for the classification model to perform structural analysis and to quickly and accurately extract and classify indicators.

[0178] Based on the teachings of the above embodiments, other aspects of the present invention also propose an OCR-based system for extracting and classifying indicators from inspection reports, including a data acquisition module, a preprocessing module, an identification and detection model, an error correction module, a data processing module, a classification module, and a data transmission module.

[0179] The data acquisition module is used to collect, capture, and photograph image data from medical test reports.

[0180] The preprocessing module is used to perform denoising, grayscale conversion, binarization, edge detection, and feature extraction on the images acquired by the data acquisition module.

[0181] The identification and detection model is used to identify indicators in preprocessed images.

[0182] The error correction module is used to correct and match the identified data.

[0183] The data processing module is used to clean and format the identified data to generate structured data.

[0184] The classification module is used to classify the identified indicators and test form types. The classification module is equipped with a classification algorithm program to reduce computer workload, optimize the algorithm, and improve efficiency. The classification algorithm program can be implemented by existing technology, and this embodiment does not limit the classification algorithm to a single one.

[0185] The data transmission module is used to provide data transmission services for the above modules.

[0186] In conjunction with the classification method of the above embodiments of the present invention, in another aspect of the present invention, a computer system is also proposed, comprising:

[0187] One or more processors;

[0188] The memory stores operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the process of executing the OCR-based inspection report indicator extraction and classification method of the foregoing embodiments.

[0189] In conjunction with the evaluation method of the above embodiments of the present invention, in another embodiment of the present invention, it can also be implemented in the form of a computer-readable medium storing software, the software including instructions executable by one or more computers, the instructions executing the process of the OCR-based inspection report index extraction and classification method of the foregoing embodiments when executed by the one or more computers.

[0190] As an optional example, the aforementioned computer-readable medium may be implemented using, but is not limited to, random access memory, read-only memory, electrically erasable programmable memory, optical disk storage, magnetic disk storage, and combinations of computer-readable storage media of the aforementioned types.

[0191] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for extracting and classifying indicators from inspection reports based on OCR recognition, characterized in that, include: Step 1: Acquire the image of the test report and preprocess it to obtain the feature image; Step 2: The recognition model built based on the deep convolutional neural network recognizes the feature image and generates recognition data. The recognition data includes at least the name, ID, numerical range, unit and type of the test report indicator. The recognition model is defined as a TextCNN model class that inherits from nn.Module, and the recognition data is generated in the following way: The horizontal and vertical lines contained in the feature image are detected using the Hough transform technique, and the number, length, and angle of the detected lines are used as the line features of the table skeleton. A word embedding layer is set up, and the line features are input into a deep convolutional neural network that integrates the Faster R-CNN object detection network for convolution operations, activation and max pooling processing, so as to identify and classify identification data that at least includes the name, ID, numerical range, unit and type of the inspection report indicators; Step 3: Calculate the edit distance between the indicator name on the inspection report and the indicator names in the dictionary using the first edit distance algorithm, and select the indicator name in the dictionary with the smallest edit distance as the correction word; Step 4: Select the indicator name from the dictionary containing the minimum edit distance, and use it as the correction word to perform error correction matching on the indicator name of the test report: a. If the match is successful, extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate the text data of the classified test indicators by combining the data post-processing. b. If the match fails, the cosine similarity algorithm is invoked to calculate the cosine value between the test report indicator name and the error correction word. The cosine value determines the cosine similarity based on its range, which is at least [-1, 1], where: b1. If 0 < cosine value ≤ 1, the higher the similarity, the higher the similarity. Extract the ID, numerical range, unit and type of the error correction word in the dictionary, replace the information of the indicator name in the test report, and generate text data of the classified test indicators by combining data post-processing. b2. If -1 ≤ cosine value < 0, then the lower the similarity, the more likely the error correction words will be ignored.

2. The method for extracting and classifying inspection report indicators based on OCR recognition according to claim 1, characterized in that, Step 1 specifically includes the following processing: Salt-and-pepper noise and Gaussian noise in the image of the test report were removed using median filtering, Gaussian filtering, and bilateral filtering, respectively. The image is denoised by grayscale conversion, and the edges of objects in the denoised image are identified by pixel edge detection. Based on the object's edge, a convolutional neural network algorithm is used to calculate the ratio of the number of pixels in the region formed by adjacent pixels in the image to the area of ​​the image's bounding box, thus obtaining the pixel density. The pixel density is integrated to generate the feature image.

3. The method for extracting and classifying inspection report indicators based on OCR recognition according to claim 1, characterized in that, In step 2, generating the identification data includes: A recognition model is built based on a deep convolutional neural network; The training samples are input into the recognition model in the form of text sequences for deep learning training; Training ends when the character sequence of the training sample is output. The feature image is substituted into the trained recognition model for recognition and detection, and the output tensor of shape (batch size, num classes) is the recognition data. Where batch size represents the batch size of the input feature images, and num classes represents the number of output classes.

4. The method for extracting and classifying indicators from inspection reports based on OCR recognition according to claim 3, characterized in that, The deep learning training includes: Collect structured data of medical test report text and OCR image, and preprocess them using the preprocessing method implemented in step 1 to obtain training samples; Initialize the word segmenter and build the recognition model; The training samples are converted into text sequences and input into the recognition model for training. Training stops after N iterations. Save the trained model and use it as the recognition model after deep learning training is complete.

5. The method for extracting and classifying inspection report indicators based on OCR recognition according to claim 3, characterized in that, In step 3, the first edit distance algorithm is calculated based on the number of character-level operations. The number of character-level operations includes at least the number of insertions, deletions, and replacements. The fewer the number of operations, the smaller the edit distance.

6. The method for extracting and classifying indicators from inspection reports based on OCR recognition according to claim 5, characterized in that, In step 4, the cosine similarity algorithm is invoked to calculate the cosine value between the indicator name on the inspection report and the error correction word, including: The test report indicator names and the error correction words are subjected to word embedding processing and converted into vector A and vector B, respectively; Calculate the dot product and length of vector A and vector B respectively. The cosine value is obtained from the ratio of the dot product to the length, and its mathematical expression is as follows: Vector A·Vector B = A1B1 + A2B2 + ... + An*Bn ||vector A|| = ; ||vector B|| = ; Cosine similarity = ; Where vector A·vector B represents the dot product of vectors A and B, ||vector A|| represents the length of vector A, ||vector B|| represents the length of vector B, which is the square root of the sum of the squares of each element, An represents the nth element of vector A, Bn represents the nth element of vector B, and n is a constant.

7. The method for extracting and classifying inspection report indicators based on OCR recognition according to claim 1, characterized in that, The data post-processing involves cleaning and standardizing the identified data to generate structured data, i.e., text data of classified test indicators.

8. A system for extracting and classifying indicators from inspection reports based on OCR recognition, characterized in that, It includes a data acquisition module, a preprocessing module, a recognition and detection model, an error correction module, a data processing module, a classification module, and a data transmission module, among which: The data acquisition module is used to collect, capture, and photograph image data from medical test reports; The preprocessing module is used to perform denoising, grayscale conversion, binarization, edge detection, and feature extraction on the images acquired by the data acquisition module; A recognition and detection model is used to identify indicators in preprocessed images; The error correction module is used to correct and match the identified data; The data processing module is used to clean and format the identification data to generate structured data. The classification module is used to classify the identified indicators and test report types; The data transmission module is used to provide data transmission services for the above modules.

9. A computer system, characterized in that, include: One or more processors; The memory stores instructions to be operated, which, when executed by the one or more processors, cause the one or more processors to perform operations, including the process of performing the OCR-based inspection report indicator extraction and classification method according to any one of claims 1-7.

10. A computer-readable medium for storing software, characterized in that, The software includes instructions executable by one or more computers, which, when executed by the one or more computers, perform the process of the OCR-based indicator extraction and classification method for inspection reports as described in any one of claims 1-7.

Images