Medical form information extraction method and device

Through the combination of optical character recognition and regular expressions, the problems of slow processing speed, insufficient privacy protection and low accuracy in medical form information extraction are solved, and efficient and robust information extraction is achieved, which is suitable for a variety of medical form formats.

CN119091458APending Publication Date: 2024-12-06JIAHE MEIKANG BEIJING TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411079021.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-12-06

Smart Images

  • Figure CN119091458A_ABST
    Figure CN119091458A_ABST
Patent Text Reader

Abstract

The invention provides a medical form information extraction method and device, and the method comprises the steps: carrying out the character recognition and layout adjustment and optimization of a medical form image based on an optical character recognition algorithm, and obtaining text content; and constructing a regular expression of the text content, and extracting information of the text content based on the regular expression. In the mode, the medical form image can be preprocessed by adopting an image enhancement technology so as to optimize the image quality and improve the performance of a subsequent OCR recognition system, the enhanced image result has readability and recognizability, and the accuracy and speed of OCR recognition can be effectively improved; after the image is subjected to enhancement, information extraction and information optimization, the regular expressions matched with the marked fields can be automatically generated by combining artificial intelligence and programming technologies and utilizing an advanced large language model according to the marked fields, and the efficiency and accuracy of document analysis can be remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method and device for extracting medical form information. Background Art

[0002] In the current medical industry, it is a common practice to save test reports in the form of images, especially in certain quick, temporary or technically limited situations. Test reports usually contain key indicators and values ​​of clinical tests, such as hematological analysis, biochemical test results, etc. Although relevant test information can be obtained through images, medical reports saved in the form of images have the following problems: information accessibility issues, compared with electronic documents, image data is not easy to directly access and search; data analysis limitations, cannot be directly applied to advanced data processing methods such as statistical analysis and machine learning, limiting the exploration of data potential; maintaining data quality and consistency issues, the generation, storage and transmission of image files may be affected by a variety of factors, such as scanning quality, file compression, etc., which may reduce the clarity and accuracy of the data and increase the risk of medical errors.

[0003] The development of optical character recognition (OCR) technology has made it possible to automatically extract text information from medical forms in image form, greatly accelerating the data processing process. Currently, OCR technology has been applied to electronic medical records systems to convert paper medical records into electronic formats, which not only saves storage space but also makes it easier to retrieve and share records. OCR technology reduces the workload of manual data entry by automatically identifying and transcribing documents such as test reports and case summaries. In addition, OCR technology has greatly promoted the electronicization of medical information, which not only facilitates information sharing among medical teams, but also provides rich resources for data mining and analysis in medical research and public health management.

[0004] Although the current OCR technology has made some progress in the collection of medical document information, there are still some problems such as processing speed, privacy protection, accuracy and robustness. First of all, the processing speed is a challenge. For complex medical document OCR, it may be necessary to use a relatively large model for processing. The CPU inference speed is slow without GPU resources. Secondly, the privacy issue is an issue that OCR technology needs to consider. Medical documents contain sensitive personal health information. Data security and privacy protection in the OCR processing process. Although there are currently offline OCR processing solutions, there is no offline processing solution for form extraction. In addition, the accuracy problem directly affects the practicality of OCR technology. The OCR recognition accuracy of medical documents is affected by many factors, including the quality of the document, the clarity of the text, the complexity of the layout, and the correct understanding of professional terms. Finally, the robustness problem is a major difficulty at present. The current form information extraction is limited to forms of specific formats. Each style of form requires a special model processing, and there is a lack of universal solutions. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a method and device for extracting medical form information to improve the processing speed, privacy protection capability, accuracy and robustness of information extraction.

[0006] In a first aspect, an embodiment of the present invention provides a method for extracting medical form information, the method comprising: performing character recognition and layout adjustment optimization on a medical form image based on an optical character recognition algorithm to obtain text content; constructing a regular expression of the text content, and extracting information about the text content based on the regular expression.

[0007] In an optional embodiment of the present application, the above-mentioned step of performing character recognition and layout adjustment optimization on the medical form image based on the optical character recognition algorithm to obtain the text content includes: performing image enhancement processing on the medical form image; performing character recognition on the medical form image based on the optical character recognition algorithm to extract the text content; wherein the text content includes extracted character information and character box candidate information; performing layout adjustment optimization on the text content based on the optical character recognition adaptive optimization algorithm; wherein the text content includes: text segmentation box and text height.

[0008] In an optional embodiment of the present application, the above-mentioned image enhancement processing includes at least one of the following: brightness contrast adjustment, noise removal, sharpening processing, color correction, and document correction.

[0009] In an optional embodiment of the present application, the above-mentioned step of optimizing the layout of the text content based on the optical character recognition adaptive optimization algorithm includes: evaluating the position and size of the text segmentation box based on the optical character recognition adaptive optimization algorithm, and adjusting the positioning of the text block based on the evaluation result to ensure that the text content is consistent with the layout of the medical form image.

[0010] In an optional embodiment of the present application, the above-mentioned step of constructing a regular expression of text content includes: selecting multiple sample documents from a document library to be processed, marking multiple fields to be extracted of the multiple sample documents; and constructing a regular expression of text content based on the multiple fields to be extracted.

[0011] In an optional embodiment of the present application, the above-mentioned step of constructing a regular expression of text content based on multiple fields to be extracted includes: inputting multiple fields to be extracted into a large language model, and outputting a basic regular expression of the text content; determining the feedback result of the basic regular expression during the application process of the basic regular expression; inputting the feedback result into the large language model, and outputting a general regular expression of the text content.

[0012] In an optional embodiment of the present application, the above-mentioned step of adjusting and optimizing the basic regular expression includes at least one of the following: enhancing the matching accuracy of the basic regular expression, determining the character set of the basic regular expression, and determining the character length of the basic regular expression.

[0013] In an optional embodiment of the present application, the above-mentioned step of extracting information of text content based on regular expressions includes: integrating regular expressions into the data processing flow through scripts to extract fields from the text content as information of the text content.

[0014] In an optional embodiment of the present application, the above-mentioned step of integrating regular expressions into the data processing flow through scripts to extract specific fields from text content includes: encoding the regular expression into a pattern in the script; applying the encoded pattern to the text content, and extracting specific fields from the text content based on the objective function.

[0015] In a second aspect, an embodiment of the present invention further provides a medical form information extraction device, the device comprising: a character recognition and layout adjustment module, for performing character recognition and layout adjustment optimization on a medical form image based on an optical character recognition algorithm to obtain text content; and a text content information extraction module, for constructing a regular expression of the text content and extracting information about the text content based on the regular expression.

[0016] The embodiments of the present invention bring the following beneficial effects:

[0017] The embodiment of the present invention provides a method and device for extracting information from medical forms. The method performs character recognition and layout adjustment optimization on medical form images based on an optical character recognition algorithm to obtain text content; a regular expression of the text content is constructed, and information of the text content is extracted based on the regular expression. In this method, image enhancement technology can be used to pre-process the medical form image to optimize the image quality and improve the performance of the subsequent OCR recognition system. The enhanced image results are more readable and recognizable, which can effectively improve the accuracy and speed of OCR recognition. After the image is enhanced, information is extracted, and information is optimized, it is also possible to combine artificial intelligence and programming technology to use an advanced large language model to automatically generate a regular expression that matches the annotated field, which can significantly improve the efficiency and accuracy of document analysis.

[0018] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0019] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a method for extracting medical form information provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a technical route of a medical form information extraction method provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of an original medical form image to be extracted provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of an image-enhanced medical form image provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the PaddleOCR model structure provided in an embodiment of the present invention;

[0026] Figure 6A schematic diagram of a PaddleOCR recognition result of a selected character provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a medical form information extraction device provided by an embodiment of the present invention;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] At present, although the current OCR technology has made some progress in the collection of medical document information, there are still some problems such as processing speed, privacy protection, accuracy and robustness. First of all, the processing speed problem is a challenge. For complex medical document OCR, it may be necessary to use a relatively large model for processing. The CPU inference speed is slow without GPU resources. Secondly, the privacy issue is an issue that OCR technology needs to consider. Medical documents contain sensitive personal health information. Data security and privacy protection in the OCR processing process. Although there are currently offline OCR processing solutions, there is no offline processing solution for form extraction. In addition, the accuracy problem directly affects the practicality of OCR technology. The OCR recognition accuracy of medical documents is affected by many factors, including the quality of the document, the clarity of the text, the complexity of the layout, and the correct understanding of professional terms. Finally, the robustness problem is a major difficulty at present. The current form information extraction is limited to forms of specific formats. Each style of form requires a special model processing, and there is a lack of universal solutions.

[0031] Based on this, a medical form information extraction method and device provided by an embodiment of the present invention are applied to the field of information processing technology, involving text data analysis and automatic information extraction technology, and specifically relates to a method for quickly extracting medical form information based on PaddleOCR (PaddlePaddle Optical Character Recognition) and automatic regularization, which can improve the processing speed, privacy protection capability, accuracy and robustness of information extraction.

[0032] To facilitate understanding of this embodiment, a medical form information extraction method disclosed in an embodiment of the present invention is first introduced in detail.

[0033] Embodiment 1:

[0034] The embodiment of the present invention provides a method for extracting medical form information. Figure 1 A flowchart of a method for extracting medical form information is shown, and the method for extracting medical form information includes the following steps:

[0035] Step S102: performing character recognition and layout adjustment optimization on the medical form image based on an optical character recognition algorithm to obtain text content.

[0036] The optical character recognition algorithm in this embodiment can be a PaddleOCR algorithm or a traditional OCR algorithm, which is not limited in this application. In this embodiment, the PaddleOCR algorithm is taken as an example, and the method provided in this embodiment includes character recognition and layout adjustment of medical form images based on the PaddleOCR algorithm.

[0037] In some embodiments, the medical form image can be subjected to image enhancement processing; character recognition can be performed on the medical form image based on an optical character recognition algorithm to extract text content; wherein the text content includes extracted character information and character box candidate information; and the text content is subjected to layout adjustment and optimization based on an optical character recognition adaptive optimization algorithm; wherein the text content includes: text segmentation box and text height.

[0038] In this embodiment, an image enhancement algorithm can be used to improve image quality. Medical form images often have various quality problems due to scanning quality, lighting conditions, and the condition of the original paper document. The purpose of image enhancement is to improve the quality of medical form images through a series of preprocessing technologies to facilitate OCR recognition. Among them, the above-mentioned image enhancement processing includes at least one of the following: brightness contrast adjustment, noise removal, sharpening, color correction, document correction, etc.

[0039] In this embodiment, the PaddleOCR algorithm can also be used to perform character recognition to extract character information and character frame selection information; the OCR adaptive optimization algorithm is used to optimize the information extracted by PaddleOCR.

[0040] In some embodiments, the position and size of the text segmentation box may be evaluated based on an optical character recognition adaptive optimization algorithm, and the positioning of the text block may be adjusted based on the evaluation result to keep the text content consistent with the layout of the medical form image.

[0041] This embodiment proposes an OCR adaptive optimization algorithm to further adjust the layout information extracted by PaddleOCR to ensure the consistency of the PaddleOCR extraction results. This algorithm makes full use of the key layout information provided by PaddleOCR, including the text segmentation box and text height, and performs a series of refined calculations and adjustments. The core of the algorithm is to comprehensively evaluate the position and size of the text segmentation box, and then automatically adjust the positioning of the text block to ensure that the information extracted by OCR is highly consistent with the layout of the original image.

[0042] The position of the above-mentioned text segmentation frame may be an absolute position such as the coordinates of the text segmentation frame, or may be a relative position of the text segmentation frame relative to other modules of the layout, which is not limited in this embodiment.

[0043] Step S104: construct a regular expression of the text content, and extract information of the text content based on the regular expression.

[0044] The method provided in this embodiment also includes regular automatic construction and rapid information extraction.

[0045] In some embodiments, multiple sample documents may be selected from a library of documents to be processed, multiple fields to be extracted of the multiple sample documents may be marked, and a regular expression of text content may be constructed based on the multiple fields to be extracted.

[0046] In this embodiment, the sample document may be selected to annotate the required extraction fields; and a regular expression may be automatically constructed based on the annotated selected fields.

[0047] In some embodiments, multiple fields to be extracted are input into a large language model, and a basic regular expression of the text content is output; a feedback result of the basic regular expression is determined during the application of the basic regular expression; the feedback result is input into the large language model, and a universal regular expression of the text content is output.

[0048] In this embodiment, the advanced capabilities of a large language model (such as ChatGLM) can be used to automatically generate and optimize regular expressions. For example, first, by providing 5 representative samples to the model, the model will analyze and identify the common patterns in these samples. Based on these patterns, the model will build a preliminary regular expression to match texts of similar formats. Next, 100 samples are obtained from the document library to be processed, and the 100 samples are processed using the above regular expression and the results are fed back. Then, the above regular expression can be further adjusted and optimized according to the feedback results to ensure high accuracy and wide applicability. Finally, a general regular expression that can be used directly is generated. In addition, after creating a regular expression, it can be directly used to process the documents in the document library to be processed. When an exception is detected in the processed document (such as a null value, or inconsistent with the processing result of the previous document to be processed), the processing flow is suspended, the regular expression is optimized, and the processing of the document to be processed is finally completed. Therefore, the optimization of the above regular expression is not used to limit this application, and other optimization methods are also within the scope of protection of this application.

[0049] In this embodiment, regular expressions can also be integrated to automatically extract the remaining document information.

[0050] In some embodiments, regular expressions may be integrated into a data processing flow through a script to extract fields from text content as information about the text content.

[0051] Among them, the above-mentioned script can be a Python script or a script built in other languages, such as JAVA, MATLAB, etc. In this embodiment, the automatically generated regular expression can be integrated into the data processing flow through Python's re module to automatically extract specific fields from the text data as information about the text content.

[0052] In some embodiments, the regular expression may be encoded as a pattern in a script; the encoded pattern is applied to the text content, and specific fields are extracted from the text content based on a target function.

[0053] The generated regular expression is first encoded as a pattern in a Python script and then applied to stored or on-the-fly generated text data. Using functions such as re.search or re.findall, matching text passages are quickly identified and extracted, and the extracted data can then be further processed and stored.

[0054] The embodiment of the present invention provides a method for extracting information from a medical form. The method performs character recognition and layout adjustment optimization on a medical form image based on an optical character recognition algorithm to obtain text content; a regular expression of the text content is constructed, and information of the text content is extracted based on the regular expression. In this method, image enhancement technology can be used to pre-process the medical form image to optimize the image quality and improve the performance of the subsequent OCR recognition system. The enhanced image result is more readable and recognizable, which can effectively improve the accuracy and speed of OCR recognition. After the image is enhanced, information is extracted, and information is optimized, it is also possible to combine artificial intelligence and programming technology to use an advanced large language model to automatically generate a regular expression that matches the annotated field, which can significantly improve the efficiency and accuracy of document analysis.

[0055] Embodiment 2:

[0056] This embodiment provides another method for extracting medical form information. This method is implemented on the basis of the above embodiment. Figure 2 The figure shows a technical route diagram of a medical form information extraction method, focusing on the specific steps of character recognition and layout adjustment based on PaddleOCR.

[0057] First, the scanned document images are processed using an algorithm based on PaddleOCR in order to improve the image quality and perform character recognition. In the image enhancement stage, the readability of the image is optimized through steps such as contrast adjustment, noise reduction, and sharpening. Then, the PaddleOCR algorithm is used to accurately extract text content from the optimized image, and supports multi-language recognition and precise character selection. The final stage involves refining the recognized text information using the OCR adaptive optimization algorithm, which includes adaptive threshold adjustment and post-processing scripts to correct errors in the recognition process.

[0058] In this embodiment, an image enhancement method can be first used for the input image, and then the enhanced image is used for character extraction using PaddleOCR, and finally the proposed correction algorithm is used for optimization.

[0059] 1. Image enhancement methods

[0060] The key role of image enhancement technology in OCR recognition systems is to improve recognition accuracy and speed. By preprocessing the scanned or photographed document images, enhancement technology optimizes the readability of the image, allowing the OCR algorithm to recognize the characters in the image more accurately and quickly.

[0061] See also Figure 3The schematic diagram of an original medical form image to be extracted is shown. During the storage and transmission process, the image data will have format changes and image compression. In addition, different scanning methods will also cause changes in image pixels and image compression, which will lead to reduced quality of the medical form image. The specific enhancement process is as follows:

[0062] 1.1 Non-local mean denoising

[0063] Non-local mean denoising (NLMeans) calculates a weighted average at each pixel, which is the weighted average of the local area centered on pixel p and the similar area in the entire image. This weight depends on the similarity between the two areas. If the two areas are more similar, the contribution to the central pixel will be greater.

[0064] For the input image I, NLMeans outputs I′(p) at each pixel p, which can be expressed as:

[0065]

[0066] Among them, Ω is all the pixels in the image, Z(p) is the normalization factor to ensure that the sum of the weights is 1. I(p) is the pixel value at each pixel p in image I, w(p, q) is the weight between pixel p and pixel q, which is usually calculated by the similarity between the pixel values ​​of pixel p and pixel q. Gaussian weights or other functions can be used to measure the similarity.

[0067] 1.2 Local Binarization

[0068] For the input image, select a window size w×w (which can be set to 25 in this embodiment), and for each pixel p(x, y) in the image, calculate the w×w neighborhood average μ including p(x, y), and set a threshold T(x, y), which can be the local average μ, or μ plus or minus a constant C.

[0069] T(x, y) = μ(x, y) ± C

[0070] For each pixel the following transformation is applied:

[0071]

[0072] Among them, B(x, y) is the pixel value after binarization. Local binarization can adapt to local illumination changes and clearly separate the foreground and background, which is suitable for text recognition and other applications requiring high contrast.

[0073] 1.3 Image Sharpening

[0074] After the image is binarized, a convolution kernel-based image sharpening method is used. This method enhances the edges and details of the image by applying a specific sharpening convolution kernel. The sharpening kernel usually has a matrix form with positive central values ​​and negative surrounding values, as shown below:

[0075]

[0076] The sharpening kernel increases the clarity and detail contrast of the image by emphasizing the contrast difference between the central pixel of the image and its neighboring pixels. This method is suitable for highlighting the edges of the image or improving the image quality, making the details more clearly visible.

[0077] 1.4 Other Processing

[0078] Furthermore, the image quality is further enhanced by adjusting the contrast and brightness of the image. This operation is achieved through a linear transformation, where the contrast factor α and the brightness factor β are used to adjust the pixel values ​​of the image. The contrast adjustment is expressed by the following formula:

[0079] enhanced(x,y)=clip(α·sharpened(x,y)+β)

[0080] Among them, enhanced(x, y) is the pixel value of the pixel at the (x, y) coordinate after enhancement, sharpened(x, y) is the pixel value of the pixel at the (x, y) coordinate after sharpening, α controls the intensity of the contrast, β adjusts the overall brightness of the image, and the clip function ensures that all pixel values ​​remain within the legal color range.

[0081] Image enhancement technology is used to pre-process the medical form image to optimize the image quality and improve the performance of the subsequent OCR recognition system. Combining the above enhancement methods, the enhanced image results are more readable and recognizable, which can effectively improve the accuracy and speed of OCR recognition. The results can be seen in Figure 4 A schematic diagram of an image-enhanced medical form image is shown.

[0082] 2. Character recognition based on PaddleOCR

[0083] PaddleOCR is a powerful OCR tool with efficient detection and recognition modules. Its structure can be found in Figure 5 A schematic diagram of the PaddleOCR model structure is shown.

[0084] In terms of detection module, PaddleOCR uses LCNetV3 as the backbone network to improve detection accuracy. The more accurate backbone network PFHead structure uses parallel head branch fusion to further improve detection performance. The DSR (Dynamic Super Resolution) technology can dynamically increase the shrink ratio during training, optimizing the model training effect. At the same time, the CML (Collaborative Multi-Loss) strategy is introduced to improve model performance by adding the KL (KullbackLeible) divergence of the Student and Teacher network outputs.

[0085] In terms of the recognition module, PaddleOCR uses SVTR_LCNetV3 as the backbone network to improve recognition accuracy; the Lite-Neck structure is used to achieve a lightweight design; the GTC (Guided Training of Connectionist Temporal Classification)-NRTR (a non-repeating sequence to sequence model for scene text recognition) stable Attention guidance branch helps improve the robustness of the model; the multi-scale training strategy and data mining scheme (DF, Data Filter) are used to further improve the recognition performance; and the DKD (Decoulped Knowledge Distillation) distillation strategy also helps to improve the generalization ability and performance of the model.

[0086] PaddleOCR recognition results can be found in Figure 6 The following is a schematic diagram of the PaddleOCR recognition result of the selected characters. The field is selected and the recognition results are as follows:

[0087] [[129.0,52.0],[615.0,52.0],[615.0,136.0],[129.0,136.0]]

[0088] ('Name:',0.9630592)

[0089] [[1793.0,54.0],[2271.0,50.0],[2272.0,120.0],[1793.0,125.0]]

[0090] ('Department: Emergency Department', 0.95352477)

[0091] [[942.0,56.0],[1525.0,56.0],[1525.0,116.0],[942.0,116.0]]

[0092] ('Registration number: 0001506134', 0.99596846)

[0093] [[2701.0,58.0],[3365.0,60.0],[3365.0,116.0],[2701.0,114.0]]

[0094] ('Date of Birth: 1955-12-09', 0.9364492)

[0095] …

[0096] The above partial results contain field-related coordinate information, and the results are given according to the upper left, upper right, lower right, and lower left. The recognition results and related confidence information are also given at the same time. The extracted text information and text box selection orientation are convenient for subsequent regular construction.

[0097] 3. OCR frame selection information optimization

[0098] OCR technology often faces challenges of errors and inaccuracies when processing text box selection and positioning in documents, especially when processing documents with complex layouts or diverse text alignments. These problems often lead to incorrect text order or missing information after recognition, affecting the efficiency of subsequent text parsing and information extraction.

[0099] This method proposes a new algorithm to optimize the text box selection and positioning in OCR, thereby improving the accuracy and efficiency of information extraction. The algorithm consists of two main parts: the function is_same_line that determines whether the text boxes are in the same line, and the function sort_by_coordinates that sorts the text boxes according to their positions on the page.

[0100] First, the algorithm determines whether the two text boxes are in the same row by calculating the vertical center coordinates of each text box. Specifically, if the average vertical coordinate difference between two text boxes is less than a preset threshold (adjusted according to the text box height), the two boxes are considered to be in the same row. This method can effectively handle slight position deviations caused by factors such as scanning tilt and irregular page layout, ensuring that text boxes that are visually in the same row are classified together.

[0101] For each text box, define its average y coordinate y avg for:

[0102]

[0103] Where y1, y3 are the y coordinates of the first and third points of the box.

[0104] Next, compare whether the absolute value of the average y coordinate difference of the two boxes is less than the threshold threshold:

[0105] same_line=|y avg1 -y avg2 |<threshold

[0106] Among them, y avg1 ,y avg2 are the average y coordinates of the two boxes respectively.

[0107] Then, after initially classifying the text boxes into corresponding rows by the vertical coordinates, the algorithm further sorts the text boxes in each row by the horizontal coordinates. This not only ensures that the order of the text boxes conforms to the natural reading order (from left to right), but also facilitates subsequent text merging, parsing, and information extraction. The sorting process first globally sorts by y coordinate to identify rows, and then sorts by x coordinate within each row.

[0108] Global sorting uses a list sorting method to sort the text boxes according to their average y coordinates, which can be achieved using the sort function.

[0109] The steps for grouping and row sorting are as follows:

[0110] Step 1, traverse the sorted text boxes and use the is_same_line function to check whether consecutive text boxes belong to the same line.

[0111] Step 2, collect the text boxes in the same row into a list. When it is detected that the next text box is not in the current row, sort the text boxes in the current row by x coordinates, which can be achieved by the sort function.

[0112] Step 3: Add the sorted row data to the result list, and then start processing the next row.

[0113] Through the above steps, the algorithm effectively organizes and optimizes the positioning of text boxes, so that the order of text after OCR processing is consistent with the layout and reading order in the actual document.

[0114] Embodiment three:

[0115] This embodiment provides another method for extracting medical form information. This method is implemented on the basis of the above embodiment. Figure 2 As shown, the specific steps of automatic construction of regular expressions and rapid information extraction are described in detail.

[0116] After image enhancement, information extraction, and information optimization, the efficiency and accuracy of document analysis can be significantly improved by combining artificial intelligence and programming techniques. First, the fields to be extracted in the annotated sample documents are selected. This step lays the foundation for automated processing. Subsequently, advanced large language models, such as ChatGLM, which can generate large prediction models for regular expressions, are used to automatically generate regular expressions that match the annotated fields. ChatGLM analyzes samples, identifies patterns, and creates regular expressions. These regular expressions generated by ChatGLM not only analyze samples and identify patterns in them, but also effectively create rules for matching text. Ultimately, these regular expressions are integrated in the Python environment through the re module and applied to a wider set of documents to accurately extract the required information.

[0117] 1. Select sample documents to mark the fields you want to extract

[0118] Select 20 documents from the document library to be processed, and then manually annotate the key-value pairs in the documents. For example, the sample information extracted by OCR is as follows:

[0119] Name: Registration number: 0001506134 Department: South District Geriatrics Department Date of birth: 1955-12-09 Serial number: 842 Gender: Male Medical record number: 177455 Requesting physician: Specimen type: Serum Initial diagnosis: Acute myocardial infarction Age: 66 years old Bed number: Bed 03 Application time: 2022-07-12 10:15 Collection time: 2022-07-13 05:45……

[0120] For the sample information provided, a series of key fields will be processed, including name, registration number, department, date of birth, serial number, gender, medical record number, application physician, specimen type, preliminary diagnosis, age, bed number, application time and collection time. In the document, each key field will be clearly marked as a key and recorded together with its corresponding value. After marking, the dictionary information is as follows:

[0121] {'Name':'','Registration Number':'0001506134','Department':'South District Geriatrics Department','Date of Birth':'1955-12-09','Serial Number':'842'……}

[0122] The results are converted into JSON files for subsequent data processing.

[0123] 2. Automatically build regular expressions based on the selected fields

[0124] The ChatGLM-6B open source conversational language model is used. The model is based on the General Language Model (GLM) architecture and has 6.2 billion parameters. ChatGLM-6B uses similar technology to ChatGPT and is optimized for Chinese question-answering and conversation. After about 1T of identifiers in Chinese and English, supplemented by supervised fine-tuning, feedback self-help, human feedback reinforcement learning and other technologies, the 6.2 billion parameter ChatGLM-6B can already generate answers that are quite in line with human preferences.

[0125] 2.1. Automatic generation of regular expressions

[0126] In order for the large language model ChatGLM to accurately understand the task requirements, it is necessary not only to provide annotated key-value pairs, but also to give clear indicators or descriptions to inform the model that regular expressions matching these fields need to be generated. For example, this method prompts the model: "Generate a regular expression matching each field based on the following key-value pairs."

[0127] The model will give the corresponding regular expression, as shown below:

[0128] Name:(.+?)\s+

[0129] Registration number: (\d+)\s+

[0130] Department:(.+?)\s+

[0131] Date of Birth:\s+(.+?)\s+

[0132] Serial number:(\d+)\s+

[0133] …

[0134] ChatGLM will analyze the input key-value data, identify the patterns and formats, and then automatically build regular expressions suitable for matching these specific fields. This step is done through the model's internal algorithm, which takes into account the variability and complexity of the field content.

[0135] 2.2 Regular Expression Optimization

[0136] Although the large prediction model can provide relevant regular expressions, in order to improve the efficiency and accuracy of pattern matching, the regular expressions need to be optimized.

[0137] In some embodiments, at least one of the following may be included: strengthening the matching accuracy of the basic regular expression, clarifying the character set of the basic regular expression, and clarifying the character length of the basic regular expression.

[0138] The specific optimization strategies are as follows:

[0139] Improve matching accuracy: Use anchors such as ^ and $ to specify that the match must appear at the beginning or end of the text. This not only improves matching accuracy, but also avoids unnecessary searches in long texts.

[0140] Character set determination: Specifying an explicit character set rather than relying on a generic character class, such as using [0-9] instead of \d, can avoid character class ambiguity in some cases.

[0141] Character length determination: In some cases, such as date information, the length is fixed, so the regular expression can be used to find a character set of a specific length.

[0142] The results of optimizing the regular expression using the above method are as follows:

[0143] Name:(.*?)\s+

[0144] Registration number: (\d+)\s+

[0145] Department:(.*?)\s+

[0146] Date of Birth:\s+(\S{10})\s+

[0147] Serial number:(\d+)\s+

[0148] …

[0149] 2.3. Regular expression integration and automatic extraction of other document information

[0150] After forming the regular expression, the remaining documents are quickly and accurately extracted. To meet this requirement, a regular automated text extraction method reMatch can be built based on Python, aiming to optimize and simplify the process of extracting key information from the text.

[0151] The reMatch function is designed to search for a pattern defined by a regular expression in the specified text and return the corresponding result based on the match found. This function can first define three parameters:

[0152] tag: A string containing a regular expression to search for.

[0153] text: The target text string to be searched.

[0154] num: Integer specifying the length of the list containing None returned when no matches are found, default is 1.

[0155] The execution flow is as follows:

[0156] Step 1: Use the search method in Python's re module to find the first match of the regular expression that matches tag in text.

[0157] Step 2, if a match is found, the function checks the groups in the match result. If the match object contains multiple groups, it returns the values ​​of all groups; if it contains only one group, it returns only the value of that group.

[0158] Step 3: If no matches are found, the function returns a list containing num None elements (None indicates a match failure) to indicate that the search was unsuccessful.

[0159] The reMatch function is designed with ease of operation and efficiency of execution in mind, enabling it to quickly and accurately extract required information from large amounts of text. Through parameterized regular expressions and a flexible return mechanism, this function can be widely used in a variety of scenarios, such as automated document processing, information retrieval, and data analysis.

[0160] Embodiment 4:

[0161] Corresponding to the above method embodiment, the present invention provides a medical form information extraction device, see Figure 7 The structure diagram of a medical form information extraction device shown in FIG. 1 includes:

[0162] The character recognition and layout adjustment module 71 is used to perform character recognition and layout adjustment optimization on the medical form image based on the PaddleOCR algorithm to obtain text content;

[0163] The text content information extraction module 72 is used to construct a regular expression of the text content and extract information of the text content based on the regular expression.

[0164] The embodiment of the present invention provides a medical form information extraction device, which performs character recognition and layout adjustment optimization on a medical form image based on an optical character recognition algorithm to obtain text content; constructs a regular expression of the text content, and extracts information of the text content based on the regular expression. In this method, image enhancement technology can be used to pre-process the medical form image to optimize the image quality and improve the performance of the subsequent OCR recognition system. The enhanced image result is more readable and recognizable, which can effectively improve the accuracy and speed of OCR recognition; after the image is enhanced, information is extracted, and information is optimized, it can also be combined with artificial intelligence and programming technology to use an advanced large language model to automatically generate a matching regular expression based on the annotated field, which can significantly improve the efficiency and accuracy of document analysis.

[0165] The above-mentioned character recognition and layout adjustment module 71 is used to perform image enhancement processing on the medical form image; perform character recognition on the medical form image based on the optical character recognition algorithm to extract text content; wherein the text content includes extracted character information and character box candidate information; perform layout adjustment and optimization on the text content based on the optical character recognition adaptive optimization algorithm; wherein the text content includes: text segmentation box and text height.

[0166] The above-mentioned image enhancement processing includes at least one of the following: brightness contrast adjustment, noise removal, sharpening, color correction, and document correction.

[0167] The character recognition and layout adjustment module 71 evaluates the position and size of the text segmentation frame using an adaptive optimization algorithm based on optical character recognition, and adjusts the positioning of the text block based on the evaluation result to keep the text content consistent with the layout of the medical form image.

[0168] The text content information extraction module 72 is used to select multiple sample documents from the to-be-processed document library, mark multiple to-be-extracted fields of the multiple sample documents, and construct a regular expression of the text content based on the multiple to-be-extracted fields.

[0169] The above-mentioned text content information extraction module 72 is used to input multiple fields to be extracted into a large language model and output a basic regular expression of the text content; determine the feedback result of the basic regular expression during the application process of the basic regular expression; input the feedback result into the large language model and output a universal regular expression of the text content.

[0170] The text content information extraction module 72 is used for at least one of the following: enhancing the matching accuracy of the basic regular expression, determining the character set of the basic regular expression, and determining the character length of the basic regular expression.

[0171] The text content information extraction module 72 is used to integrate regular expressions into the data processing flow through scripts to extract fields from the text content as information of the text content.

[0172] The text content information extraction module 72 is used to encode the regular expression into a pattern in the script; apply the encoded pattern to the text content, and extract specific fields from the text content based on the target function.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the medical form information extraction system described above can refer to the corresponding process in the embodiment of the aforementioned medical form information extraction method, and will not be repeated here.

[0174] Embodiment five:

[0175] The embodiment of the present invention also provides an electronic device for executing the above-mentioned medical form information extraction method; see Figure 8 A structural schematic diagram of an electronic device is shown, the electronic device includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned medical form information extraction method.

[0176] Further, Figure 8 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0177] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0178] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.

[0179] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned medical form information extraction method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0180] The computer program product of the medical form information extraction method and device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be referred to the method embodiment, which will not be repeated here.

[0181] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0182] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0183] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0184] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0185] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for extracting medical form information, characterized in that: The method comprises: Based on the optical character recognition algorithm, the medical form image is subjected to character recognition and layout adjustment optimization to obtain the text content; A regular expression of the text content is constructed, and information of the text content is extracted based on the regular expression.

2. The method according to claim 1, characterized in that: The steps of performing character recognition and layout optimization on the medical form image based on the optical character recognition algorithm to obtain the text content include: Perform image enhancement on medical form images; Performing character recognition on the medical form image based on an optical character recognition algorithm to extract text content; wherein the text content includes extracted character information and character box candidate information; The text content is optimized for layout adjustment based on an optical character recognition adaptive optimization algorithm; wherein the text content includes: a text segmentation frame and a text height.

3. The method according to claim 2, characterized in that The image enhancement processing includes at least one of the following: brightness contrast adjustment, noise removal, sharpening, color correction, and document correction.

4. The method according to claim 2, characterized in that: The step of optimizing the layout of the text content based on an optical character recognition adaptive optimization algorithm includes: The position and size of the text segmentation frame are evaluated based on an optical character recognition adaptive optimization algorithm, and the positioning of the text block is adjusted based on the evaluation result so that the text content is consistent with the layout of the medical form image.

5. The method according to claim 1, characterized in that The step of constructing a regular expression of the text content comprises: Selecting multiple sample documents from a to-be-processed document library, and marking multiple to-be-extracted fields of the multiple sample documents; A regular expression of the text content is constructed based on the multiple fields to be extracted.

6. The method according to claim 5, characterized in that The step of constructing a regular expression of the text content based on the multiple fields to be extracted includes: Inputting a plurality of the fields to be extracted into a large language model, and outputting a basic regular expression of the text content; Determining a feedback result of the basic regular expression during the application of the basic regular expression; The feedback result is input into the large language model, and a universal regular expression of the text content is output.

7. The method according to claim 6, characterized in that The steps of adjusting and optimizing the basic regular expression include at least one of the following: The matching accuracy of the basic regular expression is enhanced, the character set of the basic regular expression is determined, and the character length of the basic regular expression is determined.

8. The method according to claim 1, characterized in that The step of extracting information about the text content based on the regular expression comprises: The regular expression is integrated into a data processing flow through a script to extract fields from the text content as information of the text content.

9. The method according to claim 8, characterized in that The step of integrating the regular expression into the data processing flow through a script to extract specific fields from the text content includes: encoding the regular expression as a pattern in a script; The encoded pattern is applied to the text content, and specific fields are extracted from the text content based on a target function.

10. A medical form information extraction device, characterized in that: The device comprises: A character recognition and layout adjustment module is used to perform character recognition and layout adjustment optimization on the medical form image based on an optical character recognition algorithm to obtain text content; The text content information extraction module is used to construct a regular expression of the text content and extract information of the text content based on the regular expression.