Method, apparatus, device, and storage medium for inputting medical information

Abstract

The present invention relates to the field of artificial intelligence, and discloses a method, device, equipment and storage medium for entering medical information, which is used to solve the technical problems of low efficiency and easy errors in the identification and entry of medical information in the prior art. The method includes: obtaining an initial image of the medical information to be entered and performing image correction to obtain a corrected image; parsing the image size of the corrected image, calling an identification code detection model to detect the position of the identification code on the corrected image, and determining the position coordinates of the identification code; generating a region box to be recognized according to the image size and the position coordinates, and segmenting the corrected image to obtain a set of images to be recognized; calling an optical character recognition model to recognize the set of images to be recognized to obtain a recognition result and a corresponding confidence level; extracting the recognition results with a confidence level greater than a confidence threshold, and entering the medical information according to the recognition results. In addition, the present invention also relates to blockchain technology, and the medical information can be stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular, to a method, device, equipment, and storage medium for inputting medical information. Background Art

[0002] With the development of the social economy, the improvement of people's living standards, and the continuous improvement of the basic medical security system for urban and rural residents, people's medical needs, especially the demand for physical examination services, are increasing day by day. In order to meet the physical examination needs of different groups in society, hospitals need to strengthen resource management, improve the quality of physical examination services, optimize the physical examination operation process of medical staff, and provide user-friendly physical examination management to achieve functions such as intelligent diagnosis and treatment and remote consultation.

[0003] In the existing technology, in operations such as the establishment and management of intelligent health records, medical staff may need to manually fill in the basic information and test information of each user on the physical examination client information form and affix the identification code of the physical examination institution; after the tests are completed, the obtained information also needs to be formatted and stored manually, which is prone to errors and low efficiency. Summary of the Invention

[0004] The main objective of the present invention is to solve the technical problems of low efficiency and easy occurrence of errors when identifying and inputting medical information in the existing technology.

[0005] The first aspect of the present invention provides a method for inputting medical information, including: obtaining an initial image of the medical information to be input; performing image correction on the initial image to obtain a corrected image corresponding to the initial image; analyzing the image size of the corrected image, calling a preset identification code detection model to detect the position of the identification code on the corrected image, and determining the position coordinates of the identification code; generating a region of interest (ROI) box based on the image size and the position coordinates; segmenting the corrected image according to the ROI box to obtain a set of images to be recognized; inputting the set of images to be recognized into a preset optical character recognition model for text content recognition to obtain a recognition result and a confidence level corresponding to the recognition result; extracting the recognition results with a confidence level greater than a preset confidence threshold, and inputting the medical information according to the recognition results.

[0006] Optionally, in the first implementation manner of the first aspect of the present invention, the correcting the initial image to obtain a corrected image corresponding to the initial image includes: inputting the initial image into a direction correction tool to determine the direction of the image content, and adjusting the direction of the image content based on the determination result to obtain a first image in a preset direction; inputting the first image into a preset image segmentation tool to segment irrelevant content, and obtaining a second image with the background removed; detecting whether the second image is tilted, and if so, inputting the second image into a preset tilt correction tool to perform perspective transformation to obtain the corrected image corresponding to the initial image.

[0007] Optionally, in the second implementation manner of the first aspect of the present invention, the direction correction tool includes a feature extraction layer and a fine-grained classification layer. The inputting the initial image into the direction correction tool to determine the direction of the image content, and adjusting the direction of the image content based on the determination result to obtain a first image in a preset direction includes: inputting the initial picture into the feature extraction layer to extract features, and obtaining initial feature information, where the feature extraction layer is established based on the DenseNet tool; inputting the initial feature information into the fine-grained classification layer to classify the picture direction, and obtaining the direction category of the initial picture, where the fine-grained classification layer is established based on the DFL fine-grained classification network; rotating the initial picture to the preset direction according to the direction category to obtain the first image.

[0008] Optionally, in the third implementation manner of the first aspect of the present invention, the inputting the first image into a preset image segmentation tool to segment irrelevant content and obtaining a second image with the background removed includes: identifying the foreground and background of the first image to obtain the identification results of the foreground and background; generating a binary image of the first image according to the identification results; multiplying the binary image by the first image in a matrix to obtain the second image with the background removed.

[0009] Optionally, in the fourth implementation manner of the first aspect of the present invention, the inputting the second image into a preset tilt correction tool to perform perspective transformation to obtain the corrected image corresponding to the initial image includes: calling the Canny operator in the preset tilt correction tool to detect the image edges in the second image; performing Hough transformation on the image edges to detect the straight line segments in the image edges; positioning the straight line intersection coordinates according to the straight line segments; performing perspective transformation on the second image according to the straight line intersection coordinates to obtain the corrected image.

[0010] Optionally, in the fifth implementation manner of the first aspect of the present invention, the step of inputting the to-be-recognized image set into a pre-set optical character recognition model to recognize the text content, obtaining a recognition result and a confidence level corresponding to the recognition result includes: detecting the text positions in each to-be-recognized image in the to-be-recognized image set to obtain the coordinates of the text positions; cropping each to-be-recognized image according to the coordinates of the text positions to obtain at least one text image slice; performing equal ratio scaling on at least one of the image slices to obtain at least one scaled image slice with the same short side length; performing text recognition on at least one of the scaled image slices to obtain a recognition result, and outputting the confidence level corresponding to the recognition result.

[0011] Optionally, in the sixth implementation manner of the first aspect of the present invention, before obtaining the initial image of the medical information to be entered, it further includes: obtaining a template image of the initial image of the medical information to be entered; obtaining common characters to create a character dictionary, and calling a text generation tool to generate at least one test text in different fonts; generating a test picture set based on the test text and the template image, and training a pre-set original optical recognition network according to the test picture set to obtain an optical character recognition model.

[0012] The second aspect of the present invention provides a medical information input device, including: an acquisition module, configured to acquire an initial image of medical information to be entered; a correction module, configured to perform image correction on the initial image to obtain a corrected image corresponding to the initial image; a positioning module, configured to analyze the image size of the corrected image, and call a pre-set identification code detection model to detect the position of the identification code on the corrected image to determine the position coordinates of the identification code; a region delineation module, configured to generate a to-be-recognized region box according to the image size and the position coordinates; a region segmentation module, configured to segment the corrected image according to the to-be-recognized region box to obtain a to-be-recognized image set; an identification module, configured to input the to-be-recognized image set into a pre-set optical character recognition model to recognize the text content, obtaining a recognition result and a confidence level corresponding to the recognition result; and an input module, configured to extract the recognition results with confidence levels greater than a preset confidence threshold, and enter the medical information according to the recognition results.

[0013] Optionally, in the first implementation manner of the second aspect of the present invention, the correction module includes: a direction adjustment unit, configured to input the initial image into a direction correction tool to determine the direction of the image content, and adjust the direction of the image content based on the determination result to obtain a first image in a preset direction; a background segmentation unit, configured to input the first image into a preset image segmentation tool to segment irrelevant content, and obtain a second image with the background removed; a perspective transformation unit, configured to detect whether the second image is tilted. If so, input the second image into a preset tilt correction tool for perspective transformation to obtain a corrected image corresponding to the initial image.

[0014] Optionally, in the second implementation manner of the second aspect of the present invention, the direction adjustment unit includes: a feature extraction subunit, configured to input the initial picture into the feature extraction layer to extract feature information, where the feature extraction layer is established based on the DenseNet tool; a direction classification subunit, configured to input the initial feature information into the fine-grained classification layer to classify the picture direction, and obtain the direction category of the initial picture, where the fine-grained classification layer is established based on the DFL fine-grained classification network; a rotation processing subunit, configured to rotate the initial picture to a preset direction according to the direction category to obtain a first image.

[0015] Optionally, in the third implementation manner of the second aspect of the present invention, the background segmentation unit includes: a background recognition subunit, configured to recognize the foreground and background of the first image to obtain the recognition results of the foreground and background; a binary image generation subunit, configured to generate a binary image of the first image according to the recognition results; a background removal subunit, configured to multiply the binary image by the first image in matrix form to obtain a second image with the background removed.

[0016] Optionally, in the fourth implementation manner of the second aspect of the present invention, the perspective transformation unit includes: an edge detection subunit, configured to call the Canny operator in a preset tilt correction tool to detect the image edge in the second image; a Hough transform subunit, configured to perform a Hough transform on the image edge to detect straight line segments in the image edge; an intersection coordinate positioning subunit, configured to locate the straight line intersection coordinates according to the straight line segments; a transformation subunit, configured to perform perspective transformation on the second image according to the straight line intersection coordinates to obtain a corrected image.

[0017] Optionally, in the fifth implementation manner of the second aspect of the present invention, the recognition module includes: a text position detection unit, configured to detect the text positions in each to-be-recognized image in the to-be-recognized image set to obtain the coordinates of the text positions; an image cropping unit, configured to crop each to-be-recognized image according to the coordinates of the text positions to obtain at least one text image piece; a scaling unit, configured to scale at least one of the image pieces proportionally to obtain at least one scaled image piece with the same short side length; and a text recognition unit, configured to perform text recognition on at least one of the scaled image pieces to obtain a recognition result and output the confidence corresponding to the recognition result.

[0018] Optionally, in the sixth implementation manner of the second aspect of the present invention, the medical information input device further includes a model training module, which is specifically configured to: obtain a template image of an initial image of the medical information to be input; obtain common characters to create a character dictionary, and call a text generation tool to generate at least one test text in different fonts; generate a test picture set based on the test text and the template image, and train a preset original optical recognition network according to the test picture set to obtain an optical character recognition model.

[0019] The third aspect of the present invention provides a medical information input device, including: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory to enable the medical information input device to execute the steps of the above-mentioned medical information input method.

[0020] The fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when it runs on a computer, it enables the computer to execute the steps of the above-mentioned medical information input method.

[0021] In the technical solution provided by the present invention, an initial image of the medical information to be input is obtained; the initial image is corrected to obtain a corrected image corresponding to the initial image; the image size of the corrected image is analyzed, and a preset identification code detection model is called to detect the position of the identification code on the corrected image to determine the position coordinates of the identification code; a to-be-recognized region box is generated according to the image size and the position coordinates; the corrected image is segmented according to the to-be-recognized region box to obtain a to-be-recognized image set; the to-be-recognized image set is input into a preset optical character recognition model to perform text content recognition to obtain a recognition result and the confidence corresponding to the recognition result; the recognition results with confidence greater than a preset confidence threshold are extracted, and the medical information is input according to the recognition results. In the embodiments of the present invention, the position of the recognition region of the to-be-recognized initial image is determined, the text content is recognized, and the medical information is input according to the recognition results, improving the efficiency and accuracy of medical information input. Description of the Drawings

[0022] Figure 1 Schematic diagram of the first embodiment of the medical information input method in the embodiments of the present invention;

[0023] Figure 2 Schematic diagram of the second embodiment of the medical information input method in the embodiments of the present invention;

[0024] Figure 3 Schematic diagram of the third embodiment of the medical information input method in the embodiments of the present invention;

[0025] Figure 4 Schematic diagram of the fourth embodiment of the medical information input method in the embodiments of the present invention;

[0026] Figure 5 Schematic diagram of an embodiment of the medical information input device in the embodiments of the present invention;

[0027] Figure 6 Schematic diagram of another embodiment of the medical information input device in the embodiments of the present invention;

[0028] Figure 7 Schematic diagram of an embodiment of the medical information input device in the embodiments of the present invention. Detailed implementation manners

[0029] The embodiments of the present invention provide a method for obtaining an initial image of medical information to be input; performing image correction on the initial image to obtain a corrected image corresponding to the initial image; analyzing the image size of the corrected image, calling a pre-set identification code detection model to detect the position of the identification code on the corrected image, and determining the position coordinates of the identification code; generating a region to be recognized frame according to the image size and the position coordinates of the identification code; segmenting the corrected image according to the region to be recognized frame to obtain a set of images to be recognized; inputting the set of images to be recognized into a pre-set optical character recognition model to perform text content recognition, obtaining a recognition result and a confidence level corresponding to the recognition result; extracting the recognition results with a confidence level greater than a confidence threshold, and inputting the medical information according to the recognition results. In the embodiments of the present invention, the position of the recognition region of the initial image to be recognized is determined, the text content is recognized, and the medical information is input according to the recognition results, improving the efficiency and accuracy of medical information input.

[0030] In the description and claims of the present invention and the above drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] For ease of understanding, the specific process of the embodiments of the present invention will be described below. Please refer to Figure 1 , an embodiment of the medical information entry method in the embodiments of the present invention includes:

[0032] 101. Obtain an initial image of the medical information to be entered;

[0033] It can be understood that the execution subject of the present invention can be a medical information entry device, or a terminal or a server. Specifically, no limitation is made here. The embodiments of the present invention are described by taking the server as the execution subject as an example. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0034] The information recognition method in this embodiment is specifically intended to recognize the specific information in the form file to avoid the problems of errors and low efficiency in manual recognition. Specifically, when performing information recognition, the user can obtain an initial image of the medical information to be entered by taking a photo or scanning. Among them, the form file described in this embodiment may contain various information items, and each information item has a corresponding identification code, which can be a barcode or a QR code. In addition, the identification code also has a positioning identifier, and the specific position of the identification code can be determined by recognizing the positioning identifier subsequently.

[0035] 102. Perform image correction on the initial image to obtain a corrected image corresponding to the initial image;

[0036] In this embodiment, to improve the accuracy of subsequent recognition, after obtaining the initial image, the initial image is first corrected. Specifically, in this embodiment, the direction of the initial image is first recognized, and the initial image is rotated to unify the direction of the image. Subsequently, irrelevant background information in the image is removed. Subsequently, the edge lines in the image with the background removed are detected, and perspective transformation is performed to obtain the corrected table image.

[0037] 103. Analyze the image size of the corrected image, and call the pre-set identification code detection model to detect the position of the identification code on the corrected image to determine the position coordinates of the identification code.

[0038] Obtain the image size of the corrected image, and use the pre-set identification code detection model to detect the position of the unit identification code on the corrected image. Among them, the identification code detection model in this step can recognize the unit identification code on the corrected image, and determine the specific position of each unit identification code based on the positioning marks on the unit identification code.

[0039] 104. Generate a region-of-interest (ROI) box according to the image size and position coordinates.

[0040] Specifically, in the table to be information-recognized in this embodiment, according to different specific items or samples detected, there may be one or more unit identification codes on a table. According to different specific formats, a piece of detection information may be located below or to the right of the unit identification code. When the detection information is located below the unit identification code, the width of the unit identification code in this step is the same as the width of the detection information content box. When the detection information is located to the right of the unit identification code, the width of the unit identification code in this step is the same as the height of the detection information content box. Therefore, according to the position coordinates of the foregoing identification code and the image size, the approximate position of the ROI box is calculated.

[0041] 105. Segment the corrected image according to the ROI box to obtain a set of images to be recognized.

[0042] 106. Input the set of images to be recognized into the pre-set optical character recognition model to recognize the text content, and obtain the recognition result and the confidence level corresponding to the recognition result.

[0043] Call an image cutting tool to segment the corrected image based on the obtained ROI box to obtain at least one image to be recognized, and form a set of images to be recognized with all the obtained images to be recognized.

[0044] Input each image to be recognized in the set of images to be recognized into the pre-set optical character recognition model. Among them, the optical character recognition model in this step is a handwritten OCR (Optical Character Recognition) model.

[0045] Specifically, the handwritten OCR model is established in advance through a deep neural network tool, which can extract image features in the image to be recognized through the deep neural network algorithm in the deep neural network tool, and determine the coordinates of the handwritten text field according to the image features; subsequently, determine the text image block to be recognized according to the coordinates of the handwritten text field, extract the image features in the text image block, and perform text content recognition based on the extracted image features in the text image block. Among them, when performing text content recognition, the fully connected layer in the deep neural network algorithm can be specifically used for specific classification and recognition, and the confidence corresponding to the recognition result is output at the same time.

[0046] 107. Extract the recognition results with confidence greater than the preset confidence threshold, and enter the medical information according to the recognition results.

[0047] In this embodiment, a confidence threshold is preset in advance, and the recognition results corresponding to the confidence of the foregoing recognition results that exceed or are equal to the confidence threshold are extracted, marked as recognition success, and the content of the recognition success is output; the recognition results with confidence not exceeding the confidence threshold are used as uncertain results, and their possible information content is output according to the recognition results and marked for subsequent processing.

[0048] After obtaining the recognition result content output after successful recognition, convert the recognized medical information content into computer-readable text according to the recognition result content and enter it into a spreadsheet to complete the entry of medical information.

[0049] In the embodiment of the present invention, the position of the recognition area of the initial image to be recognized is determined, the text content is recognized, and the medical information is entered according to the recognition result, improving the efficiency and accuracy of medical information entry.

[0050] Please refer to Figure 2 , the second embodiment of the medical information entry method in the embodiment of the present invention includes:

[0051] 201. Obtain the initial image of the medical information to be entered;

[0052] The content in this step is basically the same as the content in step 101 in the foregoing embodiment, so it will not be repeated here.

[0053] 202. Input the initial image into a direction correction tool to judge the direction of the image content, and adjust the direction of the image content based on the judgment result to obtain a first image with a preset direction;

[0054] 203. Input the first image into a preset image segmentation tool to segment irrelevant content to obtain a second image with the background removed;

[0055] 204. Detect whether the second image is tilted;

[0056] 205. If so, input the second image into a preset tilt correction tool for perspective transformation to obtain a corrected image corresponding to the initial image;

[0057] In this embodiment, first, a direction correction tool is called to determine the direction of the initial image based on the content in the initial image, obtaining a direction determination result. Then, according to the direction determination result, the image in different directions is rotated by a corresponding angle, so as to unify the direction of the image and obtain a first image in a preset direction for subsequent recognition.

[0058] Subsequently, to improve the subsequent recognition accuracy, the irrelevant background noise in the first image obtained after unifying the direction is removed. Specifically, a preset image segmentation tool is used to segment the irrelevant content to obtain a second image with the background removed.

[0059] Detect whether there is a tilted situation in the second image with the background removed. If not, no operation is performed; if tilted, call a preset tilt correction tool to perform perspective transformation on the tilted second image to obtain a corrected image.

[0060] 206. Analyze the image size of the corrected image, call a preset identification code detection model to detect the position of the identification code on the corrected image, and determine the position coordinates of the identification code;

[0061] Obtain the image size of the corrected image, and use a preset identification code detection model to detect the position of the unit identification code on the corrected image. Among them, in this step, the identification code detection model can identify the unit identification code on the corrected image and determine the specific position of each unit identification code based on the positioning marks on the unit identification code.

[0062] Specifically, a preset identification code detection model established in advance according to the YOLO V5 model can be called to locate the four corner coordinates of the unit identification code on the corrected image.

[0063] 207. Generate a region box to be recognized according to the image size and position coordinates;

[0064] In this embodiment, in the table to be subjected to information recognition, there may be one or more unit identification codes on a single table according to different specific items or samples to be detected; according to different specific formats, a piece of detection information may be located below or to the right of the unit identification code. When the detection information is located below the unit identification code, the width of the unit identification code in this step is the same as the width of the detection information content frame; when the detection information is located to the right of the unit identification code, the width of the unit identification code in this step is the same as the height of the detection information content frame. Therefore, according to the position coordinates of the foregoing identification code and the image size, the approximate position of the area frame to be recognized is calculated; among them, the four corner coordinates of the image are obtained by acquiring the image size of the corrected image, and the four corner coordinates of the unit identification code obtained according to the foregoing pre-set identification code detection model established based on the YOLO V5 model are used to generate the area frame to be recognized.

[0065] 208. Segment the corrected image according to the area frame to be recognized to obtain a set of images to be recognized;

[0066] Call an image cutting tool to segment the corrected image based on the area frame to be recognized obtained above to obtain at least one image to be recognized, and form a set of images to be recognized with all the obtained images to be recognized.

[0067] 209. Input the set of images to be recognized into a pre-set optical character recognition model to perform text content recognition, and obtain a recognition result and a confidence level corresponding to the recognition result;

[0068] Input each image to be recognized in the set of images to be recognized into a pre-set optical character recognition model. Among them, the optical character recognition model described in this step is a handwritten OCR (Optical Character Recognition) model, which can detect handwritten characters in the image to be recognized and identify the coordinates of the fields, extract the features of the characters with the obtained coordinates, and then perform text content determination and recognition based on the extracted features to obtain a recognition result and the confidence level corresponding to the recognition result.

[0069] 210. Extract the recognition results with a confidence level greater than a pre-set confidence threshold, and enter medical information according to the recognition results.

[0070] Extract the recognition results corresponding to the foregoing confidence levels that exceed or are equal to the pre-set confidence threshold, label them as recognized successfully, and output the content recognized successfully; regard the recognition results with confidence levels that do not exceed the confidence threshold as uncertain results, output their possible information content according to the recognition results, and make annotations for subsequent processing.

[0071] After obtaining the recognition result content output after successful recognition, convert the recognized medical information content into computer-readable text according to the recognition result content and enter it into a spreadsheet to complete the entry of medical information.

[0072] In the embodiment of the present invention, the position of the recognition area of the initial image to be recognized is determined, the text content is recognized, and the medical information is entered according to the recognition result, which improves the efficiency and accuracy of medical text entry.

[0073] Please refer to Figure 3 , the third embodiment of the method for entering medical information in the embodiment of the present invention includes:

[0074] 301. Obtain the initial image of the medical information to be entered;

[0075] The content in this step is basically the same as the content in step 101 in the foregoing embodiment, so it will not be repeated here.

[0076] 302. Input the initial picture into the feature extraction layer for feature extraction to obtain initial feature information;

[0077] In this step, first input the obtained initial picture into the feature extraction layer in the direction correction tool for feature extraction; the feature extraction layer is established based on the DenseNet (Densely connected convolutional networks) network, and specifically, DenseNet121 can be used; this network connects each layer to every other layer in a feed-forward manner, alleviating the problem of gradient disappearance, enhancing feature propagation, encouraging function reuse, and reducing the number of parameters.

[0078] 303. Input the initial feature information into the fine-grained classification layer for classifying the direction of the picture to obtain the direction category of the initial picture;

[0079] Subsequently, input the output initial feature information into the fine-grained (DFL-CNN) classification layer in the direction correction tool, perform non-maximum suppression selection preprocessing on the obtained initial feature information to obtain a preprocessed initial feature energy map, and for this preprocessed initial feature energy map, input the obtained initial feature energy map into a fully convolutional network with a convolution kernel of 1 for classification processing, and divide the initial picture into four direction categories according to its specific direction, specifically into 0-degree direction, 90-degree direction, 180-degree direction, and 270-degree direction.

[0080] 304. Rotate the initial picture to a preset direction according to the direction category to obtain the first image;

[0081] According to specific direction categories, perform corresponding rotation operations on those belonging to the 90-degree direction, 180-degree direction, and 270-degree direction, so that their directions are changed to the preset direction, that is, the 0-degree direction described in this embodiment, to obtain the first image in the preset direction, where the first image is at least one.

[0082] 305. Identify the foreground and background of the first image to obtain the recognition results of the foreground and background;

[0083] 306. Generate a binary image of the first image according to the recognition results;

[0084] 307. Multiply the binary image and the first image matrix-wise to obtain the second image with the background removed;

[0085] In this step, the preset image segmentation tool includes a DA-Net segmentation layer, a softmax layer, and a matrix processing layer. Specifically, first input the first image into the DA-Net (Dual Attention Network) segmentation layer to extract image features, identify the pixels where the foreground and background are located in the image according to the extracted features, input the obtained features into the softmax layer for classification, classify each pixel according to the recognition results, label the pixels belonging to the foreground as 1, and label the pixels belonging to the background as 0; generate the binary image corresponding to the first image based on the obtained labeling results, and multiply the obtained binary image and the corresponding first image matrix-wise. The result obtained is the second image with background noise removed.

[0086] Among them, the background noise described in this step is the content other than the information table obtained when taking or scanning the first picture, such as images of irrelevant items like the floor and table. Remove these content parts to improve the accuracy of information recognition in this embodiment.

[0087] 308. Detect whether the second image is tilted;

[0088] 309. If so, call the Canny operator in the preset tilt correction tool to detect the image edges in the second image;

[0089] In this embodiment, call the preset tilt correction tool to perform perspective transformation on the second image. First, call the Canny edge detection operator in the preset tilt correction tool to detect and process the second image.

[0090] Specifically, first, apply Gaussian filtering to the second image to smooth the image and remove noise. Subsequently, calculate the gradient of each pixel point in the smoothed second image through convolution operation. Apply the non-maximum suppression technique to eliminate false detections. Subsequently, use the double-threshold technique and boundary tracking to obtain the image edges of the second image.

[0091] 310. Perform a Hough transform on the image edges to detect straight line segments in the image edges.

[0092] 311. Locate the coordinates of the straight line intersections based on the straight line segments.

[0093] 312. Perform a perspective transformation on the second image according to the coordinates of the straight line intersections to obtain a corrected image.

[0094] Perform a Hough transform on the obtained image edges to detect the straight line segments contained in the image edges. Among them, the Hough transform is a feature detection method that can detect features in an object. After obtaining the straight line segments in the image edges, locate the coordinates of the straight line intersections according to the straight line segments, and perform a perspective transformation on the second image according to the specific positions of the calculated coordinates of the straight line intersections to obtain a corrected image. In this way, it is possible to prevent the problem that the recognition effect is poor due to the distortion of the text information caused by the angle tilt during shooting or scanning.

[0095] 313. Analyze the image size of the corrected image, call the pre-set identification code detection model to detect the position of the identification code on the corrected image, and determine the position coordinates of the identification code.

[0096] 314. Generate a region-of-interest box to be recognized according to the image size and the position coordinates.

[0097] 315. Segment the corrected image according to the region-of-interest box to be recognized to obtain a set of images to be recognized.

[0098] 316. Input the set of images to be recognized into the pre-set optical character recognition model to recognize the text content, and obtain the recognition result and the confidence level corresponding to the recognition result.

[0099] 317. Extract the recognition results with a confidence level greater than the pre-set confidence threshold, and enter the medical information according to the recognition results.

[0100] In this embodiment, the content of steps 313 to 317 is basically the same as that of steps 206 to 210 in the foregoing embodiment, so it will not be elaborated here.

[0101] In the embodiment of the present invention, the position of the recognition region of the initial image to be recognized is determined, the text content is recognized, and the medical information is entered according to the recognition result, so as to improve the efficiency and accuracy of the medical information text entry.

[0102] Please refer to Figure 4 , the fourth embodiment of the medical information input method in the embodiments of the present invention includes:

[0103] 401. Obtain an initial image of the medical information to be input;

[0104] In the embodiment, the information recognition method is specifically used to recognize specific information in the form file to avoid the problems of manual recognition errors and low efficiency. Specifically, when performing information recognition, the user can obtain the initial image of the medical information to be input by taking pictures or scanning. Among them, the form file described in this embodiment may contain various information items, and each information item has a corresponding identification code, which can be a barcode or a QR code. In addition, the identification code also has a positioning identifier, and the specific position of the identification code can be determined by recognizing the positioning identifier later.

[0105] In addition, before this step, first, a template image of the initial image of the medical information to be input needs to be obtained; and a character dictionary is made by obtaining common characters, and at least one test text of different fonts is generated by calling a text generation tool. Among them, the system dictionary contains 2,500 common Chinese characters, 26 English letters, and Arabic numerals 0-9. The text generation tool can be built based on the SynthText script; a test picture set is generated based on the test text and the template image. Specifically, the obtained test text of different fonts is subjected to a certain degree of distortion processing, and the distorted test text is embedded in the template image to obtain a large test picture. Noise is added to the test picture, and a test picture set is formed. The preset original optical recognition network is trained according to the test picture set to obtain an optical character recognition model.

[0106] 402. Input the initial picture into the feature extraction layer for feature extraction to obtain initial feature information;

[0107] 403. Input the initial feature information into the fine-grained classification layer for classifying the picture direction to obtain the direction category of the initial picture;

[0108] 404. Rotate the initial picture to a preset direction according to the direction category to obtain a first image;

[0109] 405. Recognize the foreground and background of the first image to obtain the recognition results of the foreground and background;

[0110] 406. Generate a binary image of the first image according to the recognition results;

[0111] 407. Multiply the binary image and the first image in matrix to obtain a second image with the background removed;

[0112] 408. Detect whether the second image is tilted;

[0113] 409. If so, call the Canny operator in the preset tilt correction tool to detect the image edges in the second image;

[0114] 410. Perform Hough transform on the image edges to detect the straight line segments in the image edges;

[0115] 411. Locate the coordinates of the straight line intersection points according to the straight line segments;

[0116] 412. Perform perspective transformation on the second image according to the coordinates of the straight line intersection points to obtain a corrected image;

[0117] In this embodiment, the content in steps 402 - 412 is basically the same as the content in steps 302 - 312 in the foregoing embodiment, so it will not be elaborated here.

[0118] 413. Analyze the image size of the corrected image, call the preset identification code detection model to detect the position of the identification code on the corrected image, and determine the position coordinates of the identification code;

[0119] 414. Generate a region-of-interest box to be recognized according to the image size and the position coordinates;

[0120] 415. Segment the corrected image according to the region-of-interest box to be recognized to obtain a set of images to be recognized;

[0121] In this embodiment, the content in steps 413 - 415 is basically the same as the content in steps 206 - 208 in the foregoing embodiment, so it will not be elaborated here.

[0122] 416. Detect the text positions in each image to be recognized in the set of images to be recognized to obtain the coordinates of the text positions;

[0123] In this embodiment, an optical character recognition model is called to recognize the text content in the set of images to be recognized. Among them, the optical character recognition model in this embodiment is composed of a detection model and a recognition model; the detection model detects the text positions in each image to be recognized in the set of images to be recognized to obtain the coordinates of the text positions. Specifically, the detection model in this step can be established based on ABCNet (Adaptive Bezier-Curve Network), and it can adaptively implement scene text detection of arbitrary shapes through simple and effective Bezier curves to obtain the coordinates of the edges of the text positions in each image to be recognized in the set of images to be recognized.

[0124] 417. Crop each image to be recognized according to the coordinates of the text positions to obtain at least one text image patch;

[0125] 418. Scale at least one image patch proportionally to obtain at least one scaled image patch with the same short side length;

[0126] Crop the image to be recognized according to the coordinates of the edges of the obtained text position to obtain at least one text image patch in the set of images to be recognized. Subsequently, scale the obtained at least one text image patch proportionally so that the short side of the text image patch is scaled to 720 pixels to obtain scaled image patches with the same short side length.

[0127] 419. Perform text recognition on at least one scaled image patch to obtain a recognition result and output the confidence level corresponding to the recognition result;

[0128] Input the scaled image patch into the recognition model for text recognition. The recognition model is established through a deep learning algorithm, and specifically, it can be established based on the CRNN (Convolutional Recurrent Neural Network) and Attention mechanisms. The recognition model includes two parts: feature extraction and result classification. After the scaled image patch is input into the recognition model, feature extraction is first performed to generate image patch feature information. Subsequently, the image patch feature information is classified according to the fully connected layer and the softmax layer, and at the same time, the confidence level of the recognized text result obtained by classification is calculated to obtain the recognized text content and the confidence level of the recognized text result.

[0129] 420. Extract the recognition results with confidence levels greater than the preset confidence threshold and enter the medical information according to the recognition results.

[0130] In this embodiment, a confidence threshold is preset in advance. Extract the recognition results corresponding to the aforementioned confidence levels that exceed or are equal to the confidence threshold, label them as recognition successes, and output the content of the recognition successes; regard the recognition results with confidence levels that do not exceed the confidence threshold as uncertain results, output their possible information content according to the recognition results, and make annotations for subsequent processing.

[0131] After obtaining the content of the recognition result output after successful recognition, convert the recognized medical information content into computer-readable text according to the content of the recognition result and enter it into a spreadsheet to complete the entry of medical information.

[0132] In the embodiment of the present invention, the position of the recognition area of the initial image to be recognized is determined, the text content is recognized, and the medical information is entered according to the recognition result, improving the efficiency and accuracy of medical information text entry.

[0133] The method for entering medical information in the embodiments of the present invention has been described above. Next, the device for entering medical information in the embodiments of the present invention will be described. Please refer to Figure 5 One embodiment of the device for entering medical information in the embodiments of the present invention includes:

[0134] An acquisition module 501, configured to acquire an initial image of medical information to be entered;

[0135] A correction module 502, configured to perform image correction on the initial image to obtain a corrected image corresponding to the initial image;

[0136] A positioning module 503, configured to analyze the image size of the corrected image, call a preset identification code detection model to perform position detection on the identification code on the corrected image, and determine the position coordinates of the identification code;

[0137] A region delineation module 504, configured to generate a region to be recognized frame according to the image size and the position coordinates;

[0138] A region segmentation module 505, configured to segment the corrected image according to the region to be recognized frame to obtain a set of images to be recognized;

[0139] An identification module 506, configured to input the set of images to be recognized into a preset optical character recognition model to perform text content recognition, and obtain a recognition result and a confidence level corresponding to the recognition result;

[0140] An entry module 507, configured to extract the recognition results with a confidence level greater than a preset confidence threshold, and enter the medical information according to the recognition results.

[0141] In the embodiments of the present invention, the position of the recognition region of the initial image to be recognized is determined, text content is recognized, and the medical information is entered according to the recognition results, improving the efficiency and accuracy of medical information entry.

[0142] Please refer to Figure 6 Another embodiment of the device for entering medical information in the embodiments of the present invention includes:

[0143] An acquisition module 501, configured to acquire an initial image of medical information to be entered;

[0144] A correction module 502, configured to perform image correction on the initial image to obtain a corrected image corresponding to the initial image;

[0145] A positioning module 503, configured to analyze the image size of the corrected image, call a preset identification code detection model to perform position detection on the identification code on the corrected image, and determine the position coordinates of the identification code;

[0146] An area delimitation module 504, configured to generate a region box to be recognized according to the image size and the position coordinates;

[0147] An area segmentation module 505, configured to segment the corrected image according to the region box to be recognized, to obtain a set of images to be recognized;

[0148] A recognition module 506, configured to input the set of images to be recognized into a preset optical character recognition model to perform text content recognition, to obtain a recognition result and a confidence level corresponding to the recognition result;

[0149] An input module 507, configured to extract the recognition results with the confidence levels greater than a preset confidence threshold, and input medical information according to the recognition results.

[0150] Optionally, the correction module 502 includes:

[0151] A direction adjustment unit 5021, configured to input the initial image into a direction correction tool to perform direction judgment on the image content, and adjust the direction of the image content based on the judgment result, to obtain a first image with a preset direction;

[0152] A background segmentation unit 5022, configured to input the first image into a preset image segmentation tool to perform segmentation of irrelevant content, to obtain a second image with the background removed;

[0153] A perspective transformation unit 5023, configured to detect whether the second image is tilted, and if so, input the second image into a preset tilt correction tool to perform perspective transformation, to obtain the corrected image corresponding to the initial image.

[0154] Optionally, the direction adjustment unit 5021 includes:

[0155] A feature extraction subunit, configured to input the initial picture into the feature extraction layer to perform feature extraction, to obtain initial feature information, where the feature extraction layer is established based on the DenseNet tool;

[0156] A direction classification subunit, configured to input the initial feature information into the fine-grained classification layer to perform picture direction classification, to obtain the direction category of the initial picture, where the fine-grained classification layer is established based on the DFL fine-grained classification network;

[0157] A rotation processing subunit, configured to rotate the initial picture to a preset direction according to the direction category, to obtain a first image.

[0158] Optionally, the background segmentation unit 5022 includes:

[0159] A background recognition subunit, configured to recognize the foreground and background of the first image to obtain the recognition results of the foreground and background;

[0160] A binary image generation subunit, configured to generate a binary image of the first image according to the recognition results;

[0161] A background removal subunit, configured to multiply the binary image and the first image in matrix form to obtain a second image with the background removed.

[0162] Optionally, the perspective transformation unit 5023 includes:

[0163] An edge detection subunit, configured to detect the image edges in the second image by invoking the Canny operator in a preset tilt correction tool;

[0164] A Hough transform subunit, configured to perform a Hough transform on the image edges to detect the straight line segments in the image edges;

[0165] An intersection coordinate positioning subunit, configured to locate the straight line intersection coordinates according to the straight line segments;

[0166] A transformation subunit, configured to perform a perspective transformation on the second image according to the straight line intersection coordinates to obtain a corrected image.

[0167] Optionally, the recognition module 506 includes:

[0168] A text position detection unit, configured to detect the text positions in each to-be-recognized image in the to-be-recognized image set to obtain the coordinates of the text positions;

[0169] An image cropping unit, configured to crop each to-be-recognized image according to the coordinates of the text positions to obtain at least one text image patch;

[0170] A scaling unit, configured to perform equal ratio scaling on at least one of the image patches to obtain at least one scaled image patch with the same short side length;

[0171] A text recognition unit, configured to perform text recognition on at least one of the scaled image patches to obtain the recognition results and output the confidence levels corresponding to the recognition results.

[0172] Optionally, the medical information entry device further includes a model training module, and the model training module is specifically configured to: obtain a template image of the initial image of the medical information to be entered; obtain common characters to create a character dictionary, and call a text generation tool to generate at least one test text in different fonts; generate a test picture set based on the test text and the template image, and train a preset original optical recognition network according to the test picture set to obtain an optical character recognition model.

[0173] In an embodiment of the present invention, the position of the recognition area of the initial image to be recognized is determined, the text content is recognized, and medical information is entered according to the recognition result, so as to improve the efficiency and accuracy of medical information entry.

[0174] Above Figure 5 And Figure 6 The medical information entry device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the medical information entry device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0175] Figure 7 FIG. is a schematic structural diagram of a medical information entry device provided by an embodiment of the present invention. The medical information entry device 700 may vary greatly due to configuration or performance differences, and may include one or more central processing units (CPUs) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 for storing application programs 733 or data 732 (for example, one or more mass storage devices). Among them, the memory 720 and the storage media 730 may be transient storage or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the medical information entry device 700. Further, the processor 710 may be configured to communicate with the storage media 730 and execute a series of instruction operations in the storage media 730 on the medical information entry device 700.

[0176] The medical information entry device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input / output interfaces 760, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 7 The shown structure of the medical information entry device does not limit the medical information entry device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0177] The present invention also provides a computer device, which may be any device capable of executing the medical information entry method described in the above embodiments. The computer device includes a memory and a processor. When the computer-readable instructions stored in the memory are executed by the processor, the processor executes the steps of the medical information entry method in the above embodiments.

[0178] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, a series of data blocks generated by using cryptographic methods. Each data block contains information on a batch of network transactions, which is used to verify the validity of the information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer, etc.

[0179] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the method for entering medical information.

[0180] The embodiments in the present invention can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain the best results of theory, method, technology, and application system.

[0181] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0182] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0184] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for inputting medical information, characterized in that, The method for inputting medical information includes: Obtaining an initial image of the medical information to be input, where the initial image contains various information entries, and each information entry has a corresponding identification code, and the identification code has a positioning identifier; Performing image correction on the initial image to obtain a corrected image corresponding to the initial image; Analyzing the image size of the corrected image, calling a pre-set identification code detection model to detect the position of the identification code on the corrected image, and determining the position coordinates of the identification code; In the table to be recognized for information, there is one or more unit identification codes on a single table according to the specific items or samples to be detected; generating a region of interest frame based on the image size and the position coordinates; Segmenting the corrected image according to the region of interest frame to obtain a set of images to be recognized; Inputting the set of images to be recognized into a pre-set optical character recognition model to perform text content recognition, and obtaining a recognition result and the confidence level corresponding to the recognition result; Extracting the recognition results with the confidence level greater than a pre-set confidence threshold, and inputting the medical information according to the recognition results.

2. The method for inputting medical information according to claim 1, wherein, The performing image correction on the initial image to obtain a corrected image corresponding to the initial image includes: Inputting the initial image into a direction correction tool to judge the direction of the image content, and adjusting the direction of the image content based on the judgment result to obtain a first image in a preset direction; Inputting the first image into a pre-set image segmentation tool to segment irrelevant content, and obtaining a second image with the background removed; Detecting whether the second image is tilted, and if so, inputting the second image into a pre-set tilt correction tool to perform perspective transformation to obtain a corrected image corresponding to the initial image.

3. The method for inputting medical information according to claim 2, wherein The direction correction tool includes a feature extraction layer and a fine-grained classification layer. The inputting the initial image into the direction correction tool to judge the direction of the image content, and adjusting the direction of the image content based on the judgment result to obtain a first image in a preset direction includes: Inputting the initial picture into the feature extraction layer to extract features, and obtaining initial feature information, where the feature extraction layer is established based on the DenseNet tool; Inputting the initial feature information into the fine-grained classification layer to classify the direction of the picture, and obtaining the direction category of the initial picture, where the fine-grained classification layer is established based on the DFL fine-grained classification network; Rotating the initial picture to a preset direction according to the direction category to obtain a first image.

4. The method for inputting medical information according to claim 2, wherein The inputting the first image into a pre-set image segmentation tool to segment irrelevant content, and obtaining a second image with the background removed includes: Recognizing the foreground and background of the first image to obtain recognition results of the foreground and background; Generating a binary image of the first image according to the recognition results; Multiplying the binary image by the first image in matrix form to obtain a second image with the background removed.

5. The method for inputting medical information according to claim 2, wherein The inputting the second image into a pre-set tilt correction tool to perform perspective transformation to obtain a corrected image corresponding to the initial image includes: Detect the image edges in the second image by using the Canny operator in the preset tilt correction tool; Perform Hough transform on the image edges to detect the straight line segments in the image edges; Locate the straight line intersection coordinates according to the straight line segments; Perform perspective transformation on the second image according to the straight line intersection coordinates to obtain a corrected image.

6. The method for inputting medical information according to any one of claims 1-5, characterized in that, The inputting the set of images to be recognized into a preset optical character recognition model for text content recognition to obtain a recognition result and a confidence level corresponding to the recognition result includes: Detect the text positions in each image to be recognized in the set of images to be recognized to obtain the coordinates of the text positions; Crop each image to be recognized according to the coordinates of the text positions to obtain at least one text image piece; Perform equal ratio scaling on at least one of the image pieces to obtain at least one scaled image piece with the same short side length; Perform text recognition on at least one of the scaled image pieces to obtain a recognition result and output the confidence level corresponding to the recognition result.

7. The method for inputting medical information according to claim 6, wherein Before obtaining the initial image of the medical information to be entered, further includes: Obtain a template image of the initial image of the medical information to be entered; Obtain common characters to make a character dictionary, and call a text generation tool to generate at least one test text in different fonts; Generate a test image set based on the test text and the template image, and train a preset original optical recognition network according to the test image set to obtain an optical character recognition model.

8. An input device for medical information, characterized in that, The medical information entry device includes: An acquisition module, configured to acquire an initial image of the medical information to be entered, where the initial image contains various information entries, and each information entry has a corresponding identification code, and the identification code has a positioning identifier; A correction module, configured to perform image correction on the initial image to obtain a corrected image corresponding to the initial image; A positioning module, configured to analyze the image size of the corrected image, call a preset identification code detection model to perform position detection on the identification codes on the corrected image, and determine the position coordinates of the identification codes; A region delineation module, configured to have one or more unit identification codes on a table according to different specific items or samples to be detected in the table to be recognized for information; generate a region to be recognized frame according to the image size and the position coordinates; A region segmentation module, configured to segment the corrected image according to the region to be recognized frame to obtain a set of images to be recognized; A recognition module, configured to input the set of images to be recognized into a preset optical character recognition model for text content recognition to obtain a recognition result and a confidence level corresponding to the recognition result; An entry module, configured to extract the recognition results with a confidence level greater than a preset confidence threshold, and enter the medical information according to the recognition results.

9. An input device for medical information, characterized in that, The medical information entry device includes: a memory and at least one processor, and instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the medical information entry device executes the steps of the medical information entry method according to any one of claims 1-7.

10. A computer-readable storage medium, on which instructions are stored, characterized in that, When the instruction is executed by a processor, it implements the steps of the method for entering medical information according to any one of claims 1-7.

Images