Method, System, Device and Medium for Judging Medical Form Types

Abstract

The present invention relates to the field of artificial intelligence technology, and provides a method, system, device and medium for judging the type of medical form. The method for judging the type of medical form includes: obtaining an image of the medical form; through OCR extraction technology, obtaining multiple text regions to be recognized in the image of the medical form, and extracting the text information in the image of the medical form; inputting the text information into the medical form type matching model, matching the text information with each keyword in the preset type library of medical forms, obtaining multiple keywords contained in the text information, and calculating the matching rate corresponding to each type in the type library of medical forms according to the preset weights corresponding to each keyword in the text information, so as to obtain the type of the medical form. The present invention matches the text information in the medical form with the keywords in the preset type library of medical forms, and according to the different weights of different keywords, further obtains the type corresponding to the medical form. It can conveniently, efficiently and accurately identify the type of the medical form.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly relates to a method, system, device and medium for judging the type of medical documents. Background Art

[0002] After a patient sees a doctor in a hospital, there will be various types of medical documents, such as: medical records, drug lists, medical summaries, and medical invoices, etc. For insurance business personnel, in the face of a wide variety of medical documents, how to accurately and efficiently classify medical documents has become a difficult problem that urgently needs to be solved. In the prior art, after obtaining the paper materials of medical documents, insurance business personnel usually need to go to multiple different medical units, collect a certain number of medical documents, and then through the document collection function on the PC side, open the scanner to scan the paper copies, manually classify the document collection categories and enter them into the image library. This method requires manual identification of each corresponding paper copy one by one, and the identification efficiency is relatively slow. In addition, the current OCR recognition technology can usually only recognize basic text information, and can only extract the text at the corresponding position according to a single recognition model, and cannot be assembled and improved according to the dynamic rules in the industry. Therefore, in the actual application scenario, the medical documents collected by insurance salespersons often have different differentiations, and this single-position OCR recognition cannot meet the needs of the existing market.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for judging the type of medical documents, so as to at least overcome one or more problems caused by the limitations and defects of the related technologies to a certain extent.

[0005] To achieve the above object and other related objects, the present invention provides a method for judging the type of medical documents, including:

[0006] Obtain an image of a medical document;

[0007] Through OCR extraction technology, traverse the image of the medical document, obtain multiple segments of text regions to be recognized in the image of the medical document, and extract the text information in the image of the medical document according to the text features in each segment of the text region to be recognized;

[0008] Input the text information into the medical form type matching model, match the text information with each keyword in the type library of the preset medical forms to obtain multiple keywords contained in the text information, and calculate the matching rate of each type corresponding to the medical form in the type library of the medical form according to the preset weights corresponding to each keyword in the text information to obtain the type of the medical form. The medical form type matching model is composed of cascading TextMatching and a Softmax classifier. The type library of the medical form includes each pre-stored type and the keywords corresponding to each type.

[0009] Input each group of image patch data in the image patch dataset into the document recognition model to be trained for iterative training, and update the weights of the model based on the results of the iterative training to obtain a trained document recognition model. The document recognition model is composed of cascading the ResNet34 network and a Softmax classifier.

[0010] In an embodiment of the present invention, the method for traversing the image of the medical form through the OCR extraction technology to obtain multiple regions of text to be recognized in the image of the medical form and extracting the text information in the image of the medical form according to the text features in each region of text to be recognized includes:

[0011] Preprocess the image of the medical form to obtain a preprocessed medical form image;

[0012] Traverse the preprocessed medical form image through the sliding window algorithm to extract the text region of the preprocessed medical form image;

[0013] Perform rectangular segmentation on the text region of the preprocessed medical form image to obtain multiple regions of text to be recognized, where each region of text to be recognized corresponds to a paragraph in the text region of the preprocessed medical form image;

[0014] Extract the text features in each region of text to be recognized, and obtain the text information in the image of the medical form through character recognition.

[0015] In an embodiment of the present invention, the step of preprocessing the image of the medical form to obtain a preprocessed medical form image includes:

[0016] Perform grayscale processing on the image of the medical form to obtain a grayscale medical form image;

[0017] Adopt the method of dynamic threshold segmentation to perform binarization processing on the grayscale medical form image to obtain a binarized medical form image;

[0018] Using the nearest neighbor clustering method, perform skew correction processing on the binary medical form image to obtain a preprocessed medical form image.

[0019] In an embodiment of the present invention, the method of inputting the text information into the medical form type matching model, matching the text information with each keyword in the type library of preset medical forms to obtain multiple keywords contained in the text information, and calculating the matching rate of the medical form with each type corresponding to the type library of the medical form according to the preset weights corresponding to each keyword in the text information, to obtain the type of the medical form includes:

[0020] Input the text information into the medical form type matching model, and extract multiple keywords contained in the text information by matching the text information with each keyword in the type library of the medical form;

[0021] Obtain the matching rate of the medical form with each type according to the preset weights corresponding to each keyword in the text information;

[0022] Sort the matching rates of the medical form with each type, select the type with the highest matching rate with the medical form, compare the highest matching rate with a preset matching rate threshold, and if the highest matching rate is greater than or equal to the preset matching rate threshold, use the type corresponding to the highest matching rate as the type of the medical form.

[0023] In an embodiment of the present invention, after sorting the matching rates of the medical form with each type, selecting the type with the highest matching rate with the medical form, and comparing the highest matching rate with a preset matching rate threshold, it further includes:

[0024] If the highest matching rate is less than the preset matching rate threshold, add a data label to the medical form according to the type of the medical form;

[0025] Input the medical form with the data label into the medical form type matching model for training, and update the weights of the medical form type matching model by the gradient descent method.

[0026] In an embodiment of the present invention, the medical form type matching model is obtained through pre-training, and the process of the pre-training includes:

[0027] Obtain medical form data containing various types, and the types at least include: medical summaries, medical cases, medical invoices, medical lists;

[0028] Use text recognition to extract the text information of the medical form data,

[0029] Input the text information into the medical form type matching model to be trained for training. Extract the feature vector of the text information through the convolution operation of TextMatching, and send the feature vector into the Softmax classifier for mapping to obtain the medical form type matching model.

[0030] In an embodiment of the present invention, the method for obtaining the matching rate includes:

[0031] where p im is the matching rate of the i-th medical form with the m-th type in the type library of the medical form, and N im is the total number of keywords of the i-th medical form belonging to the m-th type, is the preset weight of the j-th keyword in the m-th type, is the matching value between the j-th keyword and the text information in the i-th medical form.

[0032] To achieve the above object and other related objects, the present invention further provides a system for judging the type of medical form, including:

[0033] An image acquisition module for acquiring an image of the medical form;

[0034] A text extraction module for traversing the image of the medical form through OCR extraction technology to obtain multiple regions of text to be recognized in the image of the medical form, and extracting the text information in the image of the medical form according to the text features in each region of text to be recognized;

[0035] A type judgment module for inputting the text information into the medical form type matching model, matching the text information with each keyword in the preset type library of the medical form to obtain multiple keywords contained in the text information, and calculating the matching rate of the medical form with each type in the type library of the medical form according to the preset weights corresponding to each keyword in the text information to obtain the type of the medical form. The medical form type matching model is composed of a series connection of TextMatching and a Softmax classifier. The type library of the medical form contains pre-stored various types and the corresponding keywords for each type.

[0036] To achieve the above object and other related objects, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0037] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0038] The method, system, device and medium for judging the type of medical form and document recognition of the present invention match the text information in the medical form with the keywords in the preset type library of medical forms. Among them, the type library of medical forms has different types of medical forms and multiple keyword information for each type. According to the different weights of different keywords, the corresponding type of the medical form is obtained. Thus, the type of the medical form can be identified conveniently, efficiently and accurately, effectively improving the form collection efficiency of business personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings. The drawings are schematic and should not be construed as limiting the present invention in any way. In the drawings:

[0040] Figure 1 It shows a schematic flowchart of the method for judging the type of medical form in an embodiment of the present invention;

[0041] Figure 2 It shows a schematic flowchart of step S20 in an embodiment of the present invention;

[0042] Figure 3 It shows a schematic flowchart of step S21 in an embodiment of the present invention;

[0043] Figure 4 It shows a schematic flowchart of step S30 in an embodiment of the present invention;

[0044] Figure 5 It shows a schematic flowchart of model pre-training in an embodiment of the present invention;

[0045] Figure 6 It shows a structural block diagram of the system for judging the type of medical form in an embodiment of the present invention;

[0046] Figure 7 It shows a schematic structural diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0048] Please refer to Figure 1-7 . It should be noted that the illustrations provided in this embodiment only schematically illustrate the basic concept of the present invention. Therefore, only the components related to the present invention are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0049] Figure 1 A schematic flowchart showing the method for judging the type of medical form of the present invention is shown.

[0050] The method for judging the type of medical form is applied to one or more electronic devices. The electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0051] The electronic device can be any electronic product that can interact with users. For example, a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an Internet protocol television (IPTV), a smart wearable device, etc.

[0052] The electronic device may also include a network device and / or a user device. Among them, the network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing (Cloud Computing).

[0053] The network where the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0054] Next, the method for judging the type of medical form of the present invention will be elaborated in detail in combination with Figure 1 to elaborate on the method for judging the type of medical form of the present invention in detail.

[0055] A method for judging the type of medical form includes:

[0056] S10. Obtain an image of the medical form.

[0057] In this embodiment, after receiving a medical form that needs to be reimbursed, the business personnel upload the medical form image to the trained medical form matching model to determine the type of the medical form. As an example, the business personnel can obtain medical form data by using the "Ping An Health" software, which is beneficial for the business personnel to receive medical forms anytime and anywhere, greatly improving the geographical restrictions caused by manual form collection in the traditional form collection mode. After receiving various medical forms, the business personnel can open the "Ping An Health" software and upload one or more medical form images to the software by using the "Add Image" button on the software and taking on-site photos or directly reading images from the album. Considering that there may be selection errors when manually selecting images, for example, selecting some images with poor clarity or repeatedly selecting the same image multiple times. To make the selection result more targeted, the selected images can be previewed in small windows in the software, which is more convenient for the business personnel to select better-quality medical form images. It can be understood that the acquisition methods of medical form images in the present invention include but are not limited to software, smartphones, tablets, and digital devices, and are not limited here.

[0058] Next, perform step S20: Through OCR extraction technology, traverse the image of the medical form to obtain multiple segments of text regions to be recognized in the image of the medical form, and extract the text information in the image of the medical form according to the text features in each text region to be recognized.

[0059] Specifically, as Figure 2 shown, the process of traversing the image of the medical form through OCR extraction technology to obtain multiple segments of text regions to be recognized in the image of the medical form and extracting the text information in the image of the medical form according to the text features in each text region to be recognized includes:

[0060] S21: Preprocess the medical form image to obtain a preprocessed medical form image;

[0061] S22: Traverse the preprocessed medical form image through a sliding window algorithm to extract the text region of the preprocessed medical form image;

[0062] S23: Perform rectangular segmentation on the text region to obtain multiple segments of text regions to be recognized, where each segment of text region to be recognized corresponds to a paragraph in the text region;

[0063] S24: Extract the text features in each segment of the text region to be recognized, and obtain the text information of the medical form image through character recognition.

[0064] OCR (Optical Character Recognition) refers to the process in which electronic devices, such as scanners or digital cameras, check the characters printed on paper, determine the character shapes by detecting dark and bright patterns, and then translate the shapes into computer text using character recognition methods. After obtaining the text information on the medical form using deep OCR recognition technology, by traversing each keyword in the pre-set medical form type library for the text information, multiple keywords in the medical form type library contained in the text information can be obtained. The text information includes the text information in the medical form, such as the content of symptom descriptions in medical cases, doctor's signatures, etc. It can be understood that the process of obtaining text information is not limited to the method mentioned in the above embodiment, and can also be through methods such as EAST (Efficient and Accuracy Scene Text Detection), which is not restricted here.

[0065] Further, as Figure 3 shown, the preprocessing of the image of the medical form to obtain the preprocessed medical form image includes:

[0066] S211. Perform grayscale processing on the image of the medical form to obtain a grayscale medical form image;

[0067] S212. Use the method of dynamic threshold segmentation to perform binarization processing on the grayscale medical form image to obtain a binarized medical form image;

[0068] S213. Use the nearest neighbor clustering method to perform skew correction processing on the binarized medical form image to obtain the preprocessed medical form image.

[0069] Considering that medical forms involve various different types such as medical records, medical invoices, and medical lists, the thickness, smoothness, and printing quality of the papers for different types of medical forms are also different. And these factors may all lead to phenomena such as text distortion, broken strokes, adhesion, and stains during OCR recognition. To reduce the occurrence of such phenomena, in this embodiment, before performing OCR text recognition, a series of processes need to be carried out on the text image with noise to ensure that the image of the medical form is clearer and more distinguishable. Specifically, in this embodiment, first, the obtained medical form image is grayscale processed. This is because the obtained medical form images are usually color images, but there are often various interference information in such color images. Therefore, the method of grayscale processing is needed to filter out this interference information. By mapping the pixel points described in three dimensions in the image to pixel points described in one dimension, the process of grayscale processing of the image is realized. In addition, in order to further accurately distinguish the text from the background, the grayscale medical form image needs to be binarized. In this embodiment, using the method of dynamic threshold segmentation, after the grayscale medical form image is smoothed, by comparing the corresponding pixel values in the original image and the smoothed medical form image, difference processing is performed to obtain the binary image of the medical form. Since medical forms are usually printed texts, and most of the printed text materials are composed of horizontal or vertical texts parallel to the page edge, and their tilt angle is zero. However, during the OCR scanning and recognition process, the phenomenon of image tilt will inevitably occur. And this tilted medical form image will seriously affect the character separation and the recognition accuracy rate. To improve this situation, the medical form image needs to be tilt-corrected. Specifically, in an embodiment, the foreground pixels in the medical form image can be mapped to the polar coordinate space, and by statistically calculating the accumulated values of each point in the medical image in the polar coordinate space, the angle by which the medical image needs to be rotated can be obtained. According to this angle, the angle of the medical image is adjusted to obtain the preprocessed medical form image.

[0070] Next, execute step S30: Input the text information into the medical form type matching model, match the text information with each keyword in the preset type library of medical forms, obtain multiple keywords contained in the text information, and calculate the matching rate of the medical form with each type in the type library of medical forms according to the preset weights corresponding to each keyword in the text information, to obtain the type of the medical form. The medical form type matching model is composed of a series connection of TextMatching and a Softmax classifier. The type library of medical forms includes each pre-stored type and the keywords corresponding to each type.

[0071] Specifically, as Figure 4As shown, the steps of inputting the text information into the medical form type matching model, matching the text information with each keyword in the preset type library of medical forms, obtaining multiple keywords contained in the text information, and calculating the matching rate of the medical form with each type in the type library of medical forms according to the preset weights corresponding to each keyword in the text information, to obtain the type of the medical form include:

[0072] S31. Input the text information into the medical form type matching model, and extract multiple keywords contained in the text information by matching the text information with each keyword in the preset type library of medical forms;

[0073] S32. Obtain the matching rate of the medical form with each medical form type according to the preset weights corresponding to each keyword in the text information;

[0074] S33. Sort the matching rates of the medical form with each type, select the type with the highest matching rate with the medical form, compare the highest matching rate with a preset matching rate threshold, and if the highest matching rate is greater than or equal to the preset matching rate threshold, use the type corresponding to the highest matching rate as the type of the medical form.

[0075] The TextMatching network is an end-to-end text matching method, mainly to find the text most relevant to the target text. First, word vectors in the text are extracted through a word vector model, and the word vector model can use mainstream models, including but not limited to word2vec, fastText, glove, bert, etc. Then, the similarity scores between the text to be recognized and the keywords are calculated to obtain a similarity matrix. A convolutional neural network is used to extract text features, and two-layer convolutional neural network is used to extract features in the similarity matrix, and finally the keyword information in the text information is obtained. In this embodiment, each keyword has a corresponding preset weight, and by calculating the weights of the keywords involved in the text information, the matching rate of the medical form with each type of medical form can be obtained. It should be noted that in this embodiment, the types of medical forms include but are not limited to medical invoices, medical cases, VAT invoices, medical discharge summaries, and those skilled in the art can adaptively set different types of medical forms and the keywords corresponding to each type of medical form according to needs, so as to form different type libraries of medical forms, which will not be elaborated here.

[0076] In this embodiment, the formula for the matching rate is: where p im is the matching rate of the i-th medical form with the m-th type in the type library of medical forms, and N im is the total number of keywords of the i-th medical form belonging to the m-th type, is the preset weight of the j-th keyword in the m-th type, is the matching value between the j-th keyword and the text information of the i-th medical form. Specifically, takes a value of 0 or 1. When takes a value of 0, it means that in the m-th type of the i-th medical form, the j-th keyword matches the text information. When takes a value of 1, it means that in the m-th type of the i-th medical form, the j-th keyword does not match the text information. It can be understood that can also be other values, as long as it can represent whether the keyword matches the text information. As an example, p 12 is the 2nd type of the 1st medical form in the preset medical form type library, that is, the medical case. In the type of medical case, there are a total of 120 keywords, that is, N 12 is 120. After matching the text information with each keyword in the medical case, the 3rd and 6th keywords match the text information. Among them, the 3rd keyword is "case", and its preset weight is 4, and the 6th keyword is "disease", and its preset weight is 2. Then in the medical form and the preset medical type library, the matching rate of the medical case is 6 / 120.

[0077] After obtaining the matching rates of the types of each medical form corresponding to the medical form, they can be sorted in descending or ascending order. And select the type of the medical form with the highest matching rate as the medical form type to be set. Then compare the matching rate corresponding to the medical form to be set with the preset matching rate threshold. If the matching rate is greater than or equal to the preset matching rate threshold, it means that the type of the medical form to be set fits well with the various data of the medical form, and the type of the medical form to be set can be used as the type of this medical form. For example, after the medical form is matched with various different medical form types in the medical form type library, by calculating the corresponding weights in different medical form types, the following matching rates are obtained: the matching rate of this medical form with the invoice is 85%, the matching rate of this medical form with the medical case is 73%, and the matching rate of this medical form with the medical discharge summary is 83.7%. After sorting the three, select the invoice with the highest matching rate with this medical form as the medical form type to be set. Among them, the preset matching rate threshold is 79%, then it can be considered that the type of this medical form is an invoice. It should be noted that those skilled in the art can adaptively change the matching rate threshold according to the detection accuracy, which is not limited here. Further, considering that there are anti-counterfeiting mark coordinates in the value-added tax invoice, the anti-counterfeiting mark can be detected by coordinate detection to determine the type of the medical form. For example, after the matching is completed, the anti-counterfeiting mark in the upper left corner of this medical form is the same as the anti-counterfeiting mark in the medical form type library. The keywords in the text information are invoice, invoicing, value-added, and case. Invoice, invoicing, and value-added belong to the value-added tax invoice type, and case belongs to the medical case type. Among them, the preset weight of the invoice is 4, the preset weight of value-added is 3, the weight of invoicing is 4, the preset weight of the case is 3, the weight of the anti-counterfeiting mark coordinate rule in the upper left corner is 6, there are 90 keywords for the value-added tax invoice, and 120 keywords for the medical case. Therefore, the probability that this medical form corresponds to the value-added tax invoice type is 17 / 90, and the probability that this medical form corresponds to the medical case is 3 / 120.

[0078] Further, as Figure 5 shown, the medical form type matching model is obtained through pre-training, and the process of the pre-training includes:

[0079] S301. Obtain medical form data including various types, and the types at least include: medical summary, medical case, medical invoice, and medical list;

[0080] S302. Use text recognition to extract the text information of the medical form data;

[0081] S303. Input the text information into the medical form type matching model to be trained for training. Extract the feature vector of the text information through the convolution operation of TextMatching, and send the feature vector into the Softmax classifier for mapping to obtain the medical form type matching model.

[0082] The medical form type matching model in this embodiment is obtained through pre-training. First, training sample data needs to be obtained. The training sample data is a sample data set made from multiple medical form images of different types, where the types include but are not limited to medical summaries, medical cases, medical invoices, and medical lists. And the sample data set is divided into a training set and a test set according to a certain ratio. The training set is used to determine the model parameters, and the test set is used to test the generalization ability of the trained model. The text information in the medical form data is extracted through deep OCR recognition technology, and then the text information is input into the medical form type matching model for training. In this embodiment, the medical form type matching network model is constructed based on the TextMatching network and the Softmax layer. Specifically, after the text information is sent into the TextMatching network, the feature vector of the text information is first obtained through convolution operation, and the feature vector is sent to the Softmax layer for mapping to obtain the matching rate of the text information with each type in the preset medical form type library. The weights of the model are continuously updated through the loss function. When the loss function converges, it means that each weight in the model reaches the optimal state, and the medical form type matching model can be obtained. Considering that when the training set is too small, the neural network is prone to underfitting, resulting in the model not being able to fit the data features well, and the final prediction accuracy is small. When the training set is too large, the neural network is prone to overfitting, resulting in low generalization ability of the model. In one embodiment, the ratio of the training set to the test set in the sample data set is 8:2. In this way, the model can be effectively trained, and at the same time, the phenomenon of overfitting caused by too many training samples can be improved. Of course, the sample ratio of the training set and the test set is not fixedly limited, and those skilled in the art can adaptively change it according to actual needs.

[0083] Further, after sorting the matching rates corresponding to each type of the medical form and selecting the type with the highest matching rate with the medical form, and comparing the highest matching rate with the preset matching rate threshold, it further includes:

[0084] If the highest matching rate is less than the preset matching rate threshold, a data label is added to the medical form according to the medical form type.

[0085] The medical form with the data label is input into the medical form type matching model for training, and the weights of the medical form type matching model are updated by the gradient descent method.

[0086] In this embodiment, if the highest matching rate is less than the preset matching rate threshold, it indicates that there is a large difference between the type corresponding to the highest matching rate and the medical form. At this time, the type corresponding to the highest matching rate cannot be simply determined as the type of the medical form, thus reducing the recognition accuracy. Therefore, it is necessary to improve the recognition accuracy of the model. First, the medical form to be judged needs to be saved to the recognition pool to be trained through the background. When the medical form data in the recognition pool to be trained accumulates to a certain amount, a specific operator will be notified. The operator will identify which type of medical form in the type library of medical forms the medical form to be judged belongs to according to the characteristics of the medical form to be judged, and add a data label of the corresponding category to the medical form. For example, the data label for the invoice type is 1, the data label for the medical record type is 2, and the data label for the medical summary type is 3. Considering that due to reasons such as blurred images and incomplete images, the operator may make mistakes in judgment, resulting in adding incorrect labels to the medical form. For example, a medical form that originally belongs to an invoice may be misjudged as a medical summary due to misjudgment. Such incorrect labels will mislead the recognition direction of the model, resulting in a decrease in the recognition rate of the medical form judgment model. To improve this situation, in this embodiment, after the data label is added, the medical form with the added data label will be automatically fed back to the business personnel to enable the business personnel to further confirm whether the added data label is correct. If it is correct, the medical form with the added data label will be sent into the medical form type matching model for training, and the weights of the model will be continuously updated through the gradient descent method to further optimize the performance of the model. If it is incorrect, the business personnel will change the data label to the correct type, and the medical form marked with the correct data label will be sent into the model for training. By continuously increasing the sample data, the recognition performance of the model will be further improved.

[0087] It should be noted that in the present invention, in order to further ensure the security of data, the data and models involved can also be deployed on the blockchain to prevent the data from being maliciously tampered with.

[0088] It should be noted that the step division of the above various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of its algorithm and process are all within the protection scope of this patent.

[0089] As Figure 6 shown, it is a structural block diagram of the system for judging the type of medical form of the present invention. The system for judging the type of medical form includes: an image acquisition module 111, a text extraction module 112, and a type judgment module 113. The module referred to in the present invention means a series of computer program segments that can be executed by a processor 13 and can complete fixed functions, and are stored in a memory 12.

[0090] The image acquisition module 111 is used to acquire the image of the medical form.

[0091] In this embodiment, after the business personnel receive the medical form to be reimbursed, they upload the medical form image to the trained medical form matching model to determine the type of the medical form. As an example, the business personnel can obtain the medical form data by using the "Ping An Health" software, which is beneficial for the business personnel to receive the medical form anytime and anywhere, greatly improving the geographical restrictions caused by manual form collection in the traditional form collection mode. After receiving various medical forms, the business personnel can open the "Ping An Health" software and upload one or more medical form images to the software by using the "Add Image" button on the software and taking on-site photos or directly reading images from the album. Considering that there may be selection errors when manually selecting images, such as selecting some images with poor clarity or repeatedly selecting the same image multiple times. To make the selection result more targeted, the selected images can be previewed in a small window in the software, making it more convenient for the business personnel to select medical form images with better quality. It can be understood that the acquisition method of the medical form image in the present invention includes but is not limited to software, smartphones, tablets, digital devices, and is not limited herein.

[0092] The text extraction module 112 is used to traverse the image of the medical form through OCR extraction technology to obtain multiple regions of text to be recognized in the image of the medical form, and extract the text information in the image of the medical form according to the text features in each region of text to be recognized.

[0093] OCR (Optical Character Recognition) refers to the process in which electronic devices, such as scanners or digital cameras, determine the shape of characters by checking the characters printed on paper and detecting dark and bright patterns, and then translate the shape into computer text using character recognition methods. After obtaining the text information on the medical form using deep OCR recognition technology, by traversing each keyword in the pre-set medical form type library with the text information, multiple keywords in the medical form type library contained in the text information can be obtained. The text information includes the text information in the medical form, such as the content of symptom descriptions in medical cases, doctor signatures, etc. It can be understood that the acquisition process of the text information is not limited to the method mentioned in the above embodiment, and can also be obtained through methods such as EAST (Efficient and Accuracy Scene Text Detection), etc., and is not limited herein.

[0094] Considering that medical forms involve various different types such as medical records, medical invoices, and medical lists, the thickness, smoothness, and printing quality of different types of medical form papers are also different. And these factors may all cause phenomena such as text distortion, broken strokes, adhesion, and stains during OCR recognition. To reduce the occurrence of such phenomena, in this embodiment, before performing OCR text recognition, a series of processes need to be performed on the text image with noise to ensure that the image of the medical form is clearer and more distinguishable. Specifically, in this embodiment, first, the obtained medical form image is grayscale processed. This is because the obtained medical form images are usually color images, but such color images often contain various interference information, so it is necessary to use the grayscale processing method to filter out this interference information. By mapping the pixel points described in three dimensions in the image to pixel points described in one dimension, the grayscale processing process of the image is realized. In addition, to further accurately distinguish the text from the background, the grayscale medical form image needs to be binarized. In this embodiment, using the method of dynamic threshold segmentation, after the grayscale medical form image is smoothed, by comparing the corresponding pixel values in the original image and the smoothed medical form image, difference processing is performed to obtain the binary medical form image. Since medical forms are usually printed, and most of the printed text materials are composed of horizontal or vertical texts parallel to the page edge, and their tilt angle is zero. However, during the OCR scanning and recognition process, it is inevitable that the image will be tilted. And this tilted medical form image will seriously affect the character separation and the recognition accuracy rate. To improve this situation, it is necessary to correct the tilt of the medical form image. Specifically, in one embodiment, the foreground pixels in the medical form image can be mapped to the polar coordinate space, and by statistically calculating the cumulative values of each point in the medical image in the polar coordinate space, the angle by which the medical image needs to be rotated can be obtained. According to this angle, the angle of the medical image is adjusted to obtain the preprocessed medical form image.

[0095] The type judgment module 113 is used to input the text information into the medical form type matching model, match the text information with each keyword in the type library of the preset medical form, obtain multiple keywords contained in the text information, and calculate the matching rate of the medical form corresponding to each type in the type library of the medical form according to the preset weights corresponding to each keyword in the text information, so as to obtain the type of the medical form. The medical form type matching model is composed of a series connection of TextMatching and a Softmax classifier. The type library of the medical form includes each pre-stored type and the keywords corresponding to each type.

[0096] The TextMatching network is an end-to-end text matching method mainly for finding the text most relevant to the target text. First, word vectors in the text are extracted through a word vector model, and the word vector model can use mainstream models, including but not limited to word2vec, fastText, glove, bert, etc. Then, the similarity scores between the text to be recognized and the keywords are calculated to obtain a similarity matrix. A convolutional neural network is used to extract text features, and two-layer convolutional neural networks are used to extract features in the similarity matrix, and finally the keyword information in the text information is obtained. In this embodiment, each keyword has a corresponding preset weight, and by calculating the weights of the keywords involved in the text information, the matching rate between the medical form and each type of medical form can be obtained. It should be noted that in this embodiment, the types of medical forms include but are not limited to medical invoices, medical records, VAT invoices, medical discharge summaries, and those skilled in the art can adaptively set different types of medical forms and the corresponding keywords for each type of medical form according to needs, so as to form different type libraries of medical forms, which will not be elaborated here.

[0097] In this embodiment, the formula for the matching rate is: where p im is the matching rate of the i-th medical form and the m-th type in the type library of the medical form, N im is the total number of keywords of the i-th medical form belonging to the m-th type, is the preset weight of the j-th keyword in the m-th type, is the matching value between the j-th keyword and the text information in the i-th medical form. Specifically, takes a value of 0 or 1. When takes a value of 0, it means that in the m-th type of the i-th medical form, the j-th keyword matches the text information. When takes a value of 1, it means that in the m-th type of the i-th medical form, the j-th keyword does not match the text information. It can be understood that can also be other values as long as it can represent whether the keyword matches the text information. As an example, p 12 is the 2nd type of the 1st medical form in the preset type library of medical forms, that is, the medical record. In the type of medical record, there are a total of 120 keywords, that is, N 12 is 120. After matching the text information with each keyword in the medical record, the 3rd and 6th keywords match the text information. Among them, the 3rd keyword is "case" with a preset weight of 4, and the 6th keyword is "disease" with a preset weight of 2. Then the matching rate of the medical form and the medical record in the preset type library of medical forms is 6 / 120.

[0098] After obtaining the matching rates of the types of each medical form corresponding to the medical form, they can be sorted in descending or ascending order. And select the type of the medical form with the highest matching rate as the type of the medical form to be set. Then compare the matching rate corresponding to the medical form to be set with the preset matching rate threshold. If the matching rate is greater than or equal to the preset matching rate threshold, it means that the type of the medical form to be set is more suitable for the various data of the medical form, and the type of the medical form to be set can be used as the type of this medical form. For example, after the medical form is matched with various different medical form types in the medical form type library, by calculating the corresponding weights in different medical form types, the following matching rates are obtained: the matching rate of this medical form with the invoice is 85%, the matching rate of this medical form with the medical case is 73%, and the matching rate of this medical form with the medical discharge summary is 83.7%. After sorting the three, select the invoice with the highest matching rate with this medical form as the type of the medical form to be set. Among them, the preset matching rate threshold is 79%, then it can be considered that the type of this medical form is an invoice. It should be noted that those skilled in the art can adaptively change the matching rate threshold according to the detection accuracy, which is not limited here. Further, considering that the value-added tax invoice has anti-counterfeiting mark coordinates, the anti-counterfeiting mark can be detected through coordinate detection to determine the type of the medical form. For example, after the matching is completed, the anti-counterfeiting mark in the upper left corner of this medical form is the same as the anti-counterfeiting mark in the medical form type library. The keywords in the text information are invoice, invoicing, value-added, and case. Invoice, invoicing, and value-added belong to the value-added tax invoice type, and case belongs to the medical case type. Among them, the preset weight of the invoice is 4, the preset weight of the value-added is 3, the weight of the invoicing is 4, the preset weight of the case is 3, the weight of the anti-counterfeiting mark coordinate rule in the upper left corner is 6, there are 90 keywords for the value-added tax invoice, and 120 keywords for the medical case. Therefore, the probability that this medical form corresponds to the value-added tax invoice type is 17 / 90, and the probability that this medical form corresponds to the medical case is 3 / 120.

[0099] It should be noted that the system for judging the type of the medical form in this embodiment is a system corresponding to the above method for judging the type of the medical form. The functional modules in the system for judging the type of the medical form respectively correspond to the corresponding steps in the method for judging the type of the medical form. The system for judging the type of the medical form in this embodiment can be implemented in cooperation with the method for judging the type of the medical form. Correspondingly, the relevant technical details mentioned in the system for judging the type of the medical form in this embodiment can also be applied to the above method for judging the type of the medical form.

[0100] It should be noted that in actual implementation, all or part of the above functional modules can be integrated into one physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together or independently implemented. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, part or all of the steps of the above method, or the above functional modules, can be completed by the integrated logic circuit in hardware or the instructions in software form in the processor element.

[0101] As Figure 7 shown, it is a schematic structural diagram of the electronic device of the present invention.

[0102] The electronic device 1 may include a memory 12, a processor 13, and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a text recognition program based on direction detection.

[0103] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as: SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 12 can be an internal storage unit of the electronic device 1 in some embodiments, such as the mobile hard disk of the electronic device 1. The memory 12 can also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 1. Further, the memory 12 can also include both the internal storage unit and the external storage device of the electronic device 1. The memory 12 can not only be used to store the application software installed in the electronic device 1 and various types of data, such as the code of the text recognition program based on direction detection, etc., but also be used to temporarily store the data that has been output or will be output.

[0104] In some embodiments, the processor 13 may be composed of an integrated circuit. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 13 is the control core of the electronic device 1. It uses various interfaces and circuits to connect all components of the entire electronic device 1. By running or executing programs or modules stored in the memory 12 (such as executing a physical examination report verification program, etc.), and calling data stored in the memory 12, it performs various functions of the electronic device 1 and processes data.

[0105] The processor 13 executes the operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application programs to implement the steps in the embodiments of the judgment methods for each type of medical form above. For example Figure 1 the steps shown.

[0106] Exemplarily, the computer program may be divided into one or more modules. The one or more modules are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an image acquisition module 111, a text extraction module 112, and a type judgment module 113.

[0107] The above-mentioned integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The above-mentioned software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a processor to execute some functions of the physical examination item recommendation method described in each embodiment of the present invention.

[0108] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithms. A blockchain, essentially a decentralized database, is a string of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-counterfeiting) of the information and generate the next block. A blockchain may include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0109] The bus can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, in Figure 7 only one arrow is used to represent it, but it does not mean that there is only one bus or one type of bus. The bus is configured to implement the connection communication between the memory 12 and at least one processor 13, etc.

[0110] The method, system, device and medium for judging the type of medical form and document recognition of the present invention match the text information in the medical form with the keywords in the preset type library of medical forms. Among them, the type library of medical forms has different types of medical forms and multiple keyword information for each type. According to the different weights of different keywords, the corresponding type of the medical form is obtained. Thus, the type of the medical form can be recognized conveniently, efficiently and accurately, effectively improving the form receiving efficiency of business personnel.

[0111] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for judging the type of medical form, characterized in that, it includes: Obtain an image of the medical form; Through OCR extraction technology, traverse the image of the medical form, obtain multiple text regions to be recognized in the image of the medical form, and extract the text information in the image of the medical form according to the text features in each text region to be recognized; Input the text information into a medical form type matching model, match the text information with each keyword in the type library of the preset medical form, obtain multiple keywords contained in the text information, and calculate the matching rate of the medical form corresponding to each type in the type library of the medical form according to the preset weights corresponding to each keyword in the text information, to obtain the type of the medical form. The medical form type matching model is composed of a series connection of TextMatching and a Softmax classifier. The type library of the medical form includes each pre-stored type and the keywords corresponding to each type. The types at least include: medical summary, medical case, medical invoice, and / or drug list; wherein, the OCR recognition technology is a deep OCR recognition technology.

2. The method for judging the type of medical form according to claim 1, characterized in that, The step of traversing the image of the medical form through OCR extraction technology, obtaining multiple text regions to be recognized in the image of the medical form, and extracting the text information in the image of the medical form according to the text features in each text region to be recognized includes: Preprocess the image of the medical form to obtain a preprocessed medical form image; Traverse the preprocessed medical form image through a sliding window algorithm to extract the text region of the preprocessed medical form image; Perform rectangular segmentation on the text region to obtain multiple text regions to be recognized, wherein each text region to be recognized corresponds to a paragraph in the text region of the preprocessed medical form image; Extract the text features in each text region to be recognized, and obtain the text information in the image of the medical form through character recognition.

3. The method for judging the type of medical form according to claim 2, characterized in that, The step of preprocessing the image of the medical form to obtain a preprocessed medical form image includes: Perform grayscale processing on the image of the medical form to obtain a grayscale medical form image; Adopt a method of dynamic threshold segmentation to perform binarization processing on the grayscale medical form image to obtain a binarized medical form image; Use the nearest neighbor clustering method to perform skew correction processing on the binarized medical form image to obtain a preprocessed medical form image.

4. The method for judging the type of medical form according to claim 1, characterized in that, The step of inputting the text information into a medical form type matching model, matching the text information with each keyword in the type library of the preset medical form, obtaining multiple keywords contained in the text information, and calculating the matching rate of the medical form corresponding to each type in the type library of the medical form according to the preset weights corresponding to each keyword in the text information, to obtain the type of the medical form includes: Input the text information into the medical form type matching model, and extract multiple keywords contained in the text information by matching the text information with each keyword in the type library of the medical form; Obtain the matching rate of the medical form with each type according to the preset weights corresponding to each keyword in the text information; Sort the matching rates of the medical form with each type, select the type with the highest matching rate with the medical form, compare the highest matching rate with a preset matching rate threshold. If the highest matching rate is greater than or equal to the preset matching rate threshold, use the type corresponding to the highest matching rate as the type of the medical form.

5. The method for judging the type of medical form according to claim 4, wherein, after sorting the matching rates of the medical form with each type, selecting the type with the highest matching rate with the medical form, and comparing the highest matching rate with a preset matching rate threshold, it further includes: if the highest matching rate is less than the preset matching rate threshold, add a data label to the medical form according to the type of the medical form; Input the medical form with the data label into the medical form type matching model for training, and update the weights of the medical form type matching model by the gradient descent method.

6. The method for judging the type of medical form according to claim 1, wherein, the medical form type matching model is obtained through pre-training, and the pre-training process includes: Obtain medical form data containing various types; Adopt text recognition to extract the text information of the medical form data, Input the text information into the medical form type matching model to be trained for training, extract the feature vector of the text information through the convolution operation of TextMatching, and send the feature vector into the Softmax classifier for mapping to obtain the medical form type matching model.

7. The method for judging the type of medical form according to claim 1, wherein, The method for obtaining the matching rate includes: Among them, p im is the matching rate of the i-th medical form and the m-th type in the type library of the medical form, N im is the total number of keywords that the i-th medical form belongs to the m-th type, is the preset weight of the j-th keyword in the m-th type, is the matching value of the j-th keyword and the text information in the i-th medical form.

8. A system for judging the type of medical form, wherein, it includes: An image acquisition module for acquiring an image of a medical form; A text extraction module for traversing the image of the medical form through OCR extraction technology to obtain multiple text regions to be recognized in the image of the medical form, and extracting the text information in the image of the medical form according to the text features in each text region to be recognized; A type judgment module, configured to input the text information into a medical form type matching model, match the text information with each keyword in a type library of preset medical forms, obtain multiple keywords included in the text information, and calculate a matching rate corresponding to each type in the type library of the medical form according to preset weights corresponding to each keyword in the text information, so as to obtain the type of the medical form. The medical form type matching model is composed of a cascade of TextMatching and a Softmax classifier. The type library of the medical form includes pre-stored types and keywords corresponding to each type. The types at least include: medical summaries, medical cases, medical invoices, and / or drug lists; Among them, the OCR recognition technology is deep OCR recognition technology.

9. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images