Drug identification methods

Abstract

A drug identification method utilizes a computing device storing a drug dataset to perform the following steps: obtaining a drug image relating to at least one drug to be identified and a target drug category; segmenting at least one image block to be identified from the drug image using instance segmentation technology; determining, based on the target drug category, that the at least one drug to be identified belongs to a first drug category or a second drug category; when it is determined to belong to the first drug category, obtaining a preliminary identification result for each image block to be identified using at least one of barcode recognition technology and a drug identification model; and when it is determined to belong to the second drug category, obtaining the preliminary identification result for each image block to be identified using at least one of barcode recognition technology and optical character recognition technology.

Description

Drug identification methods This invention relates to an identification method, and more particularly to a drug identification method. Because some medications, despite their different functions, have very similar English names, errors in administration or dispensing can easily occur if not carefully monitored, potentially causing serious harm to patients. Therefore, to avoid medication errors, pharmacists and nurses must follow the "three reads and five checks" procedure when dispensing medications. The three reads refer to reading the full name of the medication aloud when picking it up, when administering it, and when returning it to the pharmacy. The five checks refer to verifying that the patient's information, medication contents, administration time, dosage, and route of administration are correct when giving the medication to the patient. However, the "three reads and five checks" procedure requires manual operation, which is time-consuming and requires a significant amount of manpower. Although artificial intelligence (AI) and computer vision (CVR) technologies are currently being used to identify medications to assist pharmacies in correctly dispensing medications, there is still room for improvement in the accuracy of medication identification. Therefore, improving the accuracy of drug identification to reduce the time and manpower costs of the "three readings and five verifications" procedure has become one of the issues that related technical fields want to address. Therefore, the object of the present invention is to provide a drug identification method that overcomes at least one drawback of the prior art. Therefore, the present invention provides a drug identification method, which is executed using a computing device storing a drug data set. The drug data set includes multiple drug records corresponding to multiple drugs, each drug record including a drug code, a barcode number, a drug category indicating the dosage form or packaging method, and descriptive text. The drug identification method includes the following steps: (A) obtaining a drug image and a target drug category for at least one drug to be identified, the drug image displaying each of the at least one drug to be identified, and the target drug category indicating one of a first drug category and a second drug category; (B) using instance segmentation technology to segment at least one image block corresponding to each of the at least one drug to be identified from the drug image, each image block displaying only the corresponding drug to be identified; (C) determining whether the at least one drug to be identified belongs to the first drug category or the second drug category based on the target drug category; (D) (E) When it is determined that the at least one drug to be identified belongs to the first drug category, for each image block to be identified, based on the image block to be identified, at least one of barcode recognition technology and a drug identification model is used to obtain a preliminary identification result corresponding to the drug to be identified corresponding to the image block to be identified, the preliminary identification result indicating one of all drug codes in the drug database; and (E) When it is determined that the at least one drug to be identified belongs to the second drug category, for each image block to be identified, based on the image block to be identified, at least one of barcode recognition technology and optical character recognition technology is used to obtain the preliminary identification result. The advantage of this invention is that by first determining whether the at least one drug to be identified belongs to the first drug category or the second drug category, and then selecting the appropriate technology according to the category to identify each drug to be identified to obtain the corresponding preliminary identification result, different categories of drugs can be identified in the most suitable way, thereby improving the accuracy of drug identification and better assisting medical staff in confirming whether the drugs taken are correct, thereby reducing the time and manpower costs when performing the "three readings and five verifications" procedure. Before the present invention is described in detail, it should be noted that similar elements are represented by the same codes in the following description. Referring to Figure 1, an embodiment of the drug identification method of the present invention is executed using a computing device and includes steps S1 to S8. The computing device stores a drug dataset, an image segmentation model, and a drug identification model. This drug information set includes multiple entries corresponding to various drugs. Each drug's information includes a drug code, a barcode number, a drug category indicating the dosage form or packaging method, and descriptive text. More specifically, the drug category indicates whether the corresponding drug belongs to one of the following categories: injection, boxed medication, eye drops, vials, ointments, powders, or other pharmaceutical preparations; the descriptive text includes, for example, the drug's scientific name, ingredient name, and Chinese drug name. This image segmentation model utilizes instance segmentation technology to segment all image blocks displaying medicine objects, while also identifying the medicine category of each medicine object. In this embodiment, multiple training image datasets used to train the image segmentation model are used to automatically label medicine objects using Anylabeling software, and the medicine category of each medicine object is labeled. This image segmentation model is based on the Mask R-CNN model architecture, and the model hyperparameters are set as follows: Batch_size=2, Box_thresh=8, Epoch=10, Learning_rate=0.002. This drug identification model is used to identify drugs displayed in an image and output the corresponding drug code. In this embodiment, the drug identification model is based on a densely connected convolutional network (DenseNet), which learns features such as the shape of the drug, label text, and drug color to identify different drugs. In this embodiment, the drug identification model is specifically trained to identify drugs corresponding to the categories of injections, eye drops, and medicine bottles. Hereinafter, each step of this embodiment will be described in detail with reference to FIG1. In step S1, the computing device obtains a target drug code corresponding to a target drug and a target drug category corresponding to the target drug. The target drug category indicates one of a first drug category and a second drug category. More specifically, the user can use a QR code (Quick Response Code) scanner connected to the computing device to scan a QR code related to the target drug on a prescription, thereby obtaining a QR code scanning result corresponding to the QR code, and extracting the target drug code and the target drug category from the QR code scanning result. In this embodiment, the first drug category includes injections, eye drops, and vials, and the second drug category includes boxed medications, ointments, powders, and other medications. In step S2, the computing device obtains a drug image relating to at least one drug to be identified. The drug image displays each of the at least one drug to be identified. For example, the drug image is, for instance, a drug image P shown in Figure 2, which displays five drugs M to be identified. In step S3, the computing device uses the image segmentation model to segment at least one image block corresponding to the at least one drug to be identified from the drug image. Each image block only displays the corresponding drug to be identified. Continuing the previous example, referring to Figure 3, the drug image P is segmented into five image blocks P' by the image segmentation model. In step S4, the computing device determines whether the at least one drug to be identified belongs to the first drug category or the second drug category based on the target drug category. When the target drug category indicates the first drug category, it is determined that the at least one drug to be identified belongs to the first drug category, and the process proceeds to step S5; when the target drug category indicates the second drug category, it is determined that the at least one drug to be identified belongs to the second drug category, and the process proceeds to step S6. In step S5, for each image block to be identified, the computing device uses at least one of barcode recognition technology and the drug identification model to obtain a preliminary identification result corresponding to the drug to be identified for that image block. This preliminary identification result indicates one of all drug codes in the drug database. More specifically, referring to Figure 4, this step also includes the following sub-steps S51-S56. In sub-step S51, the computing device determines whether there is a barcode to be identified on the drug to be identified displayed in the image block to be identified. If it is determined that there is no barcode to be identified, the process proceeds to sub-step S52; if it is determined that there is a barcode to be identified, the process proceeds to sub-step S54. In sub-step S52, the computing device uses the drug identification model to identify the drug to be identified displayed in the image block to be identified, so as to obtain an image identification result. The image identification result indicates one of all drug codes in the drug database. In sub-step S53, the computing device uses the image recognition result as the initial recognition result. Then, the process proceeds to step S7. In sub-step S54, the computing device uses barcode recognition technology to identify the barcode to be identified to obtain an identified barcode number, and obtains a barcode identification result based on the drug database and the identified barcode number. The barcode identification result indicates the barcode number of the drug corresponding to all drug codes in the drug database that is the same as the identified barcode number. In sub-step S55, the computing device uses the drug identification model to identify the drug to be identified displayed in the image block to be identified, so as to obtain the image identification result. In sub-step S56, the computing device compares the barcode recognition result and the image recognition result to obtain the preliminary recognition result. More specifically, if the drug code indicated by the barcode recognition result and the image recognition result is the same, then the preliminary recognition result is the drug code indicated by the barcode recognition result and the image recognition result; if the drug code indicated by the barcode recognition result and the image recognition result is different, then the preliminary recognition result is the drug code indicated by the barcode recognition result. Afterwards, the process proceeds to step S7. In step S6, the computing device obtains the initial recognition result based on the image block to be identified using at least one of barcode recognition technology and optical character recognition (OCR) technology. More specifically, referring to Figure 5, this step also includes the following sub-steps S61 to S68. In sub-step S61, the computing device determines whether the barcode to be identified exists on the drug to be identified displayed in the image block to be identified. If it is determined that the barcode to be identified does not exist, the process proceeds to sub-step S62; if it is determined that the barcode to be identified exists, the process proceeds to sub-step S65. In sub-step S62, the computing device uses OCR technology to identify the text on the medicine to be identified displayed in the image block to be identified, so as to obtain a text content to be compared. In sub-step S63, the computing device compares the text content to be compared with all the feature text content in the drug database to obtain a text recognition result. The text recognition result indicates the feature text content of the drug corresponding to all drug codes in the drug database that has the highest similarity to the text content to be compared. In sub-step S64, the computing device uses the character recognition result as the initial recognition result. Then, the process proceeds to step S7. In sub-step S65, the computing device uses barcode recognition technology to identify the barcode to be identified to obtain the identified barcode number, and obtains the barcode identification result based on the drug data set and the identified barcode number. In sub-step S66, the computing device uses OCR technology to identify the text on the drug to be identified displayed in the image block to be identified, so as to obtain the text content to be compared. In sub-step S67, the computing device compares the text content to be compared with all the feature text content of the drug data set to obtain the text recognition result. In sub-step S68, the computing device compares the barcode recognition result and the text recognition result to obtain the preliminary recognition result. More specifically, if the drug code indicated by the barcode recognition result and the text recognition result is the same, then the preliminary recognition result is the drug code indicated by the barcode recognition result and the text recognition result; if the drug code indicated by the barcode recognition result and the text recognition result is not the same, then the preliminary recognition result is the drug code indicated by the barcode recognition result. Afterwards, the process proceeds to step S7. It is worth noting that the packaging surfaces of boxed medicines, ointments, and powders included in the second drug category are relatively flat and often contain a large amount of text, making them more suitable for OCR technology identification. However, the packaging surfaces of injections, eye drops, and medicine bottles included in the first drug category are curved and contain less text, resulting in poor identification performance using OCR technology. Therefore, this drug identification model is used for identification. For each drug to be identified, it is first classified into the first drug category and the second drug category before identification, ensuring that different categories of drugs can be identified in the most suitable way, thereby improving the accuracy of drug identification. In step S7, for each drug to be identified, the computing device obtains a final identification result corresponding to the drug to be identified based on the target drug code and all the initial identification results corresponding to the drug to be identified. The final identification result is a matching tag or a non-matching tag. The matching tag indicates that the drug to be identified matches the target drug, and the non-matching tag indicates that the drug to be identified does not match the target drug. In this embodiment, since each drug to be identified has only one initial identification result, for each drug to be identified, if the drug code indicated by the initial identification result is the same as the target drug code, the final identification result is the matching mark; otherwise, the final identification result is the non-matching mark. However, in some other embodiments, to improve the identification accuracy, the computing device obtains two images taken directly above and below the at least one drug to be identified in step S2. Therefore, for both the image taken directly above and the image taken directly below, the computing device executes steps S3 to S6 once each. In this case, each drug to be identified will have two initial identification results. For each drug to be identified, if one of the drug codes indicated by all the initial identification results for the drug to be identified is the same as the target drug code, the final identification result is the matching mark; if none of the drug codes indicated by all the initial identification results for the drug to be identified is the same as the target drug code, the final identification result is the non-match mark. In step S8, the computing device obtains an actual quantity of the target drug based on the number of matching markers in the at least one final identification result. This allows medical personnel to determine whether the drugs taken and their quantities are correct by comparing the actual quantity with the target quantity, thereby preventing medication errors. In summary, by first determining whether at least one drug to be identified belongs to the first or second drug category, and then selecting the appropriate technology based on the category to identify the drug and obtain the corresponding preliminary identification result, different categories of drugs can be identified in the most suitable way, thereby improving the accuracy of drug identification. Furthermore, by obtaining the final identification result for each drug to be identified, it is possible not only to determine whether the drug taken by the user (i.e., the drug to be identified) is the target drug, but also to count the quantity taken. This better assists medical personnel in confirming whether the taken drugs and their quantities are correct, saving time and manpower costs when performing the "three reads and five checks" procedure. Therefore, the purpose of this invention is indeed achieved. However, the above description is merely an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the scope of the patent application and the contents of the patent specification shall still fall within the scope of the patent of the present invention. P: Drug image M: Drug to be identified P': Image block to be identified S1~S8: Steps S51~S56: Sub-steps S61~S68: Sub-steps Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: Figure 1 is a flowchart illustrating an embodiment of the drug identification method of the present invention; Figure 2 is a schematic diagram illustrating a drug image; Figure 3 is a schematic diagram illustrating multiple image blocks to be identified in the drug image; Figure 4 is a flowchart illustrating how to obtain a preliminary identification result using barcode identification technology and a drug identification model; and Figure 5 is a flowchart illustrating how to obtain the preliminary identification result using barcode identification technology and optical character recognition technology. S1~S8: Steps

Claims

1. A drug identification method is executed using a computing device storing a drug database, the drug database including multiple drug records corresponding to multiple drugs, each drug record including a drug code corresponding to the drug, a barcode number corresponding to the drug, a drug category indicating the dosage form or packaging method of the drug, and characteristic text content describing the drug. The drug identification method includes the following steps: (A) obtaining a drug image and a target drug category for at least one drug to be identified, the drug image displaying each of the at least one drug to be identified, and the target drug category indicating one of a first drug category and a second drug category; (B) using instance segmentation technology to segment at least one image block corresponding to each of the at least one drug to be identified from the drug image, each image block displaying only the corresponding drug to be identified; (C) determining whether the at least one drug to be identified belongs to the first drug category or the second drug category based on the target drug category. (D) When it is determined that the at least one drug to be identified belongs to the first drug category, for each image block to be identified, a preliminary identification result corresponding to the drug to be identified for the image block is obtained by using at least one of barcode identification technology and a drug identification model, based on the image block to be identified, and the preliminary identification result indicates one of all drug codes in the drug database; and (E) When it is determined that the at least one drug to be identified belongs to the second drug category, for each image block to be identified, the preliminary identification result is obtained by using at least one of barcode identification technology and optical character recognition technology based on the image block to be identified. The drug identification method as described in claim 1, wherein... In step (D), for each image block to be identified, the following sub-steps are included: (D-1) determining whether there is a barcode to be identified on the drug to be identified displayed in the image block to be identified; and (D-2) when it is determined that there is no barcode to be identified, using the drug identification model to identify the drug to be identified displayed in the image block to obtain an image identification result, the image identification result indicating one of all drug codes in the drug database, and using the image identification result as the initial identification result. The drug identification method as described in claim 2, wherein... In step (D), for each image block to be identified, the following sub-steps are further included: (D-3) When it is determined that the barcode to be identified exists, the barcode to be identified is identified using barcode identification technology to obtain an identified barcode number, and a barcode identification result is obtained based on the drug data set and the identified barcode number, the barcode identification result indicating that the barcode number of the drug corresponding to all drug codes in the drug data set is the same as the identified barcode number; (D-4) The drug identification model is used to identify the drug to be identified displayed in the image block to be identified to obtain the image identification result; and (D-5) The barcode identification result and the image identification result are compared to obtain the initial identification result. The drug identification method as described in claim 1, wherein... In step (E), for each image block to be identified, the following sub-steps are included: (E-1) determining whether there is a barcode to be identified on the drug to be identified displayed in the image block to be identified; (E-2) when it is determined that there is no barcode to be identified, using optical character recognition technology to identify the text on the drug to be identified displayed in the image block to be identified, so as to obtain a text content to be compared; and (E-3) comparing the text content to be compared with all the feature text content of the drug database to obtain a text recognition result, the text recognition result indicating the feature text content of the drug corresponding to all drug codes in the drug database with the highest similarity to the text content to be compared, and using the text recognition result as the initial recognition result. The drug identification method as described in claim 4, wherein... In step (E), for each image block to be identified, the following sub-steps are further included: (E-4) When it is determined that the barcode to be identified exists, the barcode to be identified is identified using barcode recognition technology to obtain an identified barcode number, and a barcode recognition result is obtained based on the drug data set and the identified barcode number. The barcode recognition result indicates that the barcode number of the drug corresponding to all drug codes in the drug data set is the same as the identified barcode number; (E-5) Optical character recognition technology is used to identify the text on the drug to be identified displayed in the image block to be identified to obtain the text content to be compared; (E-6) The text content to be compared is compared with all feature text content in the drug data set to obtain the text recognition result; and (E-7) The barcode recognition result and the text recognition result are compared to obtain the initial recognition result. The drug identification method as described in claim 1 further includes the following steps before step (A): (D) obtaining a target drug code corresponding to a target drug. The drug identification method as described in claim 6 further comprises the following steps after step (E): (F) for each drug to be identified, obtaining a final identification result corresponding to the drug to be identified based on the target drug code and all the initial identification results corresponding to the drug to be identified, the final identification result being a matching mark or a non-matching mark, the matching mark indicating that the drug to be identified matches the target drug, and the non-matching mark indicating that the drug to be identified does not match the target drug; and (G) obtaining an actual drug quantity of the target drug based on the number of matching marks in the at least one final identification result.

Images