Drug package label checking method and system

Abstract

The present invention provides a drug package label checking method, comprising steps of image acquisition, text recognition, information comparison, and anomaly detection. Specific types of label anomaly information can be intelligently classified, and feedback can be provided on the basis of the specific types, facilitating subsequent rapid handling by dispensing persons and checking persons, improving the efficiency of checking. The drug package label checking method of the present invention can solve the problem of inability to rapidly and accurately check drug package label information, and can improve the automation level, accuracy, and real-time performance of label checking. The present invention further provides a drug package label checking system and a corresponding product.

Description

A method and system for verifying medicine box labels [Technical Field]

[0001] This invention relates to the pharmaceutical industry, specifically to a method and system for verifying medicine box labels. [Background Technology]

[0002] The safe use of medicines is a crucial aspect of the pharmaceutical industry, and the government and society are highly concerned about the accuracy and safety of drug use. After a scientific and professional medical diagnosis, correct medication use is essential for disease treatment and recovery. Currently, medicines obtained by patients through various institutions and channels such as hospitals, pharmacies, and self-service vending machines often have labels indicating the drug name, drug number, dosage, method of administration / usage, and expiration date. Patients often follow the instructions on these labels. Therefore, verifying whether the information on the label matches the corresponding information in the prescription or doctor's order, and whether it matches the information recorded on the medicine itself, is a necessary step before dispensing the medicine.

[0003] The existing verification method is mainly manual. For example, in large and medium-sized hospitals and community health institutions, manual verification is generally used. Before dispensing medication at the hospital pharmacy, the information on the label of each box of medicine, such as the drug name, dosage, method of administration / usage, and expiration date, is manually checked to see if it corresponds to the corresponding information originally printed on the medicine box. Sometimes, it is also necessary to check with the corresponding information stored in the computer system and / or with the corresponding information listed in the doctor's prescription. Medication can only be dispensed if the verification is correct. If the verification finds that the label information is incorrect, it needs to be corrected. For example, the original dosage recorded on the medicine box is "1 tablet each time", but the label is printed as "2 tablets each time".

[0004] Hospital pharmacies issue a large number of medications every day. Using the manual verification method described above is not only inefficient and fails to save patients' waiting time, but also inevitably leads to verification errors by the personnel due to the large quantity of medications and the numerous checks required, which can affect patients' medication intake.

[0005] To address the error-prone nature of manual drug verification, existing technologies have proposed methods for drug appearance identification and verification. For example, Chinese invention patent application number 201810049991.0, entitled "System and Method for Automatic Drug Verification," discloses a method of capturing images of the drug box during dispensing and comparing them with pre-taken and stored photographs of six sides of the drug box. Upon finding a matching sample image, the system further checks whether the drug associated with the appearance sample image belongs to the patient's prescription and verifies the correct quantity of drugs to be dispensed. However, the technical solution proposed in this patent application still has the following problems: First, it cannot verify the accuracy of the information on the labels affixed to the drug box by the hospital, which specify the drug name, dosage, method of administration / use, and expiration date; second, it only describes identification and comparison in a general way without specific implementation plans, making it difficult to guarantee the targetedness and accuracy of the identification and comparison.

[0006] Automated medicine vending machines, also known as smart medicine cabinets, also require pre-dispensing verification. Initially designed to solve the problem of nighttime medication purchases, these machines have become an indispensable part of automated medication dispensing in hospitals, pharmaceutical retail companies, and pharmacies. They are increasingly appearing in large and medium-sized hospitals, community health centers, central pharmacies in cities, and even in airports, train stations, shopping malls, and residential communities. During the dispensing process at these machines, medication verification is also necessary to ensure accurate dispensing and the accuracy of the information on the labels.

[0007] Currently, the main focus of automated medicine vending machines in the dispensing verification process is to verify whether the medicine to be dispensed is the one listed in the prescription. For example, the authorized invention patent ZL202011542612.4, entitled "A Method for Drug Verification in Automated Medicine Vending Machines," discloses a method for pre-dispensing verification by reading the RFID tags on medicines and comparing the drug information with the prescription information stored in the hospital information system. However, this patent's technical solution still has the following problems: First, it focuses on prescription drug dispensing verification in the context of automated medicine vending machines and cannot verify the correctness of the information on the labels affixed by the hospital to the medicine box, which specify the drug name, dosage, method of administration / usage, and expiration date; Second, the use of RFID tags and the corresponding reading and comparison methods do not involve the identification and comparison of textual information such as drug name, dosage, method of administration / usage, and expiration date.

[0008] [Summary of the Invention]

[0009] Therefore, there is a need for a method and system for verifying and checking the information recorded on medicine box labels.

[0010] This invention provides a medicine box label verification method and system based on visual image recognition and machine learning algorithms, which solves the problem of not being able to accurately verify the correctness of the contents of the labels affixed to medicine boxes that specify the drug name, dosage, method of administration / use, expiration date, etc. It can be used in manual dispensing windows, automatic medicine vending machines, etc.

[0011] This invention provides a label verification method, comprising the following steps:

[0012] Image acquisition: Acquire images of the tags to obtain the acquired images;

[0013] Text recognition: Recognize the text in the acquired image and convert it into text data;

[0014] Information comparison: The text data is compared with the corresponding information in the database;

[0015] Anomaly detection: Analyze and compare the results to identify the anomaly type.

[0016] According to one embodiment of the present invention, the label verification method is used to verify the label of a medicine box, wherein the corresponding information is drug information.

[0017] According to one embodiment of the present invention, image acquisition of the label includes scanning the label or taking a picture of the label.

[0018] According to one embodiment of the present invention, an optical character recognition (OCR) method is used to recognize the text in the acquired image. The text in the acquired image is then converted into machine-readable text data.

[0019] According to one embodiment of the present invention, a machine learning algorithm model is used for anomaly detection. Specifically, for example, a machine learning algorithm model is used to label and classify anomaly information to optimize image acquisition angle, text recognition, and information comparison algorithms.

[0020] According to one embodiment of the present invention, the result of the information comparison includes: consistent and inconsistent. The anomaly detection includes analyzing the inconsistent information comparison results.

[0021] According to one embodiment of the present invention, the anomaly types include incorrect label recording and correct label recording. When the label is incorrect, the label is re-made and the image is captured. When the label is correct, the image is captured again.

[0022] According to one embodiment of the present invention, before performing the character recognition step, an image preprocessing step is further included: optimizing the acquired image to improve the accuracy of character recognition. Optimization includes noise reduction, contrast adjustment, etc.

[0023] According to one embodiment of the present invention, after the character recognition step, a data standardization step is further included: formatting the character recognition result to match the information structure in the database. The database stores standard drug information.

[0024] According to one embodiment of the present invention, after the anomaly detection step, steps such as alarm and notification, manual review and feedback, recording and reporting are performed. Specifically, this includes:

[0025] Alarms and prompts: An alarm will be issued when an anomaly is detected, and staff will be prompted to conduct a manual review;

[0026] Manual review: Staff members manually review the abnormal labels according to the prompts and provide feedback. Alternatively, they may manually review the information comparison results, or manually review the results of inconsistent information comparisons, or check the types of anomalies identified by the anomaly detection.

[0027] Records and Reports: Record all review information and exception handling details, and generate review reports.

[0028] The above three steps can be selected and used according to the actual situation.

[0029] According to one embodiment of the present invention, the drug information is updated periodically or in real time.

[0030] According to one embodiment of the present invention, the machine learning algorithm is optimized based on the feedback results of manual review.

[0031] According to one embodiment of the present invention, a decision is made based on the feedback results of manual review: whether to re-apply the label and re-acquire the image, or to directly re-acquire the image.

[0032] According to one embodiment of the present invention, the machine learning algorithm employs a One-Class Support Vector Machine (PSVM) algorithm or an Isolation Forest algorithm, etc.

[0033] According to one embodiment of the present invention, a fuzzy matching algorithm is used in the character recognition.

[0034] According to one embodiment of the present invention, a string matching algorithm is used in the information comparison. For example, the KMP (Knuth-Morris-Pratt) algorithm, the Boyer-Moore algorithm, etc.

[0035] According to one embodiment of the present invention, the text data identified and extracted in the character recognition step is subjected to pattern matching using regular expressions to verify whether the identified and extracted text data contains a correct and complete format and structure, such as dosage, batch number, etc.

[0036] This invention provides a label verification system, comprising:

[0037] The image acquisition module is used to acquire images of the label and obtain the acquired images;

[0038] The text recognition module is used to recognize the text in the acquired image and convert it into text data;

[0039] The information comparison module is used to compare the text data with corresponding information in the database;

[0040] The anomaly detection module is used to analyze and compare the results and identify the anomaly type.

[0041] According to one embodiment of the present invention, the label verification system is used to verify the medicine box label, and the corresponding information is drug information.

[0042] According to one embodiment of the present invention, the image acquisition module acquires the image by taking a picture or scanning a picture.

[0043] According to one embodiment of the present invention, the text recognition module includes an Optical Character Recognition (OCR) module for recognizing text in the acquired image and converting the text in the acquired image into machine-readable text data.

[0044] According to one embodiment of the present invention, the anomaly detection module includes a machine learning algorithm model. The machine learning algorithm model is used to label and classify anomaly information to optimize image acquisition angle, character recognition, and information comparison algorithms.

[0045] According to one embodiment of the present invention, the information comparison result of the information comparison module includes: consistent and inconsistent.

[0046] According to one embodiment of the present invention, the anomaly detection module is used to analyze the results of inconsistent information comparison.

[0047] According to one embodiment of the present invention, the anomaly types include incorrect label recording and correct label recording. When the label is incorrect, the label is re-made and the image is captured. When the label is correct, the image is captured again.

[0048] According to one embodiment of the present invention, an image preprocessing module is further included to preprocess and optimize the image acquired by the image acquisition module to improve the accuracy of text recognition. Preprocessing optimization includes noise reduction, contrast adjustment, etc.

[0049] According to one embodiment of the present invention, a data standardization module is further included, used to format the text data to match the information structure in the database. The database stores standard drug information.

[0050] According to one embodiment of the present invention, after detecting an anomaly, the anomaly detection module issues an alarm through the user interface, prompting manual review.

[0051] According to one embodiment of the present invention, a manual review module is further included for performing manual review and feedback. This includes: manually reviewing abnormal tags according to prompts and providing feedback; or manually reviewing information comparison results; or manually reviewing inconsistent information comparison results; or checking the abnormality types identified by the abnormality detection.

[0052] According to one embodiment of the present invention, a continuous optimization module is further included, which is used to optimize the machine learning algorithm based on the feedback results of manual review.

[0053] According to one embodiment of the present invention, a report generation module is further included, which is used to record all review information and anomaly handling, and generate a review report.

[0054] According to one embodiment of the present invention, it further includes a system integration module for connecting the system with other information systems such as hospitals and pharmacies to achieve data exchange, so as to realize regular or real-time updates of drug information.

[0055] According to one embodiment of the present invention, the machine learning algorithm model adopts a one-class support vector machine algorithm or an isolation forest algorithm, etc.

[0056] According to one embodiment of the present invention, a fuzzy matching algorithm is used in the character recognition module.

[0057] According to one embodiment of the present invention, a string matching algorithm is used in the information comparison module. For example, the KMP (Knuth-Morris-Pratt) algorithm, the Boyer-Moore algorithm, etc.

[0058] According to one embodiment of the present invention, regular expressions are used to perform pattern matching on the text data identified and extracted by the character recognition module to verify whether the identified and extracted text data contains a correct and complete format and structure, such as dosage, batch number, etc.

[0059] The present invention also provides a tag information verification device, comprising: a memory and a processor;

[0060] The memory is used to store program code and transmit the program code to the processor;

[0061] The processor is used to execute the label information verification method as described above, according to the instructions in the program code.

[0062] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0063] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned method.

[0064] The technical solution proposed in this invention can bring the following beneficial effects:

[0065] 1. Increased automation: Through automated image recognition and data comparison, the need for manual verification is greatly reduced, processing speed and efficiency are improved, and labor intensity is reduced.

[0066] 2. Improved accuracy: By utilizing advanced OCR technology and precise comparison algorithms, the accuracy of label information recognition has been improved, reducing review failures caused by recognition errors.

[0067] 3. Improved real-time feedback: It can provide immediate feedback on review results and immediately issue alarms upon detecting any anomalies, ensuring that problems can be handled quickly.

[0068] 4. Intelligent anomaly detection: By integrating machine learning algorithms, it can intelligently identify and learn abnormal patterns, mark and classify abnormal information, optimize image acquisition angle, text recognition and information comparison algorithms, and continuously improve detection accuracy.

[0069] 5. To ensure medication safety, more accurate verification ensures the safety of patients' medication use. [Attached Image Description]

[0070] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0071] Figure 1 is a diagram of the main steps of the label verification method of the present invention.

[0072] Figure 2 is a flowchart of the main steps of the label verification method of the present invention.

[0073] Figure 3 shows the logical relationship between the machine learning model and manual review in the label review method of this invention.

[0074] Figure 4 is a diagram showing the logical relationship between the machine learning model and manual review in the label review method of this invention (II).

[0075] Figure 5 is a diagram showing the logical relationship between the machine learning model and manual review in the label review method of this invention (Figure III).

[0076] Figure 6 is a flowchart of the label verification method of the present invention, which includes optional steps.

[0077] Figure 7 is a diagram of the main modules of the label verification system of the present invention.

[0078] Figure 8 is a block diagram of the label verification system including optional modules of the present invention.

Detailed Implementation Methods

[0079] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] The medicine box label verification method and system of the present invention are mainly used by dispensing parties such as hospitals, pharmacies, and automatic medicine vending machines to verify the drug label information before and during dispensing. It can improve the verification accuracy, accurately distinguish the specific types of label information verification anomalies, and provide corresponding feedback based on the specific types. It reduces the number and workload of manual verification, improves the efficiency of label information verification, and improves the accuracy and efficiency of dispensing.

[0081] The artificial intelligence machine learning method of the present invention can intelligently distinguish the specific types of abnormal information in the above-mentioned label information verification. The present invention uses a trained artificial intelligence model that can intelligently and accurately distinguish the specific types of abnormal information and provide feedback based on the specific types, which facilitates the dispensing personnel and reviewers to quickly carry out subsequent processing and improves the verification and review efficiency.

[0082] Figures 1-6 are flowcharts and logical relationship diagrams of the label verification method of the present invention.

[0083] As shown in Figure 1, the present invention provides a method for verifying medicine box labels, comprising the following steps:

[0084] Image acquisition: The medicine box label is captured to obtain the acquired image.

[0085] Text recognition: Recognize the text in the acquired image and convert it into text data.

[0086] Information comparison: The text data is compared with the drug information in the database.

[0087] Anomaly detection: Analyze and compare the results to identify the anomaly type.

[0088] In the image acquisition step, the medicine box label is image acquired, including scanning the medicine box label or taking a photograph of the medicine box label. Specifically, the scanning device or the photographing device can be handheld or fixed.

[0089] The handheld scanning / photographing device, such as a barcode scanner similar to those at a cashier in a supermarket or restaurant, or other similar devices like a mobile phone, personal PDA, or medical mobile terminal, is used by a person holding the scanning / photographing device, adjusting the scanning / photographing angle, and then scanning / photographing the item after the item to be scanned / photographed is placed still.

[0090] The fixed scanning / photographing device, for example, has a medicine placement area and a scanner / camera. The relative position of the scanner / camera to the medicine placement area is fixed, allowing for scanning / photographing of medicine labels placed in the medicine placement area without the need for manual handling. Alternatively, for example, in a supermarket self-checkout, the scanning / photographing device is in a fixed position, requiring manual placement of the item to be scanned / photographed close to the fixed-position scanning / photographing device for scanning / photographing.

[0091] In the text recognition step, the text in the acquired image is recognized and converted into text data. A specific method for text recognition is, for example, using Optical Character Recognition (OCR) to recognize text in the acquired image. The converted and recognized text data is, for example, in string format.

[0092] OCR can convert encoded information in an image into machine-readable text format, and the accuracy of recognition can be improved through image preprocessing (such as denoising, binarization, and tilt correction). For example, deep learning models can be used to process images, thereby improving adaptability to complex backgrounds and different print qualities.

[0093] In character recognition, inconsistencies may occur, and there are various reasons and situations for these inconsistencies, including at least:

[0094] 1. Content error, that is, the text content in the label itself is incorrect. For example, the dosage should be "2 tablets twice a day", but the label is wrongly written as "2 tablets three times a day".

[0095] 2. Abnormal photo capture, resulting in: failure to recognize any text; or failure to recognize the complete label content, such as only capturing "twice a day, each" but not "2 pieces per time"; or recognizing the correct text as the wrong text, such as recognizing "twice a day, 2 pieces each time" as "2 times per bite, 2 jin each time", etc.

[0096] The aforementioned photographic anomalies include the following situations that may occur when taking photos handheld: the label and text in the photo are distorted due to the handheld angle not being aligned properly; the photo is blurry or ghosted when taken while moving; and the photo is blurred when taken while not in focus.

[0097] The aforementioned photographic anomalies also include the following situations that may occur with fixed-position photography: blurry or unclear photos due to incorrect label placement angles or focus failures; incomplete images due to the actual placement of the medicine deviating from the required placement area; and the medicine being placed too close or too far from the scanner / camera, exceeding the scanner / camera's closest focusing distance or furthest shooting distance. In some cases, even with fixed scanning / photographing equipment where the relative position of the scanner / camera to the medicine placement area can be adjusted, the above problems may still occur.

[0098] The aforementioned photo-taking anomalies also include anomalies caused by camera malfunctions.

[0099] In addition, abnormal photos may also be caused by unclear label printing.

[0100] In the actual process of text recognition, the applicant found that even without the aforementioned photographic anomalies, using only Optical Character Recognition (OCR) methods sometimes resulted in recognition errors, especially when drug labels might be worn, stained, or poorly printed. Therefore, the applicant proposes combining a fuzzy matching algorithm with traditional OCR methods to improve the error tolerance of the recognition comparison. Specifically, the fuzzy matching algorithm can identify approximately matching text by calculating the similarity between characters (such as edit distance), thereby correcting or confirming the output of the OCR.

[0101] The fuzzy matching algorithm mentioned is, for example, the edit distance algorithm. Edit distance (also known as Levenshtein distance) is used to measure the difference between two strings. It is defined as the minimum number of single-character edit operations required to transform one string into another. These edit operations typically include inserting, deleting, and replacing a character at a certain position in the string.

[0102] Edit distance algorithms are typically implemented using dynamic programming. They construct a matrix to store solutions to subproblems, thus avoiding redundant computation. The basic steps are as follows:

[0103] 1. Initialize the matrix: Create a matrix of size (m+1)×(n+1), where m and n are the lengths of the two input strings, respectively. The rows of the matrix represent each prefix of the first string, and the columns represent each prefix of the second string. The [i][0] and [0][i] of the matrix represent the edit distance between the empty string and the first i characters of either the first string or the second string.

[0104] 2. Fill the matrix: Iterate through all characters of both strings. For each element in the matrix, calculate the edit distance according to the following rules:

[0105] If two characters are equal, then the edit distance of the current position is equal to the edit distance of the top-left corner position.

[0106] If the two characters are not equal, then the edit distance at the current position is the minimum value of the edit distance plus 1 among the following three operations:

[0107] Insertion: Increase the edit distance of the element above by 1, i.e., [i-1][j]+1.

[0108] Deletion: Increase the edit distance of the left element by 1, i.e., [i][j-1]+1.

[0109] Replace: Increase the edit distance of the top left element by 1, i.e., [i-1][j-1]+1.

[0110] 3. Obtain the result: The last element [m][n] of the matrix contains the minimum number of edits required to convert the first string into the second string.

[0111] The time complexity of the edit distance algorithm is O(m*n), where m and n are the lengths of the two strings.

[0112] In the information comparison step, the text data extracted in the data extraction step is compared in real time with the correct drug label data that has been entered and stored in the database to determine whether the text data is the same as the correct label data of the drug.

[0113] When there are multiple versions of the drug label data, for example, doctors may provide two or more dosage recommendations for the same drug based on different patients' actual conditions or when considering co-administration when a patient is taking multiple medications simultaneously. Therefore, there may be two or more correct drug labels for the same drug. In such cases, information comparison will determine whether the aforementioned text data is identical to any correct label data for that drug, and further, whether the aforementioned text data is consistent with the diagnosis, medication list, and medical orders written by the doctor for the patient.

[0114] This invention includes at least two methods. The first method involves first finding the label data of the drug in a database based on the collected and identified drug name and number, and then comparing the collected and extracted text data with the label data of the drug already found in the database. The second method does not require finding the label data of the drug in the database first; instead, it compares all the collected and identified label data of the drug with the label data of all drugs in the database.

[0115] This invention employs efficient string matching algorithms, such as the KMP algorithm and the Boyer-Moore algorithm, to quickly compare OCR recognition results with tag information in the database.

[0116] Referring to Figure 2, if the collected and identified label data of the drug matches the correct label data of the drug stored in the database after information comparison, the information comparison result is consistent. At this time, the drug label detection and verification work is completed, and the drug can be distributed. If they do not match, the information comparison result is inconsistent. In this case, the following anomaly detection steps of this invention need to be performed, instead of directly handing over the verification, differentiation, and feedback to staff as in the prior art.

[0117] If the anomaly detection result indicates that the label is incorrect, the system will notify you that the label needs to be remade and affixed, and then image acquisition should be performed again. If the anomaly detection result indicates that the label is correct, the system will notify you that image acquisition should be performed again.

[0118] In the anomaly detection step, a machine learning algorithm model is used to identify and analyze inconsistent information comparison results, classifying them into specific types. This invention employs a machine learning algorithm model to label and classify anomalous information, optimizing image acquisition angle, text recognition, and information comparison algorithms.

[0119] This invention addresses several common scenarios of inconsistencies in information comparison results by designing a machine learning algorithm model. The model first manually verifies the inconsistencies and provides a classification result, which is then used as training sample data. Once the training sample data meets the quantity requirements, the model can automatically determine the parameter features used to classify the specific types of inconsistencies. For subsequent inconsistencies in information comparison results, the model will automatically identify and analyze them based on the parameter features to classify the specific types.

[0120] In actual drug label verification, the proportion of inconsistencies is relatively low. Furthermore, through the efficient string matching algorithm, fuzzy matching algorithm, and pattern matching using regular expressions employed in this invention, the proportion of inconsistencies is further reduced. To address this issue, this invention, building upon the innovative use of anomaly detection algorithms to categorize inconsistencies, further employs One-Class Support Vector Machine (PSVM) and Isolation Forest algorithms to improve the accuracy and efficiency of anomaly detection. The PSVM and Isolation Forest algorithms are described below.

[0121] One-Class Support Vector Machine (SVM):

[0122] 1. By training on normal drug label data, One-Class SVM learns a decision boundary that maps and encloses the normal data in a high-dimensional feature space.

[0123] 2. Kernel functions, such as Gaussian kernels (RBF), are used to process nonlinearly separable data.

[0124] 3. The gamma parameter controls the influence range of a single training sample. For the RBF kernel, it defines the distribution density of sample points in the feature space.

[0125] 4. The nu parameter is a value between 0 and 1 that specifies the allowable proportion of outliers, thus affecting the construction of the decision boundary.

[0126] 5. The decision_function method is used to calculate the distance from a new data point to the decision boundary. If the distance is less than a certain threshold, it is considered normal data; otherwise, it is considered abnormal.

[0127] Isolation Forest:

[0128] 1. By constructing multiple isolated trees, each tree randomly selects features and split points to quickly isolate data points.

[0129] The 2.n_estimators parameter controls the number of isolated trees constructed; more trees will improve the accuracy and stability of the detection.

[0130] 3. The max_samples parameter defines the maximum number of samples used when building each tree.

[0131] 4. The contamination parameter represents the proportion of outliers in the dataset. This parameter affects the construction of the isolation tree and the calculation of outlier scores.

[0132] 5. The Isolation Forest algorithm determines the outlier score of each data point by calculating the average path length of each data point across all isolated trees. The shorter the path, the more likely it is to be an outlier.

[0133] In this invention, the aforementioned machine learning algorithm model is further optimized through subsequent manual review, primarily addressing situations where the machine learning algorithm model misclassifies data types. For example, if the machine learning algorithm model classifies a tag indicating a photo anomaly as having content errors, without intervention, since the tag itself is correct, the re-created tag will likely still be classified as having content errors by the machine learning algorithm model, leading to repeated re-creation.

[0134] For example, when a machine learning algorithm classifies a label with incorrect content as a photo error, without intervention, due to the error in the label itself, there will be two possible outcomes after retaking the photo: one is that it will still be classified as a photo error, leading to repeated retaking; the other is that it will be identified as having incorrect content, prompting a re-labeling.

[0135] For the aforementioned situations involving repeated re-pasting and re-photographing, manual verification by staff is required. The results of this manual verification are then fed back to the machine learning algorithm model. The model will automatically learn and optimize based on the feedback, such as adjusting and optimizing parameter features to adapt to new situations that arise in actual operation, thereby improving the accuracy and efficiency of detection and verification.

[0136] Referring to Figures 3, 4, and 5, the logical relationship between the machine learning model and manual review in the label review method of the present invention is as follows.

[0137] Taking Figure 3 as an example, if the actual content of the label is correct, but the anomaly detection result is inconsistent due to an abnormal photo capture, if the machine learning model determines that the photo capture is abnormal, then a new photo needs to be taken, and the image acquisition process needs to be repeated. If the machine learning model determines that the content is incorrect, then manual verification is required. Since the actual content of the label is correct, the manual verification will confirm that it is actually a photo capture error and will issue an instruction / feedback to retake the photo. At the same time, the result of the manual verification will be fed back to the machine learning model for optimization.

[0138] Taking Figure 4 as an example, in cases where no anomalies are detected during photography, but the anomaly detection results are inconsistent due to errors in the label content itself, if the machine learning model determines that the content is incorrect, the label needs to be remade and reattached, and then the image should be captured again. If the machine learning model determines that the photography is abnormal, manual review is required. Since the actual label content is incorrect, manual review will confirm that it is indeed a content error and will provide instructions / feedback to remake and reattach the label. At the same time, the result of manual review will be fed back to the machine learning model for optimization.

[0139] Taking Figure 5 as an example, after image acquisition and text recognition, the machine learning model performs a consistency judgment. If the judgment result indicates an image capture error, the next step needs to be determined based on the actual situation. If the actual situation is that the label itself is incorrect, manual review is required. Since the actual situation is a content error, manual review will provide instructions / feedback to remake and re-affix the label. Simultaneously, the result of this manual review will be fed back to the machine learning model for optimization. If the actual situation is an image capture error, the image needs to be retaken, and then image acquisition will be repeated. Similarly, if the machine learning model determines that the content is incorrect, the next step needs to be determined based on the actual situation. If the actual situation is an image capture error, manual review is required. Since the label itself is not incorrect but the image capture is abnormal, manual review will confirm that the image capture is indeed abnormal and will provide instructions / feedback to retake the image. Simultaneously, the result of this manual review will be fed back to the machine learning model for optimization. If the actual situation is a content error, the image needs to be remade and re-affixed, and then image acquisition will be repeated.

[0140] It should be noted that image capture is required after the re-painting process. Taking new photos as described above means that image capture needs to be performed again.

[0141] By performing the above-mentioned anomaly detection step after the information comparison step, the technical solution proposed in this invention can bring the following beneficial effects:

[0142] 1. Increased automation: Through automated image recognition and data comparison, the need for manual verification is greatly reduced, processing speed and efficiency are improved, and labor intensity is reduced.

[0143] 2. Improved accuracy: By utilizing advanced OCR technology and precise comparison algorithms, the accuracy of label information recognition has been improved, reducing review failures caused by recognition errors.

[0144] 3. Improved real-time feedback: The system can provide immediate feedback on the review results and immediately issue an alarm if any abnormality is detected, ensuring that problems can be dealt with quickly.

[0145] 4. Intelligent anomaly detection: By integrating machine learning algorithms, the system can intelligently identify and learn anomaly patterns, continuously optimizing detection accuracy.

[0146] 5. To ensure medication safety, more accurate verification ensures the safety of patients' medication use.

[0147] Figure 6 illustrates the medicine box label verification method of the present invention, which includes optional method steps. Based on the method in Figure 1, it may also include the following method steps.

[0148] Before the text recognition step, an image preprocessing step is also included: optimizing the acquired image to improve the accuracy of text recognition. Optimization includes noise reduction, contrast adjustment, etc.

[0149] Following the character recognition step, a data standardization step is also included: formatting the character recognition results to match the information structure in the database. This database stores standard drug information.

[0150] Following the anomaly detection step, the process includes alarm and notification, manual review and feedback, recording and reporting, and more. Specifically, this includes:

[0151] Alarms and prompts: When an anomaly is detected, an alarm will be issued and staff will be prompted to verify the information.

[0152] Manual review and feedback: Staff members manually review the abnormal labels according to the prompts and provide feedback. This includes providing feedback on error messages.

[0153] Records and Reports: Record all review information and exception handling details, and generate review reports.

[0154] The machine learning algorithm is optimized based on the feedback from manual review.

[0155] According to one embodiment of the present invention, the drug information is updated periodically or in real time.

[0156] According to one embodiment of the present invention, the present invention uses regular expressions to perform pattern matching on the text data identified and extracted in the character recognition step to verify whether the identified and extracted text data contains a correct and complete format and structure, such as dosage, batch number, etc.

[0157] In the specific verification and review of drug labels, in addition to the aforementioned label content errors (e.g., the dosage should be "2 tablets twice daily," but the label incorrectly states "2 tablets three times daily"), errors involving omitted items may also occur. For example, the label should read "100mg / tablet, 2 tablets twice daily," but it is actually written as "100mg / tablet, twice daily." Simple string comparison methods cannot identify the omission of "2 tablets each time." This invention uses regular expressions for pattern matching of text data, which optimizes the string comparison algorithm, identifies errors such as omitted items, and ensures that the text data used for comparison has the correct and complete format and structure.

[0158] Figures 7 and 8 show the main module diagram and optional module diagram of the label verification system of the present invention.

[0159] Referring to Figures 7 and 8, the present invention provides a label verification system, comprising:

[0160] The image acquisition module is used to acquire images of the label and obtain the acquired images;

[0161] The text recognition module is used to recognize the text in the acquired image and convert it into text data;

[0162] The information comparison module is used to compare the text data with corresponding information in the database;

[0163] The anomaly detection module is used to analyze and compare the results and identify the anomaly type.

[0164] The label verification system is used to verify the labels on medicine boxes, and the corresponding information is drug information.

[0165] The image acquisition module acquires images by taking pictures or scanning.

[0166] The text recognition module includes an Optical Character Recognition (OCR) module, used to recognize the text in the acquired image and convert it into machine-readable text data.

[0167] The anomaly detection module includes a machine learning algorithm model. This model is used to label and classify anomalous information to optimize image acquisition angle, text recognition, and information comparison algorithms.

[0168] The information comparison results of the information comparison module include: consistent and inconsistent.

[0169] The anomaly detection module is used to analyze inconsistent information comparison results.

[0170] The anomaly types include incorrect label information and correct label information. When the label information is incorrect, the label is re-made and the image is captured again. When the label information is correct, the image is captured again.

[0171] The label verification system of the present invention also includes an image preprocessing module, which performs preprocessing optimization on the images acquired by the image acquisition module to improve the accuracy of text recognition. Preprocessing optimization includes noise reduction, contrast adjustment, etc.

[0172] The label verification system of the present invention also includes a data standardization module for formatting the text data to match the information structure in the database. The database stores standard drug information.

[0173] After detecting an anomaly, the anomaly detection module will issue an alarm through the user interface, prompting manual review.

[0174] The label verification system of the present invention also includes a manual verification module for performing manual verification and providing feedback. This includes: manually verifying abnormal labels according to prompts and providing feedback; or manually verifying information comparison results; or manually verifying inconsistent information comparison results; or checking the anomaly types identified by the anomaly detection.

[0175] The label verification system of the present invention also includes a continuous optimization module, which is used to optimize the machine learning algorithm based on the feedback results of manual verification.

[0176] The label verification system of the present invention also includes a report generation module, which is used to record all verification information and abnormal handling situations, and generate a verification report.

[0177] The label verification system of the present invention also includes a system integration module, which is used to connect the system with other information systems of hospitals, pharmacies, etc., to realize data exchange and achieve regular or real-time updates of drug information.

[0178] The machine learning algorithm model adopts the One-Class Support Vector Machine algorithm or the Isolation Forest algorithm, etc.

[0179] The text recognition module employs a fuzzy matching algorithm.

[0180] The information comparison module employs string matching algorithms, such as the KMP (Knuth-Morris-Pratt) algorithm and the Boyer-Moore algorithm.

[0181] The text data identified and extracted by the character recognition module is used to perform pattern matching using regular expressions to verify whether the identified and extracted text data contains the correct and complete format and structure, such as dosage and batch number.

[0182] The present invention also provides a tag information verification device, comprising: a memory and a processor;

[0183] The memory is used to store program code and transmit the program code to the processor;

[0184] The processor is used to execute the label information verification method as described above, according to the instructions in the program code.

[0185] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0186] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned method.

[0187] The handheld and fixed scanning / photographing devices of the present invention are merely examples. The scanning and photographing devices of the present invention are not limited to handheld or fixed types, nor are they limited to the specific implementation methods described in the specification. Any device capable of scanning and photographing drug labels falls within the scope of the present invention.

[0188] In this invention, the drug label may include full information such as drug name, drug number, dosage, method of administration / use, and expiration date, or it may only include partial information, such as only drug name, method of administration / use, or only drug number and method of administration.

[0189] Those skilled in the art can determine that the drug label information verification method, system, and corresponding artificial intelligence model in this invention can be applied to label information verification scenarios for items other than drugs, such as, but not limited to, labels affixed by supermarkets to near-expiry products, specifying the product name, production date, shelf life, and before which to consume / use the product, as well as various other similar scenarios, which will not be listed here.

Claims

1. A method for verifying label information, characterized in that, The method includes: Image acquisition: Acquire images of the tags to obtain the acquired images; Text recognition: Recognize the text in the acquired image and convert it into text data; Information comparison: The text data is compared with the corresponding information in the database; Anomaly detection: Analyze and compare the results to identify the anomaly type.

2. The label information verification method according to claim 1, characterized in that: The image is acquired by taking a picture or scanning a document; the text is recognized by optical character recognition (OCR).

3. The label information verification method according to claim 1, characterized in that: The anomaly detection is performed using a machine learning algorithm model.

4. The label information verification method according to claim 1, characterized in that: The results of the information comparison include: consistent, inconsistent; The anomaly detection includes analyzing inconsistent information comparison results.

5. The label information verification method according to claim 1, characterized in that: The anomaly types include incorrect labeling and correct labeling. If the label is incorrect, re-label it and then capture the image. If the label information is correct, the image will be captured again.

6. The label information verification method according to claim 1, characterized in that, Also includes: Image preprocessing involves optimizing the acquired image before performing the text recognition. Data standardization involves formatting the text recognition results to match the information structure in the database.

7. The label information verification method according to claim 1, characterized in that, Also includes: Manual review involves manually checking the anomaly types identified by the anomaly detection.

8. The label information verification method according to claim 7, characterized in that, Also includes: Based on the results of the manual review, the machine learning algorithm model used in the anomaly detection is optimized.

9. The label information verification method according to claim 7, characterized in that, Also includes: Based on the results of the manual review, the following decision is made: Re-label and capture images; or Reacquire the image.

10. A label information verification method according to any one of claims 3 and 8, characterized in that: The machine learning algorithm used is either a single-class support vector machine algorithm or an isolated forest algorithm.

11. The label information verification method according to claim 2, characterized in that: The text recognition process employs a fuzzy matching algorithm.

12. The label information verification method according to claim 1, characterized in that: A string matching algorithm is used in the information comparison.

13. The label information verification method according to claim 1, characterized in that: Regular expressions are used to perform pattern matching on the text data.

14. A label information verification method according to any one of claims 1-13, characterized in that: The label information verification method is used to verify the information on medicine box labels.

15. A label verification system, comprising: The image acquisition module is used to acquire images of the label and obtain the acquired images; The text recognition module is used to recognize the text in the acquired image and convert it into text data; The information comparison module is used to compare the text data with corresponding information in the database; The anomaly detection module is used to analyze and compare the results and identify the anomaly type.

16. A label information verification system according to claim 15, characterized in that: The image acquisition module acquires images using either photographing or scanning methods. The text recognition module uses optical character recognition (OCR) to recognize the text.

17. A label information verification system according to claim 15, characterized in that: The anomaly detection module includes a machine learning algorithm model, which is used to perform the anomaly detection.

18. A label information verification system according to claim 15, characterized in that: The information comparison results of the information comparison module include: consistent, inconsistent; The anomaly detection module is used to analyze inconsistent information comparison results.

19. A label information verification system according to claim 15, characterized in that: The anomaly types include incorrect labeling and correct labeling. If the label is incorrect, re-label it and then capture the image. If the label information is correct, the image will be captured again.

20. A label information verification system according to claim 15, characterized in that, Also includes: Image preprocessing module: used to preprocess and optimize the images acquired by the image acquisition module; Data standardization module: Used to format the text data to match the information structure in the database.

21. A label information verification system according to claim 1, characterized in that, Also includes: The manual review module is used to manually check the anomaly types identified by the anomaly detection.

22. The label information verification system according to claim 21, characterized in that, Also includes: Continuous optimization module: used to optimize the machine learning algorithm model in the anomaly detection module based on the results of the manual review.

23. A label information verification system according to any one of claims 17 and 22, characterized in that: The machine learning algorithm in the anomaly detection module adopts either the single-class support vector machine algorithm or the isolated forest algorithm.

24. A label information verification system according to claim 16, characterized in that: The text recognition module employs a fuzzy matching algorithm.

25. A label information verification system according to claim 15, characterized in that: A string matching algorithm is used in the information comparison module.

26. A label information verification system according to claim 15, characterized in that: The information comparison module is used to perform pattern matching on the text data using regular expressions.

27. A label information verification system according to any one of claims 15-26, characterized in that: The label information verification system is used to verify the information on medicine box labels.

28. A label information verification device, characterized in that, include: Memory and processor; The memory is used to store computer program code; The processor executes the computer program code to implement the label information verification method according to any one of claims 1-14.

29. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-14.

30. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1-14.

Images