A drug verification and automatic distribution method based on machine vision

Abstract

The present invention discloses a method for checking and automatically distributing medicines based on machine vision, comprising the steps of determining a medicine review area, obtaining images of medicines picked by a pharmacist through an acquisition device, extracting a feature vector of the medicines, constructing a medicine recognition model, inputting the feature vectors into the model for recognition, obtaining medicine identification information, comparing the information with data in a medicine database, and deciding on automatic distribution of the medicines or providing feedback on abnormal information based on the comparison results. The present invention significantly improves the accuracy of medicine identification and the robustness of the system by reasonably setting the position of the acquisition device, optimizing the medicine feature vector processing flow, and introducing an automated distribution and abnormality feedback mechanism, thereby effectively reducing errors in manual operation and improving the efficiency and safety of medicine checking and distribution.

Description

Technical Field

[0001] The present invention relates to the technical field of drug image processing, and in particular to a drug verification and automatic distribution method based on machine vision. Background Art

[0002] With the continuous advancement of digitalization and intelligentization in the medical field, the demand for automated drug management systems is increasing. Traditional drug verification and distribution processes rely primarily on manual operations. While this method may meet requirements in small-scale scenarios, the limitations of manual operations are becoming increasingly apparent as medical institutions expand and the variety and quantity of drugs increase. Furthermore, visual fatigue and negligence during manual verification can easily lead to misidentification and misdispensing of drugs, compromising patient safety. Furthermore, drug packaging varies greatly, with some packaging being highly similar, increasing the difficulty and potential for error in manual identification. To reduce human error and improve the efficiency of drug verification and distribution, machine vision recognition technology has become a research hotspot. Machine vision technology can automatically identify drugs through image acquisition and intelligent algorithms, eliminating the need for manual operation, reducing error rates, and improving overall drug verification efficiency.

[0003] However, existing machine vision recognition technology still faces some technical problems in the verification and distribution of medicines. For example, the image acquisition area of ​​the medicine is not fully considered before being fed into the model for processing, resulting in low image recognition in the image acquisition area of ​​the medicine, which limits the robustness of the model. In particular, when the medicine packaging is similar, the image quality is greatly affected by ambient light, and the size of the acquisition area is uncertain, the recognition accuracy and reliability of the model cannot reach the ideal goal. In addition, if the solution adopts a fully automatic method, the drug information of new drugs needs to be entered immediately, otherwise the model will be difficult to recognize and affect subsequent recognition operations. Model recognition requires a large amount of labeled data to be fed into the model for training. On the one hand, it is easy to make the recognition model cost too high. On the other hand, there is currently a lack of large labeled data sets in China. At the same time, the model must be trained and convergence judged to ensure that the model can achieve accurate recognition in large-scale data sets. This invisibly increases the high requirements for the model, making it difficult to apply in some hospitals. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method for checking and automatically dispensing medicines based on machine vision to solve the problems mentioned in the background technology.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a method for checking and automatically dispensing medicines based on machine vision, comprising:

[0007] Determine a drug review area, collect images of drugs selected by a pharmacist in the drug review area using a collection device, obtain drug feature vectors based on the drug images, and store the drug information selected by the pharmacist in a drug database;

[0008] Constructing a drug recognition model, inputting the drug feature vector into the drug recognition model, performing drug recognition, obtaining drug recognition information, and storing the drug recognition information in a drug recognition database;

[0009] The drug identification database is compared with the drug information database, and according to the comparison result, the drug is distributed or abnormal feedback of the drug information is selected.

[0010] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, determining the drug review area includes:

[0011] Consider the total length L of the drug conveyor belt t With conveyor belt speed v t and the time the drug stays in the review area t r The minimum value of the drug review area determines the length L r ;

[0012] Consider the maximum height H of the drug max The height difference h between the maximum height of the collection equipment and the drug c , determine the height H of the review area r .

[0013] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, it further includes:

[0014] A positioning mark is set at the center of the drug review area, and the positions of the entrance and exit sensors in the review area are set by the positioning mark.

[0015] As a preferred embodiment of the machine vision-based drug verification and automatic dispensing method of the present invention, wherein: collecting images of drugs picked by a pharmacist in the drug review area by a collection device includes:

[0016] According to the height H of the review arear and the maximum height H of the drug max Determine the height H of the acquisition device at the horizontal viewing angle c , and according to the horizontal viewing angle of the acquisition equipment and the width W of the drug p , get the distance D between the acquisition device and the conveyor belt from the side view c ;

[0017] According to the conveyor belt speed v t and the length L of the drug in the field of view of the collection device v , determine the interval time for each camera to collect drug images and the shooting interval Δt between drug images.

[0018] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, obtaining a drug feature vector based on the drug image includes:

[0019] For the obtained drug feature vector V f ,The normalization method is used to standardize each component in the drug feature vector into an interval form.

[0020] As a preferred embodiment of the machine vision-based drug verification and automatic dispensing method of the present invention, a drug recognition model is constructed, the drug feature vector is input into the drug recognition model, and drug recognition is performed to obtain drug recognition information, including:

[0021] The components of the standardized drug feature vector are passed to the feature fusion layer, which performs a weighted summation on each component of the different drug feature vectors, and the result of the weighted summation is passed to the convolution layer to form a low-dimensional feature map;

[0022] The low-dimensional feature map is linearly combined through the feature expansion layer to generate a new drug feature vector, and the low-dimensional feature map and the new drug feature vector are fused by two fully connected layers. The output of the drug recognition model is mapped to a probability distribution using the Softmax activation function, and the probability value of each drug category is output. The category with the highest probability value is selected as the drug recognition result to obtain drug recognition information.

[0023] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, it further includes:

[0024] The drug identification information includes drug category ID, drug identification confidence and drug difference analysis.

[0025] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, it further includes:

[0026] The two fully connected layers contain n and m neurons respectively, and a Dropout layer is introduced after the fully connected layer. For each fusion operation of the low-dimensional feature map and the new drug feature vector, half of the neurons are randomly discarded.

[0027] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, the drug difference analysis includes:

[0028] By calculating the highest probability value and the second highest probability value for each drug category, a difference analysis result is generated, and the drug recognition model is evaluated based on the difference analysis result.

[0029] As a preferred embodiment of the machine vision-based drug verification and automatic distribution method of the present invention, the drug identification database is compared with the drug information database, and according to the comparison result, the drug is selected to be distributed or abnormal drug information is fed back, including:

[0030] According to the drug category ID, each record in the drug identification database is compared with the corresponding record in the drug information database. If the current drug identification confidence is equal to or lower than the average drug identification confidence, the currently compared drug information record is marked as "drug information abnormal", the drug distribution operation is stopped, and the pharmacist is notified to recheck the drug information; otherwise, the drug distribution operation is executed, the currently compared drug information record is marked as "drug information normal", and the current timestamp of the successfully distributed drug, the distributed patient information, and the current picking pharmacist information are stored in the drug management database.

[0031] Compared with the prior art, the invention has the following beneficial effects:

[0032] 1. The present invention rationally sets the position and angle of the acquisition equipment within the drug review area, ensuring high-quality acquisition of drug images, enabling the drug recognition model to more accurately obtain drug feature vectors, thereby improving the accuracy of drug recognition;

[0033] 2. Normalization and difference analysis of drug feature vectors improves the model's adaptability to different drug packaging and complex environments. Regardless of the diversity and similarity of drug packaging or changes in image acquisition conditions, the model can maintain high recognition accuracy without the need for frequent iterations, thereby ensuring the stability of drug verification.

[0034] 3. Through the semi-automated drug verification and distribution process, the records of the drug identification database and the drug information database can be automatically compared based on the drug identification information. When the drug identification confidence level is lower than the preset threshold, abnormal feedback will be issued in a timely manner to notify the pharmacist to recheck, thereby significantly reducing drug picking errors caused by manual operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0036] Figure 1 This is an overall flow chart of a method for checking and automatically dispensing medicines based on machine vision according to an embodiment of the present invention;

[0037] Figure 2 This is a comparison chart of system performance in complex drug identification, exception handling, and high-load operation of the machine vision-based drug verification and automatic distribution method according to one embodiment of the present invention;

[0038] Figure 3 A model analysis comparison diagram of drug recognition accuracy of a machine vision-based drug verification and automatic dispensing method according to an embodiment of the present invention;

[0039] Figure 4 This is a comparison chart of the system fault tolerance of the machine vision-based drug verification and automatic distribution method according to one embodiment of the present invention. DETAILED DESCRIPTION

[0040] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0043] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0044] In the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0045] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0046] Example 1

[0047] Reference Figure 1 , which is the first embodiment of the present invention, provides a method for checking and automatically distributing medicines based on machine vision, comprising:

[0048] S1. Determine a drug review area, use a collection device to collect images of drugs picked by a pharmacist in the drug review area, obtain drug feature vectors based on the drug images, and store the drug information picked by the pharmacist in a drug database.

[0049] Furthermore, considering the total length L of the drug conveyor belt t With conveyor belt speed v t and the time the drug stays in the review area t r The minimum value of the drug review area determines the length L r ;

[0050] It should be noted that the longer the drug stays in the review area (the longer the stay time, the slower the conveyor belt speed), the higher the quality of the image captured by the acquisition device, and the more accurate the drug feature vector will be. On the contrary, if the drug stays in the review area for a shorter time (the shorter the stay time, the faster the conveyor belt speed), the lower the quality of the image captured by the acquisition device, and the deviation of the drug feature vector will be too large. Since the traditional conveyor belt moves horizontally, v t It also affects the length of the review area; if v t If the speed is too fast, the length of the review area will be insufficient, and the drug may leave the review area before the image acquisition is completed, resulting in image acquisition errors or failure.

[0051] Specifically, the above description relationship is expressed as:

[0052] L r =min(L t ,v t ×t r )

[0053] It should be noted that this formula ensures that the length of the review area does not exceed the total length of the conveyor belt and adapts to different conveying speeds and dwell times;

[0054] Furthermore, considering the maximum height H of the drug max The height difference h between the maximum height of the collection equipment and the drug c , determine the height H of the review area r ;

[0055] Specifically, the height H of the review area r Expressed as:

[0056] H r =H max +2×h c

[0057] It should be noted that the total size of the review area can be determined by the height and length of the review area;

[0058] Furthermore, a positioning mark is set at the center of the drug review area, and the positions of the entrance and exit sensors in the review area are set by the positioning mark;

[0059] Specifically, a positioning mark is set at the center of the drug review area to indicate:

[0060]

[0061] Among them, x c It represents the center position of the drug review area, and x0 is the review area L r×H r The starting position on the conveyor belt;

[0062] Specifically, the positions of the inlet and outlet sensors are expressed as:

[0063] x in =x c -δ

[0064] x out =x c +δ

[0065] Among them, δ is a small distance used to ensure the sensing accuracy of the sensor;

[0066] It should be noted that the entrance and exit sensors can accurately detect when drugs enter or leave the review area to reduce manual errors. Without sensors, the collection device will miss the moment when drugs enter or leave the area, causing the collection device to be mistakenly activated before the drugs reach the review area. In addition, it can prevent multiple drugs from entering the review area at the same time, ensuring that only a single drug is processed each time.

[0067] Furthermore, according to the height H of the review area r and the maximum height H of the drug max Determine the height H of the acquisition device at the horizontal viewing angle c , and according to the horizontal viewing angle of the acquisition equipment and the width W of the drug p , get the distance D between the acquisition device and the conveyor belt from the side view c ;

[0068] Specifically, the height H of the acquisition device at the horizontal viewing angle c Expressed as:

[0069]

[0070] Specifically, the distance D between the collection device and the conveyor belt from the side view c Expressed as:

[0071]

[0072] Among them, α c Indicates horizontal viewing angle;

[0073] Furthermore, according to the conveyor belt speed v t and the length L of the drug in the field of view of the collection device v , determine the interval time for each camera to collect drug images and the shooting interval Δt between drug images;

[0074] Specifically, the interval time for collecting drug images Expressed as:

[0075]

[0076] Specifically, the shooting interval Δt of the drug image is expressed as:

[0077]

[0078] Among them, N c The number of images captured by the camera;

[0079] Furthermore, the obtained drug feature vector V f ,Using the normalization method, each component in the drug feature vector is standardized into an interval form;

[0080] Specifically, the drug feature vector V f Expressed as:

[0081] V f ={S f ,C f ,T f}

[0082] Among them, S f represents the shape feature vector, C f represents the QR code feature vector, T f represents a text feature vector;

[0083] It should be explained that the components in each drug feature vector are represented by the corresponding drug feature vector, for example, S f ={v1,v2,v3,…,v n}, C f ={w1,w2,w3,…,w n}, T f ={z1,z2,z3,…,z n}; Each v n 、w n 、z n The range of the normalized interval form of the components is [0,1];

[0084] Furthermore, the drug information in the drug database is converted into a standard feature vector V s and the obtained drug feature vector V f Compare and determine whether the drug image is the same as the actual drug information by calculating the similarity between the feature vectors;

[0085] Specifically, the similarity formula is expressed as:

[0086]

[0087] Among them, Sim(Vf ,V s ) is the similarity, ‖·‖ is expressed in Euclidean form, that is, the size or length of the vector;

[0088] It should be noted that by considering the similarity, checking whether the drug in the drug image is the drug in the drug database, it is prevented that the drug information is incorrect when entering the drug database;

[0089] S2. Construct a drug recognition model, input the drug feature vector into the drug recognition model, perform drug recognition, obtain drug recognition information, and store the drug recognition information in a drug recognition database;

[0090] Furthermore, the components of the standardized drug feature vectors are passed to the feature fusion layer, which performs a weighted summation on each component of the different drug feature vectors. The result of the weighted summation is passed to the convolution layer to form a low-dimensional feature map.

[0091] It should be noted that the low-dimensional feature map is a representation of the drug feature vector that has been compressed in the spatial dimension. Even though the size is compressed, the information remains unchanged. The advantage of this is that redundancy is reduced.

[0092] For example, a weighted sum is performed on each component in the feature vectors of different drugs, for example: v1+w1+z1 or v1+w2..., and so on. However, it should be noted that each component must be weighted summed until all components in the feature vector are summed. The weighted sum must contain at least two components, but they cannot be repeated. For example: v1+w1, where v1 and w1 have already been weighted summed, they cannot continue to appear in the weighted sum of the remaining components.

[0093] It should be noted that by introducing the feature fusion layer, the model’s ability to perceive key drug features can be enhanced, thereby reducing the interference of drug image noise, improving the robustness of the model, and enabling it to better adapt to different application scenarios;

[0094] Specifically, three convolutional layers are used, each of which consists of multiple convolution kernels with a size of 3×3. After each convolutional layer, a maximum pooling layer is added with a pooling window size of 2×2.

[0095] Furthermore, the low-dimensional feature maps are linearly combined through the feature expansion layer to generate a new drug feature vector. The low-dimensional feature map and the new drug feature vector are then fused by two fully connected layers. The output of the drug recognition model is mapped into a probability distribution using the Softmax activation function. The probability value of each drug category is output, and the category with the highest probability value is selected as the drug recognition result to obtain drug recognition information.

[0096] Specifically, the generated new drug feature vector V ext for:

[0097] V ext =V conv +V fusion

[0098] Among them, V conv is the output result of the convolution layer, V fusion is the weighted summation result;

[0099] Specifically, drug identification information includes drug category ID, drug identification confidence, and drug difference analysis;

[0100] Furthermore, by calculating the highest probability value and the second highest probability value of each drug category, a difference analysis result is generated, and the drug recognition model is evaluated based on the difference analysis result;

[0101] Specifically, the difference analysis result formula is expressed as:

[0102] ΔP=P max -P second

[0103] It should be noted that ΔP can be used to determine the confidence value of the drug recognition model for a certain drug category. If ΔP is large, it means that the model is very confident in the recognition result, that is, the drug recognition is correct; if ΔP is small, it means that there may be confusion, and the model is not sure about the recognition result, that is, there is an anomaly in the drug recognition.

[0104] Specifically, the two fully connected layers contain n and m neurons respectively, and a dropout layer is introduced after the fully connected layer. For each fusion operation of the low-dimensional feature map and the new drug feature vector, half of the neurons are randomly dropped.

[0105] It should be noted that randomly dropping neurons can prevent the drug recognition model from over-relying on certain specific neurons (drug feature vectors with extremely high feature similarity), thereby reducing overfitting. This randomness further improves the robustness of the model and prevents poor model performance due to image or data noise. In addition, the Dropout layer only uses half of the neurons for feature fusion each time, allowing the model to still make accurate predictions when faced with new drug information.

[0106] S3. Compare the drug identification database with the drug information database, and choose to distribute the drug or provide abnormal feedback on the drug information based on the comparison results;

[0107] Furthermore, based on the drug category ID, each record in the drug identification database is compared with the corresponding record in the drug information database. If the current drug identification confidence is equal to or lower than the average drug identification confidence, the currently compared drug information record is marked as "drug information abnormal", the drug dispensing operation is stopped, and the pharmacist is notified to recheck the drug information; otherwise, the drug dispensing operation is performed, the currently compared drug information record is marked as "drug information normal", and the current timestamp of the successfully dispensed drug, the dispensed patient information, and the current picking pharmacist information are stored in the drug management database;

[0108] It should be noted that by creating a drug management database, the current timestamp of the successful distribution of drugs, patient information, and the information of the current picking pharmacist can be recorded, which facilitates subsequent traceability of drugs.

[0109] Example 2

[0110] Reference Figures 2 to 4 , which is the second embodiment of the present invention, provides a machine vision-based drug verification and automatic distribution method, including: This embodiment uses a simulation experiment to compare the performance of a fully automatic method, a traditional manual operation method, and the semi-automatic method proposed in the present invention in drug verification and distribution, particularly in terms of processing speed, recognition accuracy, error rate, system fault tolerance, and exception handling capabilities; the simulation experiment environment is conducted in a hospital pharmacy and involves various drug categories, including common drugs, similarly packaged drugs, and special drugs (such as cold chain drugs); the experiment simulates the system performance under high-load operation to evaluate its effectiveness and reliability in actual application;

[0111] System configuration and parameter setting:

[0112] The fully automated system includes a high-definition camera, a robotic arm, an automatic dispensing module, and a real-time database. The camera is mounted above the conveyor belt, and the system captures images of drugs for identification and dispensing. The manually operated system involves two experienced pharmacists manually verifying drug information and dispensing drugs via the robotic arm. The semi-automatic system (the present invention) combines machine vision, manual operation, and differential analysis. The system automatically processes high-confidence drugs, while low-confidence drugs are sent to a manual review desk.

[0113] Data collection and processing:

[0114] Processing speed: records the processing speed of each system under different load conditions, including normal load (50 units / minute), high load (100 units / minute), and extreme load (150 units / minute); where "unit" refers to a single drug or drug package, and each "unit" represents an independent drug item processed by the system; "minute" refers to the time unit, that is, the number of drugs processed by the system in one minute; recognition accuracy and error rate: records the system's recognition accuracy and error rate for different types of drugs, and compares the performance of each system; system fault tolerance and exception handling: tests the system's processing capabilities under abnormal conditions (such as damaged drug labels, missing information, etc.), and records the exception handling time and system recovery time;

[0115] Experimental data processing:

[0116] The simulation experiment collects data based on 8 hours of continuous operation. The data will be divided into three types for statistics: common drugs (60%), similarly packaged drugs (30%), and special drugs (10%). The processing data of each drug includes recognition time, recognition accuracy, error rate, exception processing time, etc. The simulation results are shown in Table 1.

[0117] Table 1

[0118]

[0119] As can be seen from Table 1, first, in terms of processing speed, although the fully automatic system performs best under normal load, under high load and extreme load, due to the processing of too many drug units, the accuracy of recognition and processing begins to decline, and the error rate of the system increases. The manual operation system shows that its processing capacity is seriously insufficient under extreme load, and the processing speed drops from 30 units / minute under normal load to 20 units / minute under extreme load. The semi-automatic system of the present invention shows a stable processing speed under various load conditions, especially under extreme load, the processing speed reaches 100 units / minute, far exceeding the performance of the manual operation system. Secondly, in terms of recognition accuracy, the recognition accuracy of the fully automatic system for common drugs reaches 98.5%, but when processing similarly packaged drugs and special drugs, the accuracy drops to 95.0% and 94.0%, respectively. The recognition accuracy of the manual operation system is relatively low, especially when processing similarly packaged drugs, which is only 92.0%. The semi-automatic system of the present invention, by combining machine recognition and manual review, has a recognition accuracy of 99.0% for common drugs and a recognition accuracy of 99.0% for similarly packaged drugs and special drugs. The accuracy rates reached 97.5% and 96.5%, respectively, significantly better than those of the fully automatic system and the manual operation system. This shows that by manually reviewing low-confidence drugs, the system of the present invention effectively improves the recognition accuracy, especially in the identification of complex drugs. In addition, the semi-automatic system of the present invention also shows advantages in terms of error rate, especially when processing similarly packaged drugs and special drugs, with error rates of 2.5% and 3.5%, respectively, which are significantly lower than the error rates of the fully automatic system and the manual operation system. This low error rate is due to the system's manual intervention measures when the recognition confidence is low, which reduces misjudgments caused by complex drug characteristics. Finally, in terms of exception handling and system recovery capabilities, the semi-automatic system of the present invention shows excellent performance. When abnormal situations such as damaged drug labels or missing database information occur, the system's average exception handling time is 2 seconds, significantly better than the 3.5 seconds of the fully automatic system and the 10 seconds of the manual operation system. At the same time, the system's average recovery time is 2 minutes per hour, which is also better than the manual operation system. This shows that the system of the present invention is not only more efficient in handling sudden abnormalities, but also can resume normal operation more quickly.

[0120] In summary, the semi-automatic drug verification and distribution system of the present invention combines machine vision with manual review, which has both the efficiency of the fully automatic system and the accuracy of the manual operation system. It has obvious advantages in complex drug identification, exception handling and high-load operation. Figure 2 ; It can be seen that the system of the present invention not only improves the recognition accuracy, but also Figure 3 , which reduces the error rate and significantly enhances the system's fault tolerance, such as Figure 4 As shown, it has extremely high reliability and practical value in practical applications.

[0121] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0125] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0126] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for checking and automatically distributing medicines based on machine vision, characterized in that: include: Determine a drug review area, collect images of drugs selected by a pharmacist in the drug review area using a collection device, obtain drug feature vectors based on the drug images, and store the drug information selected by the pharmacist in a drug information database; Determining the drug review area includes: Consider the total length of the drug conveyor belt , and conveyor belt speed and the time the drug stays in the review area The product of determines the length of the drug review area The total length and the minimum value of the product; Consider the maximum height of the drug The height difference between the maximum height of the drug and the collection equipment , determine the height of the review area ; The collecting device collects images of medicines selected by the pharmacist in the medicine review area, including: According to the height of the review area and the maximum height of the drug Determine the height of the acquisition device in horizontal viewing angle , and according to the horizontal viewing angle of the acquisition equipment and the width of the medicine , get the distance between the acquisition device and the conveyor belt from the side view ; According to the conveyor belt speed and the length of time the drug remains in the field of view of the collection device , determine the interval time for each camera to collect drug images and the interval between drug image capture ; Constructing a drug recognition model, inputting the drug feature vector into the drug recognition model, performing drug recognition, obtaining drug recognition information, and storing the drug recognition information in a drug recognition database; The drug identification model is constructed, and the drug feature vector is input into the drug identification model to perform drug identification to obtain drug identification information, including: The components of the standardized drug feature vector are passed to the feature fusion layer, which performs a weighted summation on each component of the different drug feature vectors, and the result of the weighted summation is passed to the convolution layer to form a low-dimensional feature map; The low-dimensional feature map is linearly combined through the feature expansion layer to generate a new drug feature vector, and the low-dimensional feature map and the new drug feature vector are fused by two fully connected layers. The output of the drug recognition model is mapped to a probability distribution using the Softmax activation function, and the probability value of each drug category is output. The category with the highest probability value is selected as the drug recognition result to obtain drug recognition information; The drug identification database is compared with the drug information database, and according to the comparison result, the drug is distributed or abnormal feedback of the drug information is selected.

2. The machine vision-based drug verification and automatic distribution method according to claim 1, characterized in that: Also includes: A positioning mark is set at the center of the drug review area, and the positions of the entrance and exit sensors in the review area are set by the positioning mark.

3. The machine vision-based drug verification and automatic distribution method according to claim 1, characterized in that: Obtaining a drug feature vector according to the drug image includes: The obtained drug feature vector ,The normalization method is used to standardize each component in the drug feature vector into an interval form.

4. The machine vision-based drug verification and automatic distribution method according to claim 1, characterized in that: Also includes: The drug identification information includes drug category ID, drug identification confidence and drug difference analysis results.

5. The machine vision-based drug verification and automatic distribution method according to claim 1, characterized in that: Also includes: The two fully connected layers contain and neurons, and introduce a Dropout layer after the fully connected layer. For each fusion operation of the low-dimensional feature map and the new drug feature vector, half of the neurons are randomly discarded.

6. The machine vision-based drug verification and automatic distribution method according to claim 4, characterized in that: The results of the drug difference analysis include: By calculating the highest probability value and the second highest probability value for each drug category, a difference analysis result is generated, and the drug recognition model is evaluated based on the difference analysis result.

7. The machine vision-based drug verification and automatic distribution method according to claim 1, characterized in that: Comparing the drug identification database with the drug information database, and selecting to distribute the drug or provide abnormal feedback on the drug information based on the comparison result, including: Based on the drug category ID, each record in the drug identification database is compared with the corresponding record in the drug information database. If the current drug identification confidence is equal to or lower than the average drug identification confidence, the currently compared drug information record is marked as "drug information abnormal", the drug distribution operation is stopped, and the pharmacist is notified to recheck the drug information; otherwise, the drug distribution operation is executed, the currently compared drug information record is marked as "drug information normal", and the current timestamp of the successfully distributed drug, the distributed patient information, and the current picking pharmacist information are stored in the drug management database.

Images