Intelligent pharmacy dispensing method based on visual positioning and accurate grabbing

Abstract

The invention relates to an intelligent pharmacy dispensing method based on visual positioning and accurate grabbing, and the method comprises the following steps: step 1, electronic prescription analysis and dispensing task generation, step 2, visual recognition and medicine positioning, step 3, dispensing path optimization, step 4, flexible grabbing control, and step 5, medicine verification and tracing. Step 6, intelligent storage location management and system collaboration; according to the technical scheme, manual medicine picking is replaced by automatic grabbing of the mechanical arm, the idle running time is minimized in cooperation with a path optimization algorithm, and the average dispensing time of a single prescription can be controlled within 30 seconds. Compared with traditional manual medicine dispensing, the efficiency is improved by about 5 times, the requirement for a large number of prescriptions in the outpatient peak period can be effectively met, and the medicine taking waiting time of a patient is remarkably shortened.

Description

Technical Field

[0001] This invention relates to an intelligent pharmacy dispensing method, specifically an intelligent pharmacy dispensing method based on visual positioning and precise grasping, belonging to the field of medical automation and intelligent robots. Background Technology

[0002] With the advancement of medical informatization and the continuous increase in patient visits, hospital pharmacies are facing increasingly heavy workloads in dispensing medications. The traditional manual dispensing model mainly relies on pharmacists manually searching, picking, and verifying medications according to prescriptions, which has many problems.

[0003] First, there is a significant bottleneck in medication dispensing efficiency. Large tertiary hospital outpatient pharmacies can process thousands of prescriptions daily, leading to excessively long waiting times for patients during peak hours. Pharmacists need to frequently move between shelves and repeatedly search for medications, resulting in significant physical exertion, and the dispensing time per prescription is difficult to further reduce.

[0004] Secondly, the risk of medication errors cannot be completely eliminated. Although hospitals have established strict verification systems, manual operation can still lead to incorrect medications, incorrect quantities, or omissions due to fatigue, distraction, or other reasons. Statistics show that the error rate for manual medication dispensing is approximately 0.02% to 0.05%. Although the probability is low, considering the huge number of prescriptions, the actual number of errors is still significant and may pose a potential threat to patient medication safety.

[0005] Third, the level of precision in drug management is insufficient. Under a manual system, it is difficult to monitor the quantity and expiration status of drugs in each storage location in real time, easily leading to situations where some drugs are out of stock while others are stockpiled. The screening and disposal of expired drugs relies on periodic manual inventory checks, which is inefficient and carries the risk of missed detections.

[0006] Fourth, existing automated dispensing equipment has revealed certain limitations in practical applications. Some equipment can only handle standardized drug packaging of specific sizes, and is not adaptable to mixed scenarios with multiple dosage forms such as boxes, bottles, and bags with large size differences; the visual recognition system has limited ability to handle abnormal situations such as similar drug appearance, worn labels, or tilted placement; and the dispensing path planning algorithm does not perform well in optimizing large batches of parallel prescriptions, failing to fully utilize the throughput capacity of the equipment.

[0007] Fifth, there is still room for improvement in the system's intelligence level. Current equipment mostly adopts fixed operating procedures and preset grasping parameters, lacking the ability to adaptively adjust grasping strategies based on drug attributes; prescription priority management is relatively crude, and the response to emergency prescriptions, special drugs, and other situations is not flexible enough; the human-machine collaboration mode is not well designed, making it difficult for pharmacists to conveniently participate in anomaly handling and quality control.

[0008] Therefore, there is an urgent need for an intelligent pharmacy dispensing robot system that integrates advanced visual recognition technology, intelligent planning algorithms, and flexible grasping mechanisms. This system should be adaptable to various drug packaging formats, achieve efficient and accurate automated dispensing, and be deeply integrated with pharmacy management systems to provide pharmacists with effective decision support. This invention addresses these technical needs by proposing a complete intelligent pharmacy dispensing robot solution. Summary of the Invention

[0009] This invention addresses the technical problems existing in the prior art by providing a smart pharmacy dispensing method based on visual positioning and precise grasping. This solution uses a robotic arm to automatically grasp medications, replacing manual picking, and employs a path optimization algorithm to minimize idle time. The average dispensing time per prescription can be controlled to within 30 seconds. Compared to traditional manual dispensing, efficiency is improved by approximately 5 times, effectively handling the large volume of prescriptions during peak outpatient hours and significantly reducing patient waiting time.

[0010] To achieve the above objectives, the technical solution of this invention is as follows: an intelligent pharmacy dispensing robot system based on visual positioning and precise grasping. This system comprises five core components: an electronic prescription parsing engine, a visual recognition and positioning module, a dispensing path planning module, a flexible grasping control module, and a drug verification and traceability module. This enables full automation of the pharmacy dispensing process. Starting with receiving the electronic prescription, the system accurately dispenses the medication through drug location positioning, optimal path planning, precise grasping execution, and multiple verification checks. Simultaneously, it generates complete operation records to support traceability management. All modules communicate and coordinate through a unified data bus, forming a complete closed-loop operation process.

[0011] A smart pharmacy dispensing method based on visual positioning and precise grasping, the method comprising the following steps:

[0012] Step 1: Electronic Prescription Parsing and Dispensing Task Generation. This system establishes a real-time data interface with the Hospital Information System (HIS), receives electronic prescriptions, and performs structured parsing to provide task input for subsequent visual positioning and path planning. Let the received prescription set be... ,in Indicates the first Zhang's prescription, This represents the total number of prescriptions awaiting dispensing. Each prescription... It contains several drug entries, represented as ,in Indicates the first Zhang Fangzhong The identification code of the drug This indicates the quantity of the medicine dispensed. This represents the number of different types of medications included in the prescription.

[0013] The parsing engine first performs semantic validation on the prescription, checking the validity of the drug code, the rationality of the dosage, and any incompatibilities between drugs. By querying the master drug database, it maps each drug entry to specific location information. Let the drugs... The coordinates of the storage location in the warehousing system are ,in , , These represent the three-dimensional coordinates of the storage location within the warehouse space. This location coordinate information will be passed as input to the path planning module to calculate the optimal fetching order.

[0014] The system calculates dispensing priority based on prescription receipt time, patient type, and drug attributes to determine the order of prescription processing. Prescription The priority rating is defined as follows:

[0015] ,

[0016] The meanings of each variable are as follows: :prescription The priority score ranges from 100 to 100. A larger value indicates a higher priority. The urgency coefficient of a prescription is set as follows: 0 for a regular prescription, 0.5 for an emergency prescription, and 1.0 for a critical prescription. :prescription Time elapsed (in seconds) The maximum allowed waiting time set by the system (unit: seconds). The value is 1.0 if the prescription contains any specially controlled substances such as cold chain drugs or narcotic drugs, and 0 otherwise. , , Weighting coefficients, satisfying ,

[0017] Priority rating Prescriptions with higher priority will be given priority in dispensing. After the parsing engine completes prescription parsing and priority calculation, it generates a dispensing task list and pushes it to the path planning module. At the same time, it passes the warehouse location coordinates of each drug to the visual recognition module for subsequent accurate positioning and retrieval.

[0018] Step 2: Visual recognition and drug location.

[0019] The visual recognition module receives drug location information from the prescription parsing module, guides the robotic arm to the target area, and accurately identifies and positions the drugs on the shelf. This invention employs a vision system combining an industrial camera and a depth sensor, installed at both the end of the robotic arm and the top of the shelf, responsible for close-range precise recognition and global overview scanning, respectively. The identified 3D pose information of the drugs is then transmitted to the flexible gripping module as the target input for the gripping action.

[0020] 2-1 Extraction of drug appearance features,

[0021] For the collected drug images ,in and These represent the height and width in pixels, respectively. The system uses an improved convolutional neural network to extract visual features of the drugs. The network consists of a feature extraction backbone and a multi-scale detection head, outputting the bounding box coordinates, category label, and confidence score of the drug.

[0022] Let the set of detected drug bounding boxes be... Each bounding box Including top left corner coordinates lower right corner coordinates Drug categories and confidence score , This represents the total number of detected drugs. The system matches the identification results with the target drug codes passed in from the prescription parsing module, filtering out candidate drugs relevant to the current task.

[0023] 2-2 Barcode Recognition and Information Matching

[0024] Within the bounding box area, the drug barcode region is further located, and a barcode decoding algorithm is used to read key information such as the drug batch number, production date, and expiration date. Let the decoded drug information vector be... ,in: Drug barcode number, :batch number, Production date Validity period

[0025] The system will match and verify the identification results with the main drug database and calculate the information consistency score:

[0026]

[0027] The meanings of each variable are as follows: : No. The consistency score of each candidate drug, with a value of 0 or 1. The target drug code for the current task is input from the prescription parsing module. Current system date The safety shelf life threshold is usually set at 30 days.

[0028] Only consistency score Only medications that meet the criteria are considered valid targets for retrieval, thus preventing the risk of taking the wrong medication or receiving near-expiry medications at the source. The verified medication information and location data will then be passed to the next step of 3D pose estimation.

[0029] 2-3 Three-dimensional pose estimation,

[0030] Combining point cloud data acquired by the depth sensor, the system calculates the precise 3D pose of the target drug, providing the gripping point input for the flexible grasping module. Let the 3D coordinates of the drug's center point in the depth camera coordinate system be... Hand-eye calibration matrix Transform it to the coordinate system of the robotic arm's end effector:

[0031]

[0032] in: The homogeneous transformation matrix from the camera coordinate system to the end effector coordinate system is obtained in advance through hand-eye calibration. : Target position in camera coordinate system (homogeneous coordinate form). : Target position in the end effector coordinate system

[0033] Furthermore, through forward kinematics calculations of the robotic arm, the target position is transformed into a pose representation in the coordinate system of the robotic arm base, and this pose information is transmitted to the flexible grasping control module as the target point for grasping trajectory planning.

[0034] Step 3: Optimize the medication dispensing route.

[0035] The path planning module receives a list of drug storage locations from the prescription parsing module. Before the robotic arm performs the grasping operation, it calculates the optimal access order between the storage locations to minimize the total dispensing time. The planned path sequence guides the robotic arm to visit each storage location sequentially, and at each location, it calls the vision recognition module for precise positioning and the flexible grasping module to perform the grasping action.

[0036] 3-1 Problem Modeling

[0037] Let the set of drug storage locations to be captured in the current batch be . ,in This indicates the starting position of the robotic arm (dispensing window). to Indicates the location of each medicine storage unit. This represents the total number of drug storage locations. Any two locations can be defined. and The time of movement between them is The time is determined by both the kinematic characteristics of the robotic arm and the spatial distance.

[0038] Medication dispensing route This indicates the access order of a storage location. ,in And they are all different. The total dispensing time for each route is defined as:

[0039]

[0040] The meanings of each variable are as follows: :path Total medication preparation time (in seconds), From storage location Move to storage location Exercise time, set Indicates the starting position. In storage location The time required to perform the grasping action is related to the type of drug packaging and the grasping strategy, and is estimated by the flexible grasping module based on the drug attributes.

[0041] 3-2 Optimization of Genetic Algorithm

[0042] A genetic algorithm is used to search for the optimal path. This algorithm simulates the evolutionary process in nature, gradually approaching the global optimum through population iteration. The algorithm includes four main operations: initialization, selection, crossover, and mutation.

[0043] Initialization: Generates a collection of... The initial population consists of individuals, each representing a feasible path. 70% of the individuals are generated using a greedy strategy (selecting the nearest unvisited storage location to the current position each time), while 30% are generated randomly to ensure population diversity.

[0044] Fitness calculation: The fitness function is defined as the reciprocal of the drug preparation time, such that paths with shorter drug preparation times have higher fitness.

[0045]

[0046] in: :path The fitness value indicates that the better the path is; a higher value indicates a better path. A small constant to prevent division by zero, usually taken as 0.001.

[0047] Selection operation: A roulette wheel selection strategy is used, where the probability of each individual being selected is proportional to its fitness. Let the i-th individual in the population... Individual (path) The probability of selection is ,in Individuals with high fitness are more likely to be carried into the next generation.

[0048] Crossover operation: The Order Crossover (OX) operator is used to generate offspring. The specific steps are as follows: a segment of the parent path is randomly selected and directly copied to the same position in the offspring; the remaining positions are filled with the location numbers that did not appear in the previous parent path, following the order of the previous parent path. This operator ensures that the generated offspring path is still a valid path (each location is visited exactly once).

[0049] Mutation operation: with probability (Usually 0.1-0.2) Mutate the individual. The mutation method is to randomly select two positions in the path and swap their storage location numbers, introducing random perturbation to escape the local optimum.

[0050] The algorithm iterates until it reaches a preset number of generations (e.g., 100 generations) or converges after several consecutive generations when the fitness has not significantly improved. It then outputs the best individual from each generation as the optimal path. .

[0051] 3-3 Dynamic Path Adjustment

[0052] In actual medication dispensing, dynamic events such as new prescription insertions and drug shortages may occur. The system employs an incremental path update strategy: for newly added drug storage locations, the system assesses the time cost of inserting it into each of the remaining locations on the current path and selects the insertion point with the lowest cost; for out-of-stock locations, the system directly removes them from the path and records the replenishment information, while simultaneously notifying the prescription parsing module to update the prescription status. This dynamic adjustment mechanism enables the system to flexibly respond to real-time changes and maintain high medication dispensing efficiency.

[0053] After path planning is completed, the system schedules the robotic arm to visit each storage location sequentially according to the optimal path sequence. At each storage location, the vision recognition module first accurately locates the medicine, and after confirming the target pose, the flexible grasping module performs the grasping action.

[0054] Step 4 Flexible gripping control

[0055] The flexible gripping module receives drug pose information from the visual recognition module and packaging attribute information from the drug database. Based on the drug's characteristics, it selects the optimal gripping mode and executes the gripping action. Drugs in pharmacies come in various packaging forms, including cardboard boxes, plastic bottles, aluminum-plastic blister packs, and soft bags, with significant differences in size and weight. This invention designs a composite flexible gripping mechanism integrating grippers and suction cups, adaptively selecting the gripping mode based on the drug's attributes. After gripping, the drug is placed in a dispensing box, awaiting subsequent verification.

[0056] 4-1 Decision-making regarding drug retrieval models

[0057] The system pre-stores packaging attribute information for each drug in the master drug database, including packaging type. External dimensions (in , , These represent the length, width, and height of the medicine packaging (unit: mm), and its weight, respectively. and surface properties (Smooth, rough, soft, etc.). Based on these attributes, the grabbing pattern decision function outputs the optimal grabbing method:

[0058] For boxed medicines (hard surface and regular shape), parallel grippers are used for gripping; for bottled medicines (cylindrical shape), three-finger adaptive grippers are used for gripping; for bagged medicines (soft surface), vacuum suction cups are used for gripping; for blister pack medicines (flake-shaped), a combination of suction cups and trays is used for gripping.

[0059] 4-2 Adaptive control of gripping force

[0060] To prevent damage to the drug packaging or loss due to unstable gripping during the grasping process, the system implements closed-loop control of the grasping force. Let the grasping force applied by the grippers be... The weight of the medicine generates gravity. ,in For drug quality (unit: kg). The acceleration due to gravity is constant (taken as 9.8 m / s²). Safety gripping conditions require:

[0061]

[0062] The meanings of each variable are as follows: The gripping force applied by the grippers (unit: N). Drug weight (unit: N). The coefficient of friction between the grippers and the drug surface is determined based on the drug surface characteristics. The safety factor is typically set between 1.5 and 2.0 to ensure that the medication remains stably held even when the robotic arm accelerates.

[0063] At the same time, the gripping force must not exceed the tolerance limit of the drug packaging. ,Right now The system monitors the gripping force in real time through a force sensor on the end effector and uses a PID controller to adjust the output of the gripper motor, keeping the gripping force within a safe range. Inside.

[0064] 4-3 Grasping Trajectory Planning: After determining the grasping point, the system plans the motion trajectory of the robotic arm from its current position to the grasping point and then to the placement point. To ensure smooth motion, the trajectory must meet the following boundary conditions: the starting and ending points are accurately reached; the starting and ending points have zero velocities (for starting and stopping from a standstill); and the starting and ending points have zero accelerations (to avoid motion shock). These six boundary conditions require six free parameters to satisfy, therefore a fifth-order polynomial (containing six coefficients) is used for trajectory interpolation.

[0065] Let the starting pose in joint space be... The final position is Exercise time is Then the first The angle of each joint changes over time ( The curve showing the change is as follows:

[0066]

[0067] The meanings of each variable are as follows: : No. Each joint at any time Angle (unit: degrees) : No. The starting angle of each joint, , , : No. Polynomial coefficients of each joint Total time of trajectory movement (unit: seconds).

[0068] The polynomial coefficients are solved based on the boundary conditions. Let... For joints The change in angle, given the constraint that both the initial and final velocities and accelerations are zero, can be expressed analytically as the coefficient:

[0069] ,

[0070] This trajectory planning method ensures that the robotic arm's speed and acceleration are both zero at the start and end points, resulting in smooth, shock-free movement. It is particularly suitable for drug-grabbing scenarios, preventing drugs from slipping or shaking during transport. After the grasping action is completed, the robotic arm places the drug into the dispensing box and then continues executing the next storage location access task specified by the path planning module until all drugs for the current prescription have been grasped.

[0071] Step 5: Drug verification and traceability.

[0072] Once all medications for a prescription have been retrieved and placed into the dispensing box, the system verifies the dispensing results to ensure complete consistency with the prescription requirements and generates a complete dispensing record to support subsequent traceability. The verification module is independent of the retrieval process, employing a separate vision system for secondary confirmation, forming the final line of defense for dispensing quality control.

[0073] 5-1 Multiple verification mechanism,

[0074] A separate visual verification system is set up at the dispensing window to re-capture and identify images of the medications about to be dispensed. The system compares the verification results with the prescription information, verifying items including: whether the medication type is correct, whether the quantity is complete, whether the batch number is consistent, and whether the expiration date is within the safe range.

[0075] Let the set of drugs required by the prescription be . ,in For the first The code of the drug For the quantity of the drug, This represents the number of drug types included in the prescription. The set of drugs identified during verification is... ,in and The first one identified during the review Drug codes and quantities This represents the number of drug types identified. The conditions for successful verification are:

[0076]

[0077] in: A set of prescription-required medications, each element containing a medication code. and quantity , The actual collection of medicines dispensed. : No. The consistency score of the drugs (as defined in the visual recognition module) The meaning is the same, the subscript is here. (corresponding to the drug index in the prescription), this score is calculated by the visual recognition module during the capture phase based on barcode matching and expiration date verification results.

[0078] This means that the two sets are completely equal, and the expiration dates of all drugs pass verification. If verification fails, the system automatically suspends the dispensing process and issues an alarm, prompting pharmacists to intervene manually, while simultaneously recording the abnormal information in the traceability database.

[0079] 5-2 Medication dispensing record generation,

[0080] After each medication dispensing operation is completed, the system automatically generates a detailed dispensing log, recording information including: prescription number, patient information, dispensing time, storage location and source of each medication, batch number, retrieval order, robotic arm number, and visual recognition image. All records are stored in the database in timestamp order, supporting multi-dimensional queries by prescription number, medication batch number, time range, and other dimensions, meeting the traceability requirements of drug supervision.

[0081] Step 6: Intelligent storage location management and system collaboration.

[0082] The system monitors the inventory status of each drug storage location in real time. When the inventory falls below a preset threshold, it automatically generates a replenishment reminder and synchronizes the inventory information with the prescription parsing module, allowing potential drug shortages to be identified during the prescription parsing stage. Simultaneously, based on drug retrieval frequency and storage location distribution, the system periodically optimizes the storage location layout, moving frequently used drugs to locations easily accessible by the robotic arm to further improve dispensing efficiency. Storage location adjustment information is updated in real time to the main drug database, ensuring that the storage location information obtained by all modules is always accurate and consistent.

[0083] By organically combining the above technical solutions, this invention achieves automated closed-loop control of the entire process from prescription receipt to drug dispensing. The electronic prescription parsing module provides the system with accurate input for the dispensing task and transmits the storage location information to the path planning module; the visual recognition module performs precise positioning based on the storage location guidance and transmits the pose information to the flexible grasping module; the path planning module maximizes the efficiency of the dispensing operation and coordinates the access order of each storage location; the flexible grasping module performs safe and reliable grasping actions based on the drug attributes; and the verification and traceability module ensures dispensing quality and compliance management. All modules collaborate closely, with information flowing continuously throughout, together forming a complete and efficient intelligent pharmacy dispensing robot system.

[0084] Compared with the prior art, the present invention has the following advantages:

[0085] 1) Significantly improves medication dispensing efficiency. Automated gripping by a robotic arm replaces manual medication picking, and path optimization algorithms minimize idle time, reducing the average dispensing time per prescription to under 30 seconds. Compared to traditional manual dispensing, efficiency is increased by approximately 5 times, effectively handling the large volume of prescriptions during peak outpatient hours and significantly shortening patient waiting times.

[0086] 2) Achieving zero medication dispensing errors: Through dual verification of visual recognition and barcode scanning, and an independent review mechanism at each dispensing stage, the system eliminates the risk of medication errors at multiple stages. Actual operational data shows that the system has a zero medication dispensing error rate, effectively ensuring patient medication safety.

[0087] 3) Intelligent response to emergency needs: An innovative prescription priority scoring algorithm enables the system to automatically identify emergency prescriptions and special medications, dynamically adjusting the dispensing order to ensure timely response to urgent medical needs. This mechanism requires no manual intervention, improving the pharmacy's emergency response capabilities.

[0088] 4) Adaptable to various pharmaceutical packaging: The composite flexible gripping mechanism can handle pharmaceuticals in various packaging formats such as boxes, bottles, bags, and blister packs, adaptively selecting the optimal gripping method. The force control mechanism ensures reliable gripping without damaging the pharmaceuticals, making it far more applicable than traditional automated equipment.

[0089] 5) Reduce pharmacists' workload by delegating repetitive tasks such as picking and dispensing medications to robots, reducing pharmacists' workload by approximately 70%. This frees pharmacists from heavy physical labor, allowing them to dedicate more energy to professional pharmaceutical services such as prescription review and medication guidance, thereby enhancing the value of pharmaceutical services.

[0090] 6) Robust traceability management capabilities: Automatically generated dispensing logs fully record detailed information for each operation, supporting multi-dimensional queries by prescription number, batch number, time, etc., meeting the traceability compliance requirements of drug regulatory authorities. This provides reliable data support for pharmacy quality control and continuous improvement.

[0091] 7) Real-time inventory monitoring and replenishment alerts: The system tracks the drug inventory in each storage location in real time. When the inventory falls below a threshold, a replenishment alert is automatically generated to prevent dispensing interruptions due to drug shortages. Periodic storage location optimization and adjustments bring frequently used drugs closer to the robotic arm's operating area, further improving overall efficiency.

[0092] 8) Excellent system reliability and scalability, employing industrial-grade hardware and a modular software architecture, achieving a system availability rate of over 99.6%. Supports multi-robotic arm collaborative operation to expand throughput capacity, and can be flexibly configured according to pharmacy size, suitable for various application scenarios ranging from community pharmacies to large hospital pharmacies.

[0093] 9) No large-scale modifications to the existing pharmacy layout are required. The system uses independent intelligent shelving units to store medicines, making it compatible with the existing pharmacy environment. The robotic arm and vision system are flexible to install, requiring minimal alterations to the original pharmacy building structure, thus reducing the cost and time required for system implementation. Attached Figure Description

[0094] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0095] Figure 2 This is a system architecture diagram. Detailed Implementation

[0096] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.

[0097] Example 1: See Figure 1 , Figure 2 This embodiment describes in detail the specific implementation process of an intelligent pharmacy dispensing robot system based on visual positioning and precise grasping, with the outpatient pharmacy of a large tertiary hospital as the application scenario.

[0098] 1 System Architecture and Hardware Configuration

[0099] The intelligent drug dispensing robot system is constructed with an overall architecture consisting of five main parts: a central control server, a robotic arm execution unit, a vision recognition unit, a drug storage unit, and a human-machine interaction terminal. These units are interconnected via industrial Ethernet to form a collaborative whole.

[0100] Hardware Configuration List: Central Control Server: Industrial-grade server, equipped with an Intel Xeon processor, 64GB of RAM, and an NVIDIA RTX 4090 GPU for visual inference acceleration; Robotic Arm: Six-axis collaborative robotic arm, load capacity 5kg, repeatability ±0.02mm, working radius 850mm; End Effector: Composite flexible gripping mechanism, integrating a two-finger parallel gripper (opening range 0-85mm), a three-finger adaptive gripper, and a vacuum suction cup (suction force 80N); Industrial Camera: 2-megapixel color camera, global shutter, with an 8mm fixed-focus lens; Depth Sensor: Structured light depth camera, measurement range 0.2-1.5m, depth accuracy ±1mm; Medicine Shelf Storage System: 48 intelligent shelf units, each with 6 layers and 12 columns, totaling 72 standard storage locations, for a total of 3456 storage locations; Conveyor System: Variable frequency speed-controlled conveyor belt for transporting prepared medicines to the dispensing window.

[0101] The system uses industrial Ethernet to connect all units, ensuring millisecond-level communication latency. The robotic arm controller and the central server exchange position commands and status feedback in real time via the EtherCAT protocol. The central control server runs the software programs for each functional module, responsible for data transfer and process scheduling between modules.

[0102] 2. Implementation of electronic prescription analysis and task scheduling

[0103] 2.1 HIS System Integration and Prescription Parsing

[0104] An HL7 FHIR standard interface is established with the hospital's HIS system to receive outpatient electronic prescriptions in real time. Each prescription includes fields such as patient ID, department, prescribing physician, and medication details. The prescription parsing module extracts medication entry information and queries the main medication database to obtain the location coordinates of each medication.

[0105] Taking a typical prescription as an example, the prescription Includes 3 types of medicine:

[0106]

[0107] The system queries the main drug database to obtain the storage location coordinates of each drug: - Amoxicillin capsules: storage location coordinates Corresponding to the 3rd shelf, 4th column of the 2nd group of shelves - Ibuprofen sustained-release capsules: warehouse location coordinates -Vitamin C tablets: Storage location coordinates

[0108] This storage location information will be passed to the path planning module to calculate the optimal fetching order.

[0109] 2.2 Priority Calculation

[0110] Calculate prescription priority using the aforementioned priority scoring formula. Assume the prescription... For general outpatient prescriptions ( (Waiting time) Seconds, system setting Seconds, the prescription does not contain any special medications. The weighting coefficients are set to... , , .

[0111] Substitute into the priority scoring formula:

[0112]

[0113] If there is an emergency prescription at the same time ( , (seconds), its priority score is:

[0114]

[0115] because The system prioritizes prescription allocation. The prescription parsing module pushes the priority-sorted task list and corresponding warehouse location information to the path planning module, triggering the subsequent medication dispensing process.

[0116] 3. Implementation of visual recognition and positioning.

[0117] 3.1 Drug image acquisition and detection,

[0118] Once the path planning module determines that a specific storage location should be accessed, the robotic arm moves to the vicinity of that location. The end-effector camera captures images of the medicine shelf; the image resolution is [resolution missing]. Pixels. A pre-trained YOLOv8 object detection network was used for drug identification. The network was fine-tuned and trained on a self-built drug image dataset (containing 1200 common drugs, with about 500 images of each).

[0119] Taking amoxicillin capsules as an example, the detection network outputs the bounding box:

[0120]

[0121] Indicates the location of the drug in the image coordinates arrive Within the rectangular area, the category is amoxicillin capsules, with a confidence level of 0.96.

[0122] 3.2 Barcode Decoding and Expiry Date Verification

[0123] Within the detected bounding box area, locate the barcode position and perform decoding to calculate the drug consistency score. Obtain the drug information vector:

[0124]

[0125] Set the current date Safety shelf life threshold Day. Verification conditions: Target drug code matches and expiration date. Therefore, the consistency score is:

[0126] ,

[0127] The drug passed verification and was confirmed as a valid target for grasping. After successful verification, the system continued to perform 3D pose calculations to provide the flexible grasping module with a precise target position.

[0128] 3.3 Three-dimensional pose calculation,

[0129] The depth camera measured the three-dimensional coordinates of the center point of the drug in the camera coordinate system. Meters. Through a pre-calibrated hand-eye transformation matrix. Transform it to the end effector coordinate system:

[0130]

[0131] Obtain the target position in the end effector coordinate system The pose information is transmitted to the flexible grasping control module via the data bus as the target point for grasping trajectory planning.

[0132] 4. Path optimization implementation,

[0133] 4.1 Multi-drug pathway planning,

[0134] A genetic algorithm was used to optimize the pathways to the storage locations of the three types of drugs in the current batch. Storage location set. ,in This is the location of the medication dispensing window. , , These correspond to the storage locations for amoxicillin capsules, ibuprofen sustained-release capsules, and vitamin C tablets, respectively.

[0135] Measure the motion time between each position (Unit: seconds):

[0136]

[0137] Based on the drug packaging properties, the flexible grasping module estimates the grasping time for each storage location as follows: Second.

[0138] According to the formula for total medication dispensing time Consider the path (Right now Total time:

[0139]

[0140] Consider the path (Right now Total time:

[0141]

[0142] Consider the path (Right now Total time:

[0143]

[0144] The genetic algorithm outputs the optimal path after a population size of 50 and 100 generations. or Its fitness value is:

[0145] ,

[0146] After path planning is completed, the robotic arm visits each storage location sequentially according to the optimal path sequence. At each storage location, the vision recognition module first performs precise positioning, and after confirming the target, the flexible grasping module performs the grasping action.

[0147] 5. Implementation of flexible grasping control.

[0148] 5.1 Selecting the capture mode

[0149] After the visual recognition module completes the drug location and transmits the pose information to the flexible grasping module, the system queries the drug master database to obtain packaging attributes. Amoxicillin capsules are boxed drugs; packaging attribute: type = Cardboard box, size mm (representing length, width, and height respectively), weight kg, surface properties = Smooth. Since the medicine is a boxed medicine with a hard surface and a regular shape, the system selects a two-finger parallel gripper gripping mode.

[0150] 5.2 Grasping Force Control

[0151] Based on the safety conditions of gripping force Calculations are performed. The gravitational force generated by the weight of the medicine. N. Let the coefficient of friction between the grippers and the cardboard box surface be N. Safety factor The minimum gripping force is:

[0152] ,

[0153] Maximum load-bearing capacity of cardboard packaging N, therefore the gripping force is controlled at Within the range of N, the system's PID controller stabilizes the gripping force at approximately 5 N, balancing stability and safety, thus meeting the requirements. The safety conditions are far below the limits that packaging can withstand.

[0154] 5.3 Trajectory Execution

[0155] A smooth motion curve is generated using a fifth-order polynomial trajectory planning method. The robotic arm starts from its current pose. Movement to the grasping point position Let the exercise time be... Seconds. With joint 1 ( For example, starting angle Termination angle angular change .

[0156] The formula for solving the coefficients of a fifth-degree polynomial. , , Calculate the polynomial coefficients:

[0157] ,

[0158] ,

[0159] ,

[0160] This yields the smooth trajectory of joint 1:

[0161]

[0162] This trajectory can be verified in hour ,exist seconds The starting and ending velocities and accelerations are both zero. The robotic arm moves smoothly along this trajectory without any impact or vibration at the end, ensuring a stable and reliable grasping process. After grasping, the robotic arm places the medicine into the dispensing box and then proceeds to the next storage location to continue the grasping task according to the instructions from the path planning module.

[0163] 6. Implementation of drug verification and traceability.

[0164] 6.1 Verification and review.

[0165] Once all prescription medications have been collected and placed into the dispensing box, the conveyor belt transports the box to the verification area at the dispensing window. A separate verification camera captures and identifies the medications, resulting in the actual dispensing set.

[0166]

[0167] Prescription requirements The comparison and verification results showed that the two sets were completely equal. The consistency scores of all drugs were also checked. The expiration date verification was passed, meeting the conditions for drug dispensing. After successful verification, the system indicates that the medication can be dispensed to the patient.

[0168] 6.2 Log generation,

[0169] The system automatically generates a medication dispensing record. Key fields include: - Prescription Number: RX20260202001234 - Dispensing Time: 2026-02-02 09:15:32 - Total Time: 28.7 seconds - Medication Details: Amoxicillin Capsules (Location 2-3-4, Batch No. 20250801) ×2, Ibuprofen Extended-Release Capsules (Location 5-2-8, Batch No. 20251102) ×1, Vitamin C Tablets (Location 3-4-1, Batch No. 20251215) ×1 - Operating Robotic Arm: ARM-01 - Validation Status: Passed

[0170] Records are stored in a traceability database, supporting conditional queries and quality analysis statistics. After medication is dispensed, relevant information is fed back to the prescription parsing module, updating the prescription status to "completed," thus ending the entire medication dispensing process loop.

[0171] 7. System Performance Test Results

[0172] After 30 days of actual deployment and operation in the outpatient pharmacy, the system performance statistics are as follows:

[0173] Average daily prescription volume: 3200 prescriptions.

[0174] Average prescription dispensing time: 32 seconds (including 3.5 medications).

[0175] Medication dispensing error rate: 0 (zero errors)

[0176] System availability: 99.6%

[0177] Pharmacist workload reduced by approximately 70%, medication efficiency increased by about 5 times compared to traditional manual dispensing, medication errors completely eliminated, and the patient's waiting experience for medication pickup significantly improved. Through the detailed description of this embodiment, those skilled in the art can clearly understand the technical solution of this invention. The data flow and collaborative control relationships between the modules are readily apparent, enabling them to implement this invention in actual pharmacy scenarios based on the described system configuration and method flow, achieving efficient and accurate intelligent medication dispensing operations.

[0178] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A smart pharmacy dispensing method based on visual positioning and precise grasping, characterized in that, The method includes the following steps: Step 1: Electronic prescription parsing and dispensing task generation. Step 2: Visual recognition and drug location. Step 3: Optimize the medication dispensing route. Step 4: Flexible grasping control Step 5: Drug verification and traceability. Step 6: Intelligent storage location management and system collaboration.

2. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 1, characterized in that, Step 1, Electronic Prescription Parsing and Dispensing Task Generation, details are as follows: Establish a real-time data interface with the Hospital Information System (HIS) to receive electronic prescriptions and perform structured parsing, providing task input for subsequent visual localization and path planning. Let the received prescription set be... ,in Indicates the first Zhang's prescription, This represents the total number of prescriptions awaiting dispensing, and each prescription... It contains several drug entries, represented as ,in Indicates the first Zhang Fangzhong The identification code of the drug This indicates the quantity of the medicine dispensed. This refers to the number of different types of medications included in the prescription. The parsing engine first performs semantic validation on the prescription, checking the validity of the drug code, the rationality of the dosage, and any incompatibilities between drugs. By querying the main drug database, it maps each drug entry to specific location information. Let's assume the drug... The coordinates of the storage location in the warehousing system are ,in , , These represent the three-dimensional coordinates of the storage location within the warehouse space. This location coordinate information will be passed as input to the path planning module to calculate the optimal fetching order. The prescription processing order is determined by comprehensively calculating the prescription's dispensing priority based on the prescription's receipt time, patient type, and drug attributes. The priority rating is defined as follows: The meanings of each variable are as follows: :prescription Priority rating, with a value range of A larger value indicates a higher priority. The urgency coefficient of a prescription is set as follows: 0 for a regular prescription, 0.5 for an emergency prescription, and 1.0 for a critical prescription. :prescription Waiting time (in seconds) The maximum allowed waiting time set by the system (in seconds). The value is 1.0 if the prescription contains any specially controlled substances such as cold chain drugs or narcotic drugs, and 0 otherwise. , , Weighting coefficients, satisfying , Priority rating Prescriptions with higher priority will be given priority in dispensing. After the parsing engine completes prescription parsing and priority calculation, it generates a list of dispensing tasks and pushes it to the path planning module. At the same time, it transmits the warehouse location coordinates of each drug to the visual recognition module for subsequent accurate positioning and retrieval.

3. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 2, characterized in that, Step 2, visual recognition and drug location, is detailed below. 2-1 Extraction of drug appearance features, For the collected drug images ,in and These represent the height and width in pixels, respectively. The system uses an improved convolutional neural network to extract visual features of the drugs. The network consists of a feature extraction backbone and a multi-scale detection head, and outputs the bounding box coordinates, category label, and confidence score of the drugs. Let the set of detected drug bounding boxes be... Each bounding box Including top left corner coordinates lower right corner coordinates Drug categories and confidence score , This represents the total number of detected drugs. The system matches the identification results with the target drug codes passed in from the prescription parsing module, filtering out candidate drugs relevant to the current task. 2-2 Barcode Recognition and Information Matching Within the bounding box area, the drug barcode region is further located, and a barcode decoding algorithm is used to read key information such as the drug batch number, production date, and expiration date. Let the decoded drug information vector be denoted as . ,in: Drug barcode number, :batch number, Production date Validity period The identification results are matched and verified against the main drug database, and an information consistency score is calculated: The meanings of each variable are as follows: : No. The consistency score of each candidate drug, with a value of 0 or 1. The target drug code for the current task is input from the prescription parsing module. Current system date The safety shelf life threshold is set to 30 days. Only consistency score Only drugs that meet the criteria are considered valid targets for retrieval, thus preventing the risk of taking the wrong drug or obtaining a near-expiry drug at the source. The verified drug information and location data will then be passed to the next step of 3D pose estimation. 2-3 Three-dimensional pose estimation, Combining point cloud data acquired by the depth sensor, the system calculates the precise 3D pose of the target drug, providing the gripping point input for the flexible gripping module. Let the 3D coordinates of the drug's center point in the depth camera coordinate system be... Hand-eye calibration matrix Transform it to the coordinate system of the robotic arm's end effector: in: The homogeneous transformation matrix from the camera coordinate system to the end effector coordinate system is obtained in advance through hand-eye calibration. : Target position in camera coordinate system (homogeneous coordinate form). : Target position in the end effector coordinate system The target position is transformed into a pose representation in the coordinate system of the robot arm base through forward kinematics calculation, and this pose information is transmitted to the flexible grasping control module as the target point for grasping trajectory planning.

4. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 3, characterized in that, Step 3, the optimization of the medication dispensing route is as follows: 3-1 Problem Modeling Let the set of drug storage locations to be captured in the current batch be . ,in This indicates the starting position of the robotic arm (dispensing window). to Indicates the location of each medicine storage unit. Define any two locations as the total number of drug storage locations. and The time of movement between them is This time is determined by both the kinematic characteristics of the robotic arm and the spatial distance. Medication dispensing route This indicates the access order of a storage location. ,in And they are all different. The total dispensing time for each route is defined as: ， The meanings of each variable are as follows: :path Total medication preparation time (unit: seconds). From storage location Move to storage location Exercise time, set Indicates the starting position. In storage location The time required to perform the grasping action is related to the type of drug packaging and the grasping strategy, and is estimated by the flexible grasping module based on the drug attributes. 3-2 Optimization of Genetic Algorithm A genetic algorithm is used to search for the optimal path. This algorithm simulates the evolutionary process in nature, gradually approaching the global optimum through population iteration. The algorithm includes four main operations: initialization, selection, crossover, and mutation. Initialization: Generates a collection of... The initial population consists of individuals, each representing a feasible path. 70% of the individuals are generated using a greedy strategy (selecting the nearest unvisited storage location each time), while 30% are generated randomly to ensure population diversity. Fitness calculation: The fitness function is defined as the reciprocal of the drug preparation time, such that paths with shorter drug preparation times have higher fitness. ， in: :path The fitness value indicates that the better the path is. A small constant to prevent division by zero, set to 0.

001. Selection operation: A roulette wheel selection strategy is used, where the probability of each individual being selected is proportional to its fitness. Let the i-th individual in the population be the first selected individual. Individual (path) The probability of selection is ,in Individuals with high fitness levels are more likely to be carried into the next generation. Crossover operation: The Order Crossover (OX) operator is used to generate offspring. The specific steps are as follows: a segment of the parent path is randomly selected and directly copied to the same position in the offspring. The remaining positions are filled with the missing storage location numbers in the order of another parent. This operator ensures that the generated offspring is still a valid path (each storage location is visited exactly once). Mutation operation: with probability (Take a value of 0.1-0.2) Mutate the individual by randomly selecting two positions in the path and swapping their storage location numbers, introducing random perturbation to escape local optima. The algorithm iterates until it reaches a preset number of generations or converges after several consecutive generations when the fitness has not significantly improved. It then outputs the best individual from each generation as the optimal path. , 3-3 Dynamic path adjustment employs an incremental path update strategy: For newly added drug storage locations, the time cost of inserting them into each of the remaining locations on the current path is evaluated, and the insertion point with the lowest cost is selected; for out-of-stock locations, they are directly deleted from the path, and the replenishment information is recorded, while the prescription parsing module is notified to update the prescription status. This dynamic adjustment mechanism enables the system to flexibly respond to real-time changes and maintain high dispensing efficiency; After the path planning is completed, the system schedules the robotic arm to visit each storage location in sequence according to the optimal path sequence. At each storage location, the vision recognition module first performs precise positioning of the medicine. After confirming the target pose, the flexible grasping module performs the grasping action.

5. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 2, characterized in that, Step 4, Flexible Grabbing Control, as detailed below: 4-1 Decision-making regarding drug retrieval models Pre-store packaging attribute information for each drug in the master drug database, including packaging type. External dimensions (in , , These represent the length, width, and height of the medicine packaging (unit: mm), and its weight, respectively. and surface properties Based on these attributes, the crawling pattern decision function outputs the optimal crawling method: For boxed medicines, parallel grippers are used for grasping; for bottled medicines, three-finger adaptive grippers are used for enveloping grasping; for bagged medicines, vacuum suction cups are used for adsorption grasping; and for blister pack medicines, a combination of suction cups and trays is used for grasping. 4-2 Adaptive control of gripping force Let the gripping force applied by the gripper be... The weight of the medicine generates gravity. ,in For drug quality (unit: kg). Given the gravitational acceleration constant (taken as 9.8 m / s²), the safety gripping conditions are as follows: ， The meanings of each variable are as follows: The gripping force applied by the grippers (unit: N). Drug gravity (unit: N) The coefficient of friction between the grippers and the drug surface is determined based on the drug surface characteristics. The safety factor is set between 1.5 and 2.0 to ensure that the medication remains stably held even when the robotic arm accelerates. At the same time, the gripping force must not exceed the tolerance limit of the drug packaging. ,Right now The system monitors the gripping force in real time through a force sensor on the end effector and uses a PID controller to adjust the output of the gripper motor, keeping the gripping force within a safe range. Inside, 4-3 Trajectory planning for capture After determining the gripping point, the system plans the motion trajectory of the robotic arm from its current position to the gripping point and then to the placement point. To ensure smooth motion, the trajectory must meet the following boundary conditions: the starting and ending points are accurately reached; the starting and ending points have zero velocities (for starting and stopping from a standstill); and the starting and ending points have zero accelerations (to avoid motion impact). These six boundary conditions require six free parameters to satisfy, therefore a fifth-order polynomial (containing six coefficients) is used for trajectory interpolation. Let the starting pose in joint space be... The final position is Exercise time is Then the first The angle of each joint changes over time ( The curve showing the change is as follows: The meanings of each variable are as follows: : No. Each joint at any time Angle (unit: degrees) : No. The starting angle of each joint, , , : No. Polynomial coefficients of each joint Total time of trajectory movement (unit: seconds). The polynomial coefficients are solved based on the boundary conditions. Let... For joints The change in angle, given the constraint that both the initial and final velocities and accelerations are zero, can be expressed analytically as the coefficient: After the grasping action is completed, the robotic arm places the medicine into the dispensing box, and then continues to execute the next storage location access task specified by the path planning module until all medicines for the current prescription have been grasped.

6. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 2, characterized in that, Step 5, Drug Verification and Traceability, details are as follows: 5-1 Multiple verification mechanism, A separate visual verification system is set up at the dispensing window to re-capture and recognize images of the medications about to be dispensed. The system compares the verification results with the prescription information, verifying items including: whether the medication type is correct, whether the quantity is complete, whether the batch number is consistent, and whether the expiration date is within the safe range. Let the set of drugs required by the prescription be . ,in For the first The code of the drug For the quantity of the drug, The number of drug types included in the prescription; the set of drugs to be reviewed and identified is... ,in and The first one identified during the review Drug codes and quantities The verification criteria for the number of identified drug types are as follows: ， in: A set of prescription-required medications, each element containing a medication code. and quantity , The actual collection of medicines dispensed. : No. The consistency score of the drug (as defined in the visual recognition module) The meaning is the same, the subscript is here. (corresponding to the drug index in the prescription), this score is calculated by the visual recognition module during the capture phase based on barcode matching and expiration date verification results. This means the two sets are completely equal, and the expiration date verification of all drugs passes. If verification fails, the system automatically pauses the dispensing process and issues an alarm, prompting pharmacists to intervene manually. Simultaneously, the abnormal information is recorded in the traceability database. 5-2 Medication dispensing record generation, After each medication dispensing operation is completed, the system automatically generates a detailed medication dispensing log, which includes: prescription number, patient information, dispensing time, storage location and source of each drug, batch number, grabbing order, robotic arm number, and visual recognition image. All records are stored in the database in the order of timestamps, and can be queried by prescription number, drug batch number, time range, and other dimensions to meet the traceability requirements of drug supervision.

7. The intelligent pharmacy dispensing method based on visual positioning and precise grasping according to claim 2, characterized in that, Step 6: Intelligent warehouse location management and system collaboration. The system monitors the inventory status of each drug warehouse location in real time. When the inventory is lower than the preset threshold, a replenishment reminder is automatically generated, and the inventory information is synchronized to the prescription analysis module so that possible drug shortages can be identified during the prescription analysis stage. At the same time, the warehouse location layout is periodically optimized according to the drug retrieval frequency and warehouse location distribution, and high-frequency drugs are adjusted to positions that are easily and quickly reached by the robotic arm to further improve dispensing efficiency. The warehouse location adjustment information is updated to the main drug database in real time to ensure that the warehouse location information obtained by each module is always accurate and consistent.

8. A smart pharmacy dispensing robot system based on visual positioning and precise grasping, characterized in that, The system is used to implement the method described in any one of claims 1-7. The system includes an electronic prescription parsing engine module, a visual recognition and positioning module, a dispensing path planning module, a flexible grasping control module, and a drug verification and traceability module. It realizes full-process automation of pharmacy dispensing operations. Starting from receiving the electronic prescription, the system completes the accurate dispensing of drugs through drug storage location positioning, optimal path planning, precise grasping execution, and multiple verification checks. At the same time, it generates complete operation records to support traceability management.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intelligent pharmacy dispensing method based on visual positioning and precise grasping as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instruction is executed by the processor, it implements the intelligent pharmacy dispensing method based on visual positioning and precise grasping as described in any one of claims 1-7.

Images