Drug distribution method and device based on artificial intelligence, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to a medical health business system platform, and discloses an artificial intelligence-based drug distribution method, device, equipment and medium, and the method comprises the steps: obtaining multi-modal data of a medical scene, carrying out the pre-training of an initial VLA model based on the multi-modal data, and generating an initial drug distribution model; training the initial drug distribution model based on the drug distribution sample data to generate a drug distribution model; inputting the medicine distribution instruction into a medicine distribution model, and generating medicine distribution action data; and obtaining matching information of the to-be-dispensed medicine and the patient based on the medicine dispensing model, and if the to-be-dispensed medicine is successfully matched with the patient, controlling the robot to execute a corresponding dispensing action based on the medicine dispensing action data. According to the invention, through pre-training of a medical drug distribution scene, high recognition rate and stability can be maintained under different hospital layouts, drug shelf arrangements and illumination conditions; the medical care natural language instruction can be flexibly understood, and the recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment and medium for dispensing medicines based on artificial intelligence. Background Technology

[0002] Currently, drug distribution in medical settings largely relies on robotic arms or conveyor systems for fixed-path operations. These systems are typically rule-based or visual detection combined with path planning, lacking intelligent semantic understanding and environmental adaptability. In multi-patient, multi-drug environments, traditional systems struggle to accurately identify the correspondence between drug labels and patient information; they cannot flexibly handle unstructured scenarios, such as changes in shelf placement, obstructions, or unstable lighting; and they lack natural language interaction interfaces, making dynamic task adjustments based on medical staff instructions impossible.

[0003] Existing vision-language-action models (such as OpenVLA and PaLM-E) possess multimodal understanding capabilities, but they are primarily geared towards general tasks or industrial robotic arm operations. Their training data lacks a medical context (such as drug recognition, medical procedures, and operational guidelines), resulting in poor transfer performance in medical scenarios. The action token generation logic is not optimized for the "safe, accurate, and sequential execution" requirements of medical tasks, posing a potential risk of misoperation. Therefore, existing models exhibit poor distribution accuracy and low fault tolerance when dispensing drugs in medical scenarios. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the present invention provides a drug distribution method, apparatus, equipment and medium based on artificial intelligence, which aims to solve the problems of poor distribution accuracy and low fault tolerance of the existing model when distributing drugs in medical scenarios.

[0005] The technical solution of the present invention is as follows: The first embodiment of the present invention provides a drug distribution method based on artificial intelligence, the method comprising: Acquire multimodal data from a medical scenario, pre-train an initial VLA model based on the multimodal data, and generate an initial drug distribution model; Obtain sample data of drug distribution, and train the initial drug distribution model based on the sample data of drug distribution to generate a drug distribution model; When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model; Based on the drug distribution model, matching information between the drug to be distributed and the patient is obtained. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0006] Another embodiment of the present invention provides an artificial intelligence-based drug dispensing device, the device comprising: The model building module is used to acquire multimodal data of medical scenarios, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model. The model training module is used to acquire drug distribution sample data, train the initial drug distribution model based on the drug distribution sample data, and generate a drug distribution model. The data processing module is used to input the drug distribution instruction into the drug distribution model when a drug distribution instruction is detected, and to generate drug distribution action data based on the drug distribution model. The action execution module is used to obtain matching information between the drug to be distributed and the patient based on the drug distribution model. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0007] Another embodiment of the present invention provides a computer device, the computer device including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the steps of the artificial intelligence-based drug distribution method described above.

[0008] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the artificial intelligence-based drug dispensing method described above.

[0009] Beneficial Effects: The artificial intelligence-based drug distribution method, apparatus, equipment, and medium of this invention include: acquiring multimodal data of a medical scenario; pre-training an initial VLA model based on the multimodal data to generate an initial drug distribution model; acquiring drug distribution sample data; training the initial drug distribution model based on the drug distribution sample data to generate a drug distribution model; when a drug distribution instruction is detected, inputting the drug distribution instruction into the drug distribution model and generating drug distribution action data based on the drug distribution model; acquiring matching information between the drug to be distributed and the patient based on the drug distribution model; if the drug to be distributed is successfully matched with the patient, controlling a robot to perform the corresponding distribution action based on the drug distribution action data. This invention, through pre-training on medical drug distribution scenarios, can maintain high recognition rate and stability under different hospital layouts, drug shelf arrangements, and lighting conditions; it can flexibly understand medical natural language instructions, achieving end-to-end closed-loop control of "instruction-vision-action". Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating the application environment of an embodiment of the artificial intelligence-based drug distribution method of the present invention; Figure 2 This is a flowchart of a preferred embodiment of an artificial intelligence-based drug distribution method according to the present invention; Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of an artificial intelligence-based drug dispensing device of the present invention; Figure 4 This is a schematic diagram of a preferred embodiment of a computer device according to the present invention; Figure 5 This is another structural schematic diagram of a preferred embodiment of a computer device according to the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0013] The embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0014] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Here, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0015] The method provided in this application can be applied to artificial intelligence (AI) scenarios. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to have the functions of perception, reasoning, and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0016] The artificial intelligence-based drug distribution method provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The client accesses the server's network or business platform, and the server can obtain multimodal data of the medical scenario. Based on the multimodal data, it pre-trains an initial VLA model to generate an initial drug distribution model; it obtains drug distribution sample data, and trains the initial drug distribution model based on the sample data to generate a drug distribution model; when a drug distribution instruction is detected, it inputs the instruction into the model and generates drug distribution action data; based on the model, it obtains matching information between the drug to be distributed and the patient. If the drug is successfully matched with the patient, it controls the robot to perform the corresponding distribution action based on the action data. In this invention, by pre-training on the medical drug distribution scenario, high recognition rate and stability can be maintained under different hospital layouts, shelf arrangements, and lighting conditions; it can flexibly understand medical natural language instructions, improving recognition accuracy. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a separate server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0017] To address the above problems, embodiments of the present invention provide an artificial intelligence-based drug distribution method. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating a preferred embodiment of an artificial intelligence-based drug distribution method according to the present invention. Figure 2 As shown, it includes: Step S100: Obtain multimodal data of the medical scenario, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model.

[0018] This invention is applied to the drug distribution scenario using medical robots in a medical setting. By constructing a VLA model framework specifically for drug distribution tasks, vision, language, and action are modeled in a unified manner. The model is pre-trained on multimodal data such as large-scale medical scene images, drug packaging, drug shelf layout, and nurse operation videos to generate an initial drug distribution model and obtain prior knowledge for medical distribution tasks.

[0019] Cross-modal alignment is the core of building multimodal VLA (Vision-Language-Action) models. Its goal is to achieve deep fusion and interaction of visual, linguistic, and action information through a unified semantic representation space. Cross-modal alignment is used to achieve semantic consistency: ensuring that different modalities (such as the visual image of a "red pillbox," the linguistic description of "aspirin," and the robotic arm trajectory of a "grasping action") are mapped to similar semantic representations in a shared space. Joint reasoning capability: supporting complementary reasoning between modalities (such as correcting visual misrecognition through linguistic instructions or optimizing linguistic understanding through action feedback). Generalization: maintaining alignment performance even in unseen combinations of modalities (such as new drug packaging or complex action instructions).

[0020] A Medical Knowledge Graph (MKG) is a semantic network built around medical knowledge, integrating multi-source heterogeneous data (such as medical literature, electronic medical records, clinical guidelines, and drug databases) in a structured manner. Its core objective is to provide interpretable and traceable knowledge support for scenarios such as medical decision-making, intelligent diagnosis, and drug development. The components of a medical knowledge graph include entities, relations, and attributes. For example, entities include the following: Diseases: such as "diabetes" and "hypertension," containing attributes such as ICD codes, symptoms, and complications. Drugs: such as "aspirin" and "insulin," containing attributes such as ingredients, dosage, indications, and contraindications. Tests and examinations: such as "blood glucose test" and "CT scan," containing attributes such as normal range and testing methods. Symptoms: such as "headache" and "fever," containing attributes such as severity and duration. Genes: such as "BRCA1 gene," containing attributes such as mutation type and associated diseases. Medical institutions: such as "Peking Union Medical College Hospital," containing attributes such as departments and specialists.

[0021] Relationships include the following: Treatment relationship: Disease - Treatment -> Drug (e.g., "Diabetes - Treatment -> Metformin"). Causal relationship: Symptom - Leads to -> Disease (e.g., "Polydipsia and Polyuria - Leads to -> Diabetes"). Interaction relationship: Drug - Interaction -> Drug (e.g., "Aspirin - Interaction -> Warfarin"). Inclusion relationship: Disease - Inclusion -> Subtype (e.g., "Lung Cancer - Inclusion -> Non-Small Cell Lung Cancer"). Detection relationship: Disease - Detection -> Examination and Testing (e.g., "Hypertension - Detection -> Blood Pressure Measurement").

[0022] The attributes include the following: Disease attributes: ICD code, incidence rate, mortality rate, high-risk groups. Drug attributes: chemical formula, half-life, manufacturer, price. Symptom attributes: location of pain, frequency of attacks, and methods of relief.

[0023] The medical atlas construction process is as follows: Data preprocessing. Removing duplicate and erroneous data. Standardization requires unified entity naming. Knowledge extraction. Entity recognition uses models such as BiLSTM-CRF and BERT to identify medical entities. Relationship extraction extracts relationships between entities through dependency resolution and attention mechanisms. Attribute extraction extracts entity attributes from text (e.g., "Aspirin - Dosage -> 100mg / day"). Knowledge fusion. First, entity alignment is performed: resolving homonyms (e.g., "Apple" in medicine may refer to "apple" or "Apple Inc.") or synonyms (e.g., "myocardial infarction" and "heart attack"); then, relationship fusion is performed: merging conflicting relationships in multi-source data (e.g., descriptions of "aspirin and warfarin interaction" in different literature). Knowledge storage can utilize graph databases: using Neo4j or JanusGraph to store graph structures, supporting efficient queries (e.g., "find all drugs for treating diabetes"). You can also use RDF triples, which are stored in the W3C standard format and support semantic reasoning (such as "If A treats B, and B causes C, then A may alleviate C").

[0024] Step S100, namely acquiring multimodal data of the medical scenario, pre-training the initial VLA model based on the multimodal data, and generating an initial drug distribution model, includes: Step S101: Construct a VLA model, wherein the VLA model forms a shared semantic representation space in the visual, linguistic, and action dimensions; Step S102: Add a task instruction decoding module to the VLA model to generate an initial VLA model; Step S103: Obtain multimodal data of the medical scenario, and pre-train the initial VLA model based on the multimodal data to generate an initial drug distribution model.

[0025] In constructing the VLA model, a hybrid expert Transformer architecture is adopted, integrating three major modules: vision, language, and action. Efficient inference is achieved through multimodal feature alignment and dynamic computation paths. For example, the model specifically includes: a vision expert module, a language expert module, an action expert module, and a multimodal fusion layer. The vision expert module's visual encoder uses a combination of DINO-v2 and SigLIP to improve the recognition capabilities of drug packaging, labels, and the placement of medicine boxes. Key feature extraction in the vision expert module involves locating the drug's position using image segmentation technology, recognizing label text (such as drug name, dosage, and expiration date), and detecting the open / closed status of the medicine box (such as "opened" or "unopened"). A dynamic attention mechanism is implemented: for drug distribution scenarios, a drug-area focused attention mechanism is designed, prioritizing visual areas relevant to the task (such as the target medicine box and patient information labels).

[0026] The language encoder in the language expert module uses the Qwen2.5-3B lightweight large language model, supporting bilingual (Chinese and English) instruction understanding. Instruction parsing involves resolving natural language instructions (such as "Distribute aspirin enteric-coated tablets (100mg × 30 tablets) to the patient in bed 201") into a structured task: Drug name: aspirin enteric-coated tablets; Specification: 100mg × 30 tablets; Target patient: bed 201; Contextual reasoning: Safety verification is performed by combining the patient's historical medication records (such as allergy history and contraindications).

[0027] The motion generator in the motion expert module is based on the DiT (Diffusion Transformer) model, generating continuous motion sequences (such as the robotic arm's grasping angle, force, and movement trajectory). The motion optimization method introduces a flow matching mechanism, refining motion accuracy by simulating physical interactions (such as force feedback when a medicine box is opened) to prevent medication damage. Safety constraints include an embedded collision detection algorithm to ensure the robotic arm does not collide with the patient or other obstacles during dispensing.

[0028] Feature alignment in the multimodal fusion layer maps visual features (drug location), linguistic features (task instructions), and state features (current pose of the robotic arm) to a unified semantic space through a cross-attention mechanism. Dynamic routing dynamically adjusts the computation path based on task complexity (e.g., simple tasks activate only the visual-motor pathway, while complex tasks enable full-modal reasoning).

[0029] A cross-modal alignment mechanism is introduced to enable the model to form a shared semantic representation space across visual, linguistic, and action dimensions. Combined with a medical knowledge graph, this helps the model understand drug categories, usage, and distribution logic, achieving a leap from "visual recognition" to "semantic understanding."

[0030] On the other hand, the task instruction decoding module is the core hub in the VLA model that connects natural language instructions with physical action execution. It needs to achieve accurate mapping from "medical instructions" to "action token sequence" through three stages: semantic parsing, knowledge association, and action planning.

[0031] Semantic parsing refers to the process of converting natural language into a structured task representation. This involves instruction preprocessing and standardization, as well as intent recognition and contextual understanding. Instruction preprocessing and standardization require noise filtering. The BERT-WWM model is used to identify and correct grammatical errors and colloquial expressions in the instructions. Entity extraction: Key entities (drug name, dosage, patient ID, time limit) are extracted using a BiLSTM-CRF model. For example, the input instruction is: "Please distribute enteric-coated aspirin tablets (100mg × 30 tablets) to the patient in bed 201, and note the expiration date." The output structured representation is: {Drug: Enteric-coated aspirin tablets, Specification: 100mg × 30 tablets, Patient: Bed 201, Constraint: Expiration date check}.

[0032] Intent recognition and context understanding involve two steps. First, multi-turn dialogue management is performed: if the instruction is ambiguous (e.g., "Give me a pill"), the model triggers a clarification submodule to obtain missing information through a preset question template. Second, context association is performed: combining historical dialogue records (e.g., the patient's previous medication was ibuprofen) and hospital HIS system data (e.g., the patient's allergy history) to supplement implicit constraints (e.g., "Avoid dispensing aspirin-containing medications to patients with a history of gastric ulcers").

[0033] Knowledge association involves the injection and verification of knowledge in the medical field. Drug knowledge graph integration includes knowledge graph embedding and patient identity and status verification. Knowledge graph embedding encodes drug attributes (indications, contraindications, interactions) from databases such as UMLS and DrugBank into graph embedding vectors, which are then injected into the decoding module through an attention mechanism. Example: When an instruction involves "warfarin," the model automatically associates it with the interaction risk with "aspirin," triggering a safety check. Rule engine: Built-in hard rules (such as "expired drugs are prohibited from distribution" and "dosage exceedance alarm") ensure that action sequences comply with medical regulations through constraint optimization. Patient identity and status verification includes multimodal identity matching: combining patient facial images (visual), ID card number / medical record number (verbal), and HIS system data for triple identity verification. Status awareness: Detecting the patient's current status (e.g., bedridden, immobile) through a visual module and dynamically adjusting action strategies (e.g., prioritizing the distribution of easily accessible drugs).

[0034] Motion planning is the process of transforming a structured task into a sequence of action tokens. Specifically, it includes the following steps: motion space definition and tokenization; motion space design: decomposing the robotic arm's movements into discrete atomic operations (grasping, moving, placing) and continuous parameters (joint angles, grasping force, moving speed) to form a structured motion space; example action token sequence: [grasping (medicine box A), moving (coordinates X, Y), placing (patient's hand), force (5N)]; token sequence generation: using a sequence-to-sequence (Seq2Seq) model (such as T5-3B), mapping the structured task representation to a sequence of action tokens. The input is the task representation, and the output is the action sequence, with cross-entropy loss used to optimize generation accuracy. Dynamic path planning and obstacle avoidance include: real-time environmental perception: continuously monitoring the dynamic changes in the medicine cabinet layout, patient position, and obstacles (such as other medical equipment) through a vision module. Path optimization: dynamically adjusting the robotic arm trajectory based on the A* algorithm or reinforcement learning (such as PPO) to ensure obstacle avoidance and minimize motion time. Example: When a sudden patient movement is detected, the model adjusts the grasping path in real time to avoid collisions. Safety constraints and anomaly handling include a safety verification layer: After the action sequence is generated, an independent safety verification module checks whether the actions comply with medical standards (such as dosage accuracy and patient allergy history). Anomaly handling mechanism: If a potential risk is detected (such as expired medication or mismatched patient identity), the model triggers an interruption mechanism, suspending distribution and notifying medical staff for manual review.

[0035] At the model architecture level, a "task instruction decoding module" is introduced to map medical natural language instructions (such as "Please distribute cold medicine on the third shelf of the medicine shelf in area A") into a sequence of action tokens, thereby achieving an interpretable conversion from semantics to actions.

[0036] Step S200: Obtain drug distribution sample data, and train the initial drug distribution model based on the drug distribution sample data to generate a drug distribution model.

[0037] The system acquires visual data of drugs, natural language instructions from medical staff, and action token sequence data to train an initial drug distribution model. Upon completion of training, a new drug distribution model is generated.

[0038] Step S200, namely, acquiring drug distribution sample data and training the initial drug distribution model based on the drug distribution sample data to generate a drug distribution model, includes: Step S201: Obtain drug distribution sample data, which includes drug visual data, natural language instructions, and action token sequence data; Step S202: Train the initial drug distribution model based on the drug visual data, language instructions, and action token sequence data; Step S203: When it is detected that the parameters of the initial drug distribution model meet the preset training conditions, the training is completed and the drug distribution model is generated.

[0039] Pre-acquire drug visual data, natural language commands, and action token sequence data. Drug visual data includes image data of drugs in different states, and includes, but is not limited to, drug data and drug shelf data.

[0040] Supervised training is performed using visual drug data, verbal commands, and action token sequence data. Visual drug data and verbal commands are used as inputs, and action token sequence data is used as the output.

[0041] During training, the training parameters of the model can be updated according to preset training conditions until the model meets the preset conditions, at which point the target training parameters are generated, and the drug distribution model is obtained.

[0042] Step S300: When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model.

[0043] When a drug dispensing instruction is detected, including but not limited to natural language instructions from medical staff and visual information about the drug, the instruction is input into a trained drug dispensing model. Based on this model, drug dispensing action data is output. This action data represents the robot's control actions.

[0044] Step S300, which involves detecting a drug distribution instruction, inputting the drug distribution instruction into the drug distribution model, and generating drug distribution action data based on the drug distribution model, includes: Step S301: When a drug distribution instruction is detected, the drug distribution instruction is parsed to obtain the target natural language instruction and drug visual information corresponding to the drug distribution instruction; Step S302: Input the target natural language instruction and drug visual information into the drug distribution model; Step S303: The target natural language instruction and the drug visual information are converted into drug distribution action data by the task instruction decoding module of the drug distribution model.

[0045] Medication dispensing instructions are entered through the hospital's HIS system or electronic prescription platform. The system automatically parses the prescription information (medication name, specifications, dosage, patient ID) and triggers the medication dispensing model. For example, after the patient pays, the system activates the automated dispensing machine by scanning a barcode / QR code, or completes identity verification by identifying the patient's wristband information using RFID.

[0046] The drug distribution model is based on multimodal data fusion (image recognition, barcode / RFID reading, weight sensing, temperature and humidity monitoring) for triple verification.

[0047] Visual Recognition: YOLOv8 or PP-OCRv4 models are used to recognize the text on the medicine box label (such as drug name, batch number, and expiration date). A rule engine filters invalid characters, and regular expressions are used to validate the format. Multimodal Verification: Barcode / QR code scanning and RFID identification are cross-verified to verify drug identity. A weight sensor performs a secondary check to ensure the consistency between the medicine box weight and the prescription dosage. For cold chain medicines, temperature and humidity sensors monitor the temperature range of 2-8℃ in real time. Rule Engine: Combining patient allergy history and medication contraindications (e.g., amoxicillin is contraindicated for penicillin allergies), conflict detection is performed using a knowledge graph to generate dynamic scheduling instructions (e.g., prioritizing the sorting of drugs nearing their expiration date).

[0048] Dispensing robots can physically move medications using robotic arm modules and conveyor systems, while intelligent medicine baskets track medication flow via GPS positioning and temperature / humidity recording modules. For example, direct-dispensing prescriptions are conveyed directly to the window via a powered spiral dispensing port, while mixed-dispensing prescriptions are conveyed to the dispensing station via a non-powered dispensing port, where pharmacists manually dispense the non-direct-dispensing medications. Path optimization: Heuristic or genetic algorithms are used for path planning, combined with constraints such as task priority (e.g., priority for emergency medications), medication expiration dates, and temperature control requirements, to generate the optimal sorting path. For example, the system automatically allocates storage locations and records three-dimensional coordinates, enabling efficient medication retrieval through a track-based shelving management system.

[0049] The system records information such as drug batch number, expiration date, temperature and humidity data, operator, and timestamp, generating an unalterable audit log and supporting end-to-end traceability (such as full-cycle management from warehousing to clinical use). Anomaly handling: An early warning mechanism is triggered for anomalies such as drug shortages or equipment malfunctions. For example, if the robotic arm of the automated dispensing machine malfunctions, it switches to manual dispensing mode, or tracks the location of the transport box via GPS to ensure uninterrupted workflow.

[0050] Step S400: Based on the drug distribution model, obtain the matching information between the drug to be distributed and the patient. If the drug to be distributed is successfully matched with the patient, control the robot to perform the corresponding distribution action based on the drug distribution action data.

[0051] Based on the drug distribution model, the matching information between the drugs to be distributed and the patients is obtained. Before each distribution action, the model visually re-identifies the matching information between the drug label and the patient. If the match is successful, the robot is controlled to perform the corresponding distribution action; if the match fails, the current distribution operation is paused and the error reason is sent to the terminal of the medical management personnel.

[0052] In step S400, the matching information between the drug to be distributed and the patient is obtained based on the drug distribution model. If the drug to be distributed and the patient are successfully matched, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data, including: Step S401: Obtain the label image of the medicine to be distributed, and input the label image of the medicine to be distributed into the medicine distribution model; Step S402: Based on the drug distribution model, identify the label image and obtain the identified label content; Step S403: Match the identified tag content with the patient ID; Step S404: If the identified tag content successfully matches the patient ID, then control the robot to perform the corresponding distribution action based on the drug distribution action data.

[0053] The visual recognition engine can identify drug labels. During recognition, it uses YOLOv8 or PP-OCRv4 models for text detection and recognition, extracting key fields such as drug name, specifications, and batch number. A rule engine filters invalid characters (such as those caused by stains or creases) and uses regular expressions to validate the format (e.g., batch numbers must be 8 digits).

[0054] Patient ID recognition employs a QR code decoding library (such as ZBar) or a deep learning model (such as CRNN), supporting robust recognition in tilted and blurred scenarios. ID validity is verified through the hospital's HIS system interface, rejecting unregistered or discharged IDs.

[0055] The matching and validation layer can employ multi-dimensional matching. Basic matching involves comparing the drug name and specifications against the drug information in the doctor's prescription. Advanced matching combines patient allergy history and contraindications (e.g., amoxicillin is contraindicated in patients with penicillin allergies) and uses a knowledge graph for conflict detection.

[0056] During matching, confidence levels are used to calculate text similarity (e.g., Levenshtein distance, BERT semantic embedding), and a threshold is set (e.g., >0.95 is considered a successful match). Low-confidence results trigger a manual review process.

[0057] When the label content successfully matches the patient ID, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0058] Step S400, which involves obtaining matching information between the drug to be distributed and the patient based on the drug distribution model, and if the drug to be distributed and the patient are successfully matched, then controlling the robot to perform the corresponding distribution action based on the drug distribution action data, includes: Step S501: During the robot's distribution action, obtain real-time correction information from the sensor feedback loop; Step S502: Perform real-time correction on the robot's actions based on the real-time correction information; Step S503: When the robot's distribution action is detected to be completed, robot trajectory data is generated.

[0059] Real-time correction via a sensor feedback loop (combining a torque sensor and a vision tracking system) is a key technology for improving the accuracy and safety of gripping operations in robots or automated equipment. This system effectively prevents over-gripping, slippage, or displacement through multimodal sensing and closed-loop control. For example, a torque sensor feedback loop can be used. It monitors changes in torque when the gripper contacts the object in real time to determine if the gripping force is appropriate. The torque value at the gripper joint is detected by strain gauges or piezoelectric sensors and converted into an electrical signal. A safe torque range is preset based on the object's material (low torque for fragile items) and weight (high torque for heavy objects).

[0060] During real-time calibration, if the torque exceeds the upper threshold, the system immediately reduces the gripper opening or stops clamping to prevent crushing the object. If the torque is below the lower threshold, the system increases the clamping force or adjusts the gripper posture to prevent the object from slipping.

[0061] The visual tracking feedback loop monitors the position and posture of the object in real time through a camera or 3D vision system and corrects the gripping deviation.

[0062] Deep learning models (such as YOLO and Mask R-CNN) are used to locate the object's coordinates in the image. Point cloud matching or keypoint detection is used to calculate the object's offset (e.g., angle, displacement) relative to the gripper. If the object's position deviates from the preset gripping point, the system adjusts the gripper's trajectory (e.g., translation, rotation). During the gripping process, the system continuously tracks the object's movement (e.g., items on a conveyor belt) and updates the gripping strategy in real time.

[0063] During the execution of the action token, the sensor feedback loop (torque + visual tracking) performs real-time correction to prevent over-gripping, slippage, or offset.

[0064] In some embodiments, step S400, which involves obtaining matching information between the drug to be dispensed and the patient based on the drug distribution model, and if the drug to be dispensed and the patient are successfully matched, further includes controlling the robot to perform the corresponding distribution action based on the drug distribution action data, after that: Step S601: Store the operation log and visual records of the drug distribution model; Step S602: Obtain the robot trajectory data set corresponding to each distribution action; Step S603: Obtain the optimal trajectory model based on the robot trajectory data set; Step S604: Using the optimal trajectory model, generate a set of safety constraints based on historical correct execution records.

[0065] In automated systems, automatically learning the optimal trajectory pattern after multiple task executions and forming a set of safety constraints based on historical correct execution records is a key technology for achieving adaptive and robust control. This process combines data-driven learning, trajectory optimization, and safety constraint generation, which can significantly improve the system's performance in complex or dynamic environments.

[0066] Extract safety constraints from historical data. Multimodal data recording: Trajectory data: Records the joint angles, end effector pose, velocity, acceleration, etc. of the robotic arm / gripper during each execution. Sensor data: Synchronously records feedback from torque sensors, vision systems, and tactile sensors (e.g., gripping force, object offset). Environmental data: Records the environmental conditions during task execution (e.g., lighting, object position changes, interference factors). Data labeling: Success / failure labels: Labels the results of each execution based on the task objectives (e.g., successful gripping, no slippage, no offset). Critical event labeling: Labels events that lead to failure (e.g., torque exceeding limits, visual target loss, trajectory collision).

[0067] Successful trajectories are clustered using K-means or DBSCAN to identify typical trajectory patterns in different task scenarios (such as "light object gripping" and "heavy object handling"). The time-series features of the trajectories are analyzed using Dynamic Time Warping (DTW) or Hidden Markov Models (HMMs) to extract key action nodes (such as gripper approach speed and contact force change points). High-dimensional trajectory data is reduced to 2D / 3D space using t-SNE or UMAP to visually observe the correlation between trajectory distribution and success / failure.

[0068] Safety constraint generation includes statistically based constraints: Torque constraints: Calculate the mean and standard deviation of gripper torques during successful historical executions and set a safety range (e.g., μ±3σ). Velocity constraints: Based on trajectory clustering results, set maximum permissible speeds for different scenarios (e.g., speed <0.1m / s during precision assembly). Acceleration constraints: Limit joint acceleration to prevent slippage due to inertia (e.g., acceleration <2rad / s²).

[0069] Rule-based constraints include the following types: Geometric constraints: Based on visual tracking data, a safe zone for the relative position of the object and the gripper is generated (e.g., "the gripper center must be within ±5mm of the object's center of gravity"). Temporal constraints: Defining the temporal sequence of key actions (e.g., "apply gripping force only after visual positioning is completed"). Machine learning-based constraints: Using decision trees or random forests to learn failure patterns from historical data and generate conditional constraints (e.g., "if the object's surface smoothness is >0.8, the gripping force needs to be increased by 20%").

[0070] After multiple executions, the system automatically learns the optimal trajectory pattern and forms a set of safety constraints based on historical correct execution records.

[0071] The model is designed to retain operation logs and visual records, allowing traceability of every step in drug distribution. The action decision path provides interpretable output (through attention visualization or semantic chain explanation), meeting the healthcare industry's requirements for safety and transparency. The interpretability of the action decision path is a core requirement for ensuring system credibility, meeting compliance requirements (such as FDA / GMP requirements for AI decision transparency), and assisting human intervention. Through attention visualization or semantic chain explanation, black-box decisions can be transformed into understandable logical chains.

[0072] Compared with the prior art, the embodiments of the present invention have the following technical advantages: The model, pre-trained for medical drug distribution scenarios, significantly enhances scene adaptability and maintains high recognition rate and stability under different hospital layouts, shelf arrangements, and lighting conditions. It can flexibly understand natural language commands from medical staff, achieving end-to-end closed-loop control of "command-vision-action".

[0073] Enhanced safety and accuracy: Dual verification through visual reconfirmation and sensor feedback significantly reduces the risk of incorrect distribution and physical collisions. A safety constraint mechanism based on historical trajectories ensures smooth and controllable action execution.

[0074] Strong scalability and self-learning capabilities: The system can adaptively optimize by continuously collecting real distribution data, and learn over a long period of time about different drug forms, packaging updates, and new task instructions. It supports migration and application in other medical subtasks (such as drug verification, sample delivery, and material delivery).

[0075] Compliance and traceability: The full-process action recording and log tracking mechanism meets medical safety and regulatory requirements, and can provide reliable assurance for the implementation of intelligent medical systems.

[0076] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0077] Another embodiment of the present invention provides an artificial intelligence-based drug dispensing device, which corresponds one-to-one with the artificial intelligence-based drug dispensing method described in the above embodiments. For example... Figure 3 As shown, device 1 includes: The model building module 100 is used to acquire multimodal data of medical scenarios, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model. The model training module 200 is used to acquire drug distribution sample data, train the initial drug distribution model based on the drug distribution sample data, and generate a drug distribution model. The data processing module 300 is used to input the drug distribution instruction into the drug distribution model when a drug distribution instruction is detected, and to generate drug distribution action data based on the drug distribution model. The action execution module 400 is used to obtain matching information between the drug to be distributed and the patient based on the drug distribution model. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0078] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0079] In one embodiment, the model building module 100 is specifically used for: Construct a VLA model, which forms a shared semantic representation space in the visual, linguistic, and action dimensions; Add a task instruction decoding module to the VLA model to generate an initial VLA model; Acquire multimodal data of the medical scenario, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model.

[0080] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0081] In one embodiment, the model training module 200 is specifically used for: Acquire drug distribution sample data, which includes drug visual data, natural language commands, and action token sequence data; The initial drug distribution model is trained based on the drug visual data, language commands, and action token sequence data. When the parameters of the initial drug distribution model are detected to meet the preset training conditions, the training is completed and a drug distribution model is generated.

[0082] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0083] In one embodiment, the data processing module 300 is specifically used for: When a drug distribution instruction is detected, the drug distribution instruction is parsed to obtain the target natural language instruction and drug visual information corresponding to the drug distribution instruction; The target natural language command and drug visual information are input into the drug distribution model; The task instruction decoding module of the drug distribution model converts the target natural language instruction and the drug visual information into drug distribution action data.

[0084] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0085] In one embodiment, the action execution module 400 is specifically used for: Obtain the label image of the medicine to be distributed, and input the label image of the medicine to be distributed into the medicine distribution model; The label image is identified based on the drug distribution model to obtain the identified label content; The identified tag content is matched with the patient ID; If the identified tag content successfully matches the patient ID, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0086] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0087] In one embodiment, the apparatus further includes a calibration module, which is specifically used for: During the robot's distribution action, real-time correction information from the sensor feedback loop is acquired; The robot's actions are corrected in real time based on the real-time correction information; When the robot's distribution action is detected to be completed, robot trajectory data is generated.

[0088] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0089] In one embodiment, the apparatus further includes a storage and analysis module, which is specifically used for: The operation logs and visual records of the drug distribution model are stored; Obtain the robot trajectory data set corresponding to each distribution action; The optimal trajectory model is obtained based on the robot trajectory data set; Using the optimal trajectory model, a set of safety constraints based on historical correct execution records is generated.

[0090] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0091] This invention provides an artificial intelligence-based drug dispensing device that, through pre-training on medical drug dispensing scenarios, can maintain high recognition rate and stability under different hospital layouts, drug shelf arrangements, and lighting conditions; it can flexibly understand medical staff's natural language instructions, thereby improving recognition accuracy.

[0092] Another embodiment of the present invention provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side artificial intelligence-based drug distribution method.

[0093] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of an artificial intelligence-based drug distribution method.

[0094] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire multimodal data from a medical scenario, pre-train an initial VLA model based on the multimodal data, and generate an initial drug distribution model; Obtain sample data of drug distribution, and train the initial drug distribution model based on the sample data of drug distribution to generate a drug distribution model; When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model; Based on the drug distribution model, matching information between the drug to be distributed and the patient is obtained. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0095] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire multimodal data from a medical scenario, pre-train an initial VLA model based on the multimodal data, and generate an initial drug distribution model; Obtain sample data of drug distribution, and train the initial drug distribution model based on the sample data of drug distribution to generate a drug distribution model; When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model; Based on the drug distribution model, matching information between the drug to be distributed and the patient is obtained. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

[0096] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0098] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can exist in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0100] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

[0101] Among other things, conditional language such as “can,” “may,” “may,” or “may,” unless otherwise specifically stated or otherwise understood as in the context in which they are used, is generally intended to convey that a particular implementation may include (but not others) certain features, elements, and / or operations. Therefore, such conditional language is also generally intended to imply that features, elements, and / or operations are necessary for one or more implementations in any way, or that one or more implementations must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or will be performed in any particular implementation.

[0102] The contents already described herein in this specification and accompanying drawings include examples of methods and apparatuses capable of providing artificial intelligence-based drug dispensing. It is certainly not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of this disclosure, but it will be appreciated that many other combinations and substitutions of the disclosed features are possible. Therefore, it will be apparent that various modifications can be made to this disclosure without departing from the scope or spirit of this disclosure. Furthermore, or in alternatives, other embodiments of this disclosure may become apparent from consideration of this specification and accompanying drawings and from practice of this disclosure as presented herein. It is intended that the examples presented in this specification and accompanying drawings be considered illustrative rather than restrictive in all respects. Although specific terminology is used herein, it is used in a general and descriptive sense and is not intended for limiting purposes.

Claims

1. A drug distribution method based on artificial intelligence, characterized in that... The method includes: Acquire multimodal data from a medical scenario, pre-train an initial VLA model based on the multimodal data, and generate an initial drug distribution model; Obtain sample data of drug distribution, and train the initial drug distribution model based on the sample data of drug distribution to generate a drug distribution model; When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model; Based on the drug distribution model, matching information between the drug to be distributed and the patient is obtained. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

2. The drug distribution method based on artificial intelligence according to claim 1, characterized in that, The process of acquiring multimodal data from a medical scenario, pre-training an initial VLA model based on the multimodal data, and generating an initial drug distribution model includes: Construct a VLA model, which forms a shared semantic representation space in the visual, linguistic, and action dimensions; Add a task instruction decoding module to the VLA model to generate an initial VLA model; Acquire multimodal data of the medical scenario, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model.

3. The drug distribution method based on artificial intelligence according to claim 1, characterized in that, The step of acquiring drug distribution sample data and training the initial drug distribution model based on the drug distribution sample data to generate a drug distribution model includes: Acquire drug distribution sample data, which includes drug visual data, natural language commands, and action token sequence data; The initial drug distribution model is trained based on the drug visual data, language commands, and action token sequence data. When the parameters of the initial drug distribution model are detected to meet the preset training conditions, the training is completed and a drug distribution model is generated.

4. The artificial intelligence-based drug distribution method according to claim 2, characterized in that, When a drug distribution instruction is detected, the drug distribution instruction is input into the drug distribution model, and drug distribution action data is generated based on the drug distribution model, including: When a drug distribution instruction is detected, the drug distribution instruction is parsed to obtain the target natural language instruction and drug visual information corresponding to the drug distribution instruction; The target natural language command and drug visual information are input into the drug distribution model; The task instruction decoding module of the drug distribution model converts the target natural language instruction and the drug visual information into drug distribution action data.

5. The artificial intelligence-based drug distribution method according to claim 1, characterized in that, The process involves obtaining matching information between the drugs to be distributed and the patients based on the drug distribution model. If a match is found, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data, including: Obtain the label image of the medicine to be distributed, and input the label image of the medicine to be distributed into the medicine distribution model; The label image is identified based on the drug distribution model to obtain the identified label content; The identified tag content is matched with the patient ID; If the identified tag content successfully matches the patient ID, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

6. The drug distribution method based on artificial intelligence according to claim 1, characterized in that, The process involves obtaining matching information between the drug to be distributed and the patient based on the drug distribution model. If the drug to be distributed and the patient are successfully matched, the robot is then controlled to perform the corresponding distribution action based on the drug distribution action data. This includes: During the robot's distribution action, real-time correction information from the sensor feedback loop is acquired; The robot's actions are corrected in real time based on the real-time correction information; When the robot's distribution action is detected to be completed, robot trajectory data is generated.

7. The artificial intelligence-based drug distribution method according to claim 6, characterized in that, The process of obtaining matching information between the drug to be distributed and the patient based on the drug distribution model, and if the drug to be distributed and the patient are successfully matched, and then controlling the robot to perform the corresponding distribution action based on the drug distribution action data, further includes: The operation logs and visual records of the drug distribution model are stored; Obtain the robot trajectory data set corresponding to each distribution action; The optimal trajectory model is obtained based on the robot trajectory data set; Using the optimal trajectory model, a set of safety constraints based on historical correct execution records is generated.

8. A drug dispensing device based on artificial intelligence, characterized in that, The device includes: The model building module is used to acquire multimodal data of medical scenarios, pre-train the initial VLA model based on the multimodal data, and generate an initial drug distribution model. The model training module is used to acquire drug distribution sample data, train the initial drug distribution model based on the drug distribution sample data, and generate a drug distribution model. The data processing module is used to input the drug distribution instruction into the drug distribution model when a drug distribution instruction is detected, and to generate drug distribution action data based on the drug distribution model. The action execution module is used to obtain matching information between the drug to be distributed and the patient based on the drug distribution model. If the drug to be distributed is successfully matched with the patient, the robot is controlled to perform the corresponding distribution action based on the drug distribution action data.

9. A computer device, characterized in that, The computer device includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the artificial intelligence-based drug dispensing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the artificial intelligence-based drug dispensing method according to any one of claims 1-7.

Images