Intelligent medicine delivery and medicine taking identification robot for inpatient department and use method of intelligent medicine delivery and medicine taking identification robot

Abstract

The invention relates to an intelligent medicine delivery and medicine taking identification robot for the inpatient department and a use method, and belongs to the field of non-medical medical services, and the intelligent medicine delivery and medicine taking identification robot comprises a robot main body structure, a high-performance micro-host, an energy supply device and an advancing device; a screen display device, an image acquisition device and a medicine storage device are arranged on the main body structure; the screen display device is used for operation of medical staff and information interaction with hospitalized patients; the image acquisition device is used for verifying the identity of a medical worker and judging the medicine taking condition of a hospitalized patient; the medicine storage device comprises a depth binocular 3D camera, a medicine carrying drawer and an air cylinder expansion piece. According to a hospital system and doctor presetting, the robot automatically provides medicine to an affected part, the medicine taking process is strictly monitored, heavy work of medical staff is effectively relieved, and it is guaranteed that a patient can correctly take medicine on time.

Description

Technical Field

[0001] The present invention belongs to the field of non-medical medical services, and in particular relates to an intelligent medicine delivery and medication recognition robot for an inpatient department and a method of using the robot. Background Art

[0002] Hospital inpatient staff are overwhelmed with workloads and are unable to ensure that every patient takes their medication during bedside delivery hours. While currently available medication delivery robots deliver medication to hospitals, they do not employ computerized methods to remind patients to take their medication or monitor their medication intake. This invention aims to reduce the workload of medical staff and ensure medication safety for patients.

[0003] Patent publication number CN116533263B determines the presence of medication in a drawer based on the drawer's weight. This method ignores the possibility of medication being underweight, which can easily lead to calculation errors. The robot invented in this patent does not provide a clear method or device to determine whether a patient has taken medication, nor does it use an information system to accurately record patient medication information in a timely manner. However, the use of a medication drawer recognition system and a medication posture recognition system addresses this issue. Using artificial intelligence algorithms, they accurately analyze images of the drawer's interior and the patient's medication posture to determine whether medications in the drawer have been removed and taken. This system is convenient, fast, and highly accurate. Patent publication number CN117584151A uses electromagnetic drawer opening and closing, which prevents medical staff from centrally controlling the drawer's opening and closing, consuming time and effort. In contrast, a pneumatic device controls the opening and closing of the drawer, which is convenient and reliable. Using compressed air as a power source eliminates the use of any hazardous chemicals and is environmentally friendly to the drawer's interior. The operation of the cylinder retractor is automatically managed by a control system, ensuring precise and smooth drawer movement. Pneumatic cylinder retractors are relatively simple and reliable, with low maintenance requirements and easy adjustment of the speed and force of the retractor. This power supply device is suitable for a variety of applications that require precise control and repeatable operation, such as industrial automation, construction and heavy machinery operation. Summary of the Invention

[0004] The purpose of the present invention is to solve the technical defects of hospital robots in the existing technology in judging and identifying drug administration, as well as the problems in the use of medicine drawers, and to propose an intelligent medicine delivery and medication identification robot for inpatient departments and a method of use.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An intelligent medicine delivery and medication recognition robot for inpatient departments, comprising a main structure, a high-performance micro host, an energy supply device, and a travel device;

[0007] The main structure is equipped with a screen display device, an image acquisition device, and a medicine storage device;

[0008] The screen display device is used for medical staff's operations and information interaction with hospitalized patients;

[0009] Image acquisition devices are used to verify the identity of medical staff and determine the medication status of hospitalized patients;

[0010] The medicine storage device includes a medicine loading drawer and a cylinder retractor, wherein the multiple medicine loading drawers are arranged in a matrix on the main structure;

[0011] The cylinder retractor includes a cylinder, a retractor, and a piston. The air pipe interface is connected to the energy supply device. The piston, cylinder, and retractor are connected in sequence. The gas in the piston is transmitted to the cylinder, and the cylinder inflates the retractor, pushing the retractor forward to drive the medicine-loading drawer out of the main structure. One cylinder retractor corresponds to one medicine-loading drawer.

[0012] The high-performance micro host connects various systems and modules in the robot for data processing and control;

[0013] The energy supply device includes a power source and an air source. The power source supplies power to various systems and modules of the robot, and the air source provides driving power for the cylinder telescope.

[0014] As a further preferred solution, the screen display device includes a high-definition touch screen, a speaker, and a microphone, wherein the high-definition touch screen is used for medical staff's operational work, the speaker is used to play prompt information, and the microphone is used for real-time conversations between medical staff and hospitalized patients.

[0015] As a further preferred solution, the image acquisition device includes a face recognition camera, an image sensor, and a deep binocular 3D camera; the face recognition camera is used to capture the action of hospitalized patients taking medicine; the deep binocular 3D camera is used to capture the storage status of medicines in the medicine drawer; the image sensor is connected to the face recognition camera and the deep binocular 3D camera via a signal line.

[0016] As a further preferred option, the traveling device is the walking component of the robot, which is controlled by a high-performance micro host and powered by a power supply. It includes a lidar, a traveling camera, and two pairs of universal wheels at the front and back of the robot. The robot uses the lidar and traveling camera to scan the environment and build or update the indoor map in real time.

[0017] As a further preferred solution, ultraviolet light strips are provided on both sides of the medicine drawer.

[0018] A method for using an intelligent medicine delivery and medication recognition robot in an inpatient department includes the following steps:

[0019] Step 1: Medical staff log in after facial recognition verification through the image acquisition device. The medication management system is connected to the hospital inpatient management system through the data connection module, and the medication management system updates the inpatient information in real time;

[0020] Step 2: Medical staff use the high-definition touch screen to open or individually open the medicine drawers in batches according to the medication management system, place the medicines according to the inpatient information and medication information, and then close the medicine drawers.

[0021] Step 3: The robot delivers medicine three times a day, ward by ward and bed by bed according to a planned route. During the delivery, the robot's facial recognition camera identifies the ward and bed numbers, transmits the captured digital image to a high-performance microcomputer, and then loads the corresponding inpatient information and displays it on a high-definition touchscreen display.

[0022] Step 4: When the robot moves to the side of the inpatient, the speaker plays a medication reminder tone and the robot automatically adjusts its position until it is facing the inpatient. Then, the facial recognition camera recognizes the inpatient's facial information and transmits the collected facial image of the inpatient to the high-performance micro host. The facial recognition algorithm compares and verifies the inpatient's facial information with the facial feature data of the inpatient information. The inpatient scans the patient identification code worn on the wrist on the facial recognition camera and compares and verifies it with the patient identification code data of the inpatient information. After the inpatient's facial information and patient identification code are simultaneously verified, the medicine drawer opens.

[0023] Step 5: The screen display device displays the inpatient's medication information, and the loudspeaker plays the name of the medication, the number of times a day the medication is taken, the dosage for each dose, and precautions;

[0024] Step 6: The robot's image acquisition device captures the patient's motion characteristics during the medication-taking process, including picking up the medication, raising the hand, placing the medication in the mouth, and swallowing. The medication posture recognition system's medication posture recognition model identifies whether the patient has taken the medication. Simultaneously, the medication drawer recognition system identifies whether the medication drawer is empty, comprehensively determining that the patient has taken the medication. The medication management system records the medication taking time, and the high-performance microcomputer stores the medication information video of the patient taking the medication process. The medication taking results are recorded in the medication management system.

[0025] Step 7: After taking the medicine, the cylinder retractor closes the medicine drawer of the current inpatient;

[0026] Step 8: After the robot completes the first round of medicine delivery, if the facial recognition camera does not identify the inpatient in the corresponding ward and bed, it will perform a second round of medicine delivery after an interval of 15 minutes; after the second round of medicine delivery, it will return to the nurse station and upload the delivery results in batches to the hospital inpatient management system.

[0027] As a further preferred solution, in step 6, the robot's image acquisition device obtains the motion characteristics of the hospitalized patient during the medication-taking process, including two stages: IndexPose-RTDETR algorithm model training and IndexPose-RTDETR algorithm deployment and use;

[0028] IndexPose-RTDETR algorithm model training segment:

[0029] Step 1: First, collect videos of hospitalized patients picking up medications captured by a deep binocular 3D camera, and videos of hospitalized patients raising their hands, placing them in their mouths, and swallowing them captured by a facial recognition camera. These videos are then processed by a high-performance microcomputer into a fused video segment as a medication information video. Video frame images are extracted from all medication information videos.

[0030] Step 2: Divide the video frame images into a training set, a validation set, and a test set; annotate the training set, validation set, and test set with action categories, and set the index number set corresponding to the action category; train the training set using the IndexPose-RTDETR algorithm, and use the validation set to determine whether the IndexPose-RTDETR algorithm training has converged; finally, obtain the average precision and recall rate of the test medication posture recognition model using the test set;

[0031] The backbone network of the IndexPose-RTDETR algorithm network model is ConvNeXt, which optimizes feature learning through a fully convolutional masked autoencoder and a global response normalization layer;

[0032] The BIFI substructure of the hybrid encoder introduces the Biformer attention mechanism; the Biformer attention mechanism first divides and projects the region, Figure X ∈R H×W×C Divided into S 2 Regions, through linear projection to obtain query Q, key K and value V; then perform regional affinity calculation, through region-level query Q r and key K r Matrix multiplication of constructs affinity graph A r , and filter the top k regions, the formula is as follows:

[0033]

[0034] I r =topkIndex(A r )

[0035] The Biformer attention mechanism finally performs token attention and collects the key-value pairs K of the routing area. g and V g, performing dense matrix multiplication, where LCE(·) is the local context enhancement module, implemented using depthwise convolution, and the formula is as follows:

[0036] O=Attention(Q,K g ,V g )+LCE(V)

[0037] The CCFM substructure of the hybrid encoder introduces the EUCB convolution block to perform cross-scale feature fusion on multi-scale action features to generate an action image feature sequence containing rich semantic information. The EUCB convolution block introduces the Shift-Channel-Mix operation, which enhances the expressiveness of the medication action feature map through channel segmentation and spatial offset. The mathematical formula of the EUCB convolution block is as follows:

[0038]

[0039] Step 3: The posture judgment part clusters the stored medication action recognition detection frames, uses the action category, index number, and detection frame coordinates as feature vectors, clusters them using the K-means clustering algorithm, extracts the action category and index number of each cluster center point, and sorts them by time information;

[0040] Step 4: Based on the index numbers of the cluster centers, after sorting by time information, determine whether they form the order of 0->1->2->3->4. If so, the medication posture recognition system determines that the inpatient has taken the medication. If the order is inconsistent, or any index number is missing, the inpatient is determined to have not taken the medication.

[0041] IndexPose-RTDETR algorithm deployment and use phase:

[0042] The trained medication posture recognition model is deployed on a high-performance microcomputer. During medication delivery, a deep binocular 3D camera captures videos of the inpatient picking up medication, while a facial recognition camera captures videos of the inpatient raising their hand, placing the medication in their mouth, and swallowing it. The two segmented videos are fused as a whole and input into the medication posture recognition system for analysis. The output of the medication posture recognition model is the inpatient's medication intake result.

[0043] The medication information video of the hospitalized patients is stored in real time in the memory of the high-performance microcomputer, and the medication delivery results are recorded in the medication management system.

[0044] As a further preferred solution, the main modules of the medication management system include data connection module, system login module, drug access module, ultraviolet light strip control module, inpatient medication management module, and medication management database.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) The present invention uses a medication posture recognition system to judge the medication process of hospitalized patients, and uses an artificial intelligence algorithm medication posture recognition model to extract features from the captured medication information video, thereby judging whether the hospitalized patients have taken medication. The medication posture recognition system can accurately judge whether hospitalized patients have taken medication, replacing the medical staff's manual review of the hospitalized patients' medication process, saving manpower and improving work efficiency. Compared with the existing solution, which simply judges whether the medicine in the medicine drawer has been taken by the hospitalized patient based on the weight difference before and after taking the medicine, the use of an advanced artificial intelligence algorithm medicine drawer recognition model has higher recognition accuracy and recognition speed. Higher, recognition speed. If the hospitalized patient does not take the medicine correctly and on time or the hospitalized patient is not in bed, the present invention ensures the accuracy of the hospitalized patient's medication and the personal safety of the hospitalized patient by sending a warning to the medical staff.

[0047] (2) The use of pneumatic devices to control the opening and closing of the medicine drawer is convenient and reliable. The use of compressed air as a power source does not involve any harmful chemicals and is friendly to the environment inside the medicine drawer. The operation of the cylinder retractor is automatically managed by the control system to ensure precise and smooth movement of the drawer. The cylinder retractor is relatively simple and reliable, with low maintenance requirements and easy adjustment of the speed and force of the drawer. By using compressed air, a clean, reliable, low-cost and high-efficiency power solution is provided for the cylinder retractor.

[0048] (3) An intelligent medication delivery and medication recognition robot for inpatient departments replaces medical staff in delivering medications on a daily basis. It can place medications for each inpatient in a medication drawer and uses facial recognition and patient identification code authentication to prevent the wrong medication from being taken. This invention has multiple functions, including autonomous navigation and obstacle avoidance, to ensure the safety of medications during transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is the left side view of the overall appearance of the device;

[0050] Figure 2 This is the rear view of the overall appearance of the device;

[0051] Figure 3 This is a top view of the medicine drawer device;

[0052] Figure 4 This is a top view of the energy supply device;

[0053] Figure 5 This is the rear view of the high-performance micro host;

[0054] Figure 6 This is a top view of the interior of the high-performance micro host;

[0055] Figure 7 The main view of the screen display device;

[0056] Figure 8 This is a schematic diagram of the medication management system structure;

[0057] Figure 9 This is the network framework structure diagram of the IndexPose-RTDETR algorithm;

[0058] Figure 10 It is the EUCB convolutional block network structure;

[0059] Figure 11 This is the software and hardware structure diagram of the robot's overall device;

[0060] Among them, the main structure includes 1, partition 2, high-definition touch screen 3, screen embedded frame 4, speaker 5, microphone 6, face recognition camera 7, image sensor 8, connection interface 9, deep binocular 3D camera 10, medicine drawer 11, ultraviolet light strip 12, extension plate 13, cylinder 14, telescope 15, piston 16, electric control valve 17, trachea interface 18, laser radar 19, high-performance micro host 20, computer CPU 21, memory 22, storage 23, data transmission interface 24, traveling device 25, universal wheel 26, traveling camera 27, energy supply device 28, 12V low-temperature lithium battery 29, wireless charging device 30, hydraulic gas storage tank 31, and fixed bracket 32. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0062] like Figure 1 As shown, an intelligent medicine delivery and medication recognition robot for an inpatient department includes a main structure 1, a screen display device, an image acquisition device, a medicine storage device, a high-performance micro host 20, a traveling device 25, and an energy supply device 28.

[0063] The main structure 1 comprises three layers: upper, middle, and lower, each separated by partitions 2. The upper layer includes a screen display and image acquisition device. The middle layer includes a drug storage device and a high-performance microcomputer 20. The lower layer includes a travel device 25 and an energy supply device 28.

[0064] The screen display device includes a high-definition touch screen 3, a display screen embedded frame 4, a speaker 5, and a microphone 6. The display screen embedded frame 4 is located at the top of the main structure 1 at a 45° angle to the horizontal plane of the main structure 1, and the high-definition touch screen 3 is embedded in the display screen embedded frame 4. After medical staff pass facial verification, they can use the medication management system of the high-definition touch screen 3. After the inpatient passes facial verification, the high-performance micro host 20 can issue a command to display the inpatient's medication information on the high-definition touch screen 3, so that the inpatient can view the doctor's orders more intuitively and take the medicine correctly. The speaker 5 is embedded in the lower right corner of the display screen embedded frame 4 and is used to play safety reminder information when the robot is moving, medication information of inpatients, and timely remind inpatients to take medicine when they have not taken medicine. The microphone 6 is embedded in the upper right corner of the display screen embedded frame 4 and is used for real-time communication between medical staff and inpatients.

[0065] The image acquisition device includes a facial recognition camera 7, an image sensor 8, a connection interface 9, a depth binocular 3D camera 10, and an image analysis and processing system. The facial recognition camera 7 is embedded in the upper end of the display screen embedded frame 4. The image sensor 8 is located at the lower left of the display screen embedded frame 4. The connection interface 9 connects the image sensor 8 to a high-performance microcomputer 20, facilitating the image sensor's conversion of raw image data and transmission to the image analysis and processing system. The image analysis and processing system runs within the high-performance microcomputer 20. The image acquisition device is used to capture facial images of medical staff and hospitalized patients, and is used to verify the identity of medical staff and determine whether hospitalized patients have taken medication.

[0066] The medicine storage system, located in the middle layer of the main structure 1, consists of 40 medicine drawers 11, UV light strips 12, and a cylinder expansion joint. A deep binocular 3D camera 10 is located at the center of the bottom of an extension plate 13 above the middle layer of the main structure 1. The 40 medicine drawers 11 are arranged in a 4×10 matrix. Each medicine drawer 11 is equipped with a UV light strip 12 on the inner side of the left and right panels. The rear of each medicine drawer 11 is connected to a cylinder expansion joint.

[0067] The cylinder retractor consists of a cylinder 14, a retractor 15, a piston 16, an electrically controlled valve 17, and an airway interface 18. When medical staff place medications or an inpatient passes facial verification, the gas in the hydraulic reservoir 31 is transferred into the cylinder 14 through the airway interface 18 under the control of the electrically controlled valve 17. This increases the air pressure in the cylinder 14, pushing the piston 16 outward, which in turn pushes the retractor 15 forward, opening the corresponding medicine drawer 11. Simultaneously, the deep binocular 3D camera 10, located in the center of the extension plate 13 above the middle layer of the main structure 1, begins capturing image data of the medicine drawer 11. After the medicine drawer recognition system determines that the inpatient has taken their medication, the gas in the cylinder 14 is discharged into the hydraulic reservoir 31 through the airway interface under the control of the electrically controlled valve 17. This reduces the air pressure in the cylinder 14, causing the piston 16 to retract inward, causing the retractor 15 to reset and close the medicine drawer 11. The air pipe interface 18 is connected to the hydraulic air storage tank 31 in the energy supply device 28, and the cylinder 14 is inflated and exhausted under the control of the electric control valve 17; the solenoid valve is controlled by the system in the high-performance micro host.

[0068] A deep binocular 3D camera 10 is installed in the center of the bottom of the extension plate 13 above the middle layer of the main structure 1. It is used to capture the internal situation of the medicine drawer 11 after it is opened, and then transmit the captured image to the high-performance micro host 20 for analysis. The medicine drawer recognition system is used to determine that the medicine drawer 11 is empty, and it is determined that the hospitalized patient has taken the medicine. Then the high-performance micro host 20 drives the cylinder retractor to close the medicine drawer 11.

[0069] The ultraviolet light strip 12 can sterilize and disinfect the medicine drawer 11. After the medical staff passes the face recognition verification, they can set the ultraviolet light strip 12 disinfection start time, disinfection duration and daily disinfection frequency on the high-definition touch screen 3.

[0070] A deep binocular 3D camera 10 is located at the center of the bottom of the extension plate 13 above the middle layer of the main structure 1. It is primarily used to capture the storage status of the medicines in the drawers and transmit the captured images to the high-performance microcomputer 20. The medicine drawer recognition system analyzes the valid features in the images to determine whether the medicines in the medicine drawer 11 have been removed. The detection system in the high-performance microcomputer 20 also recognizes the information on the medicine packaging in the images to ensure the accuracy of the medicines placed.

[0071] The LiDAR 19 is located in the center above the front of the traveling device 25. During travel, it actively emits an invisible laser beam toward the environment ahead and captures the echo signal reflected by obstacles. Based on the time-of-flight (ToF) principle, it accurately calculates the laser round-trip time difference and analyzes the obstacle's spatial parameters (distance, three-dimensional dimensions) and dynamic properties (stationary or moving state) in real time. The high-precision point cloud data it collects is processed in real time by a high-performance microcomputer 20. The fusion algorithm constructs an environmental model and generates navigation decision instructions (such as steering, acceleration and deceleration, and emergency braking). It drives the motion control system to dynamically adjust the path, achieving autonomous obstacle avoidance and optimal route planning in complex scenarios, ensuring the efficient and safe execution of drug delivery tasks.

[0072] A high-performance microcomputer 20 is located in the middle layer of the main structure 1. Its hardware includes a computer CPU 21, memory 22, storage 23, and a data transmission interface 24. Its software includes a computer operating system, a medication management system, a medication posture recognition system, a medication drawer recognition system, an image analysis and processing system, a detection system, and an electronic control system. The medication management system interface is displayed on a high-definition touch screen 3 of the screen display device.

[0073] The main modules of the medication management system include data connection module, system login module, drug access module, ultraviolet light strip control module, inpatient medication management module, and medication management database. Figure 8 FIG. 1 is a structural diagram of a medication management system according to an embodiment of the present invention.

[0074] The data connection module is responsible for data communication between the medication management system and the hospital's inpatient management system. It sends service access requests, receives inpatient and medication information from the inpatient management system, and stores it in a database. This ensures that data in the medication management system is updated in real time, reflecting the latest patient status. The data communication hardware uses SIMCom's SIM8200EA-M2, which supports 3GPP R15 / R16 protocols. This ensures stable 4G / 5G data communication between the robot and the hospital's inpatient management system in different areas of the hospital's inpatient department, preventing data updates from being delayed due to network issues. The SIM8200EA-M2 connects to a high-performance microcomputer via PCIe or USB interfaces, enabling high-speed data transmission.

[0075] Inpatient information includes hospitalization number, name, gender, age, medical history, medication information, length of stay, ward number, bed number, facial features, patient identification code, and allergy information. Medication information includes medication name, number of doses per day, dosage per dose, and precautions. Medication delivery results include hospitalization number, name, medication administration time, medication administration results, and medication administration video. Medication administration results include whether the medication has been taken or not.

[0076] The system login module's primary function is to verify that anyone attempting to log into the medication management system is an authorized healthcare professional. This module uses an image acquisition device to capture the login individual's facial information and compares it with pre-stored facial feature data for healthcare professionals in the database to prevent unauthorized access.

[0077] The image acquisition device captures and crops the detected facial region, then performs face alignment to adjust the facial image to a consistent pose and position for easier feature extraction, generating the logged-in person's facial information. Using face detection algorithms, such as a cascaded classifier based on Haar features or a deep learning-based object detection algorithm, high-level semantic features of the face are extracted and compared with the facial feature data from the healthcare provider information table in the medication management database.

[0078] The medication access module is used to load and display each inpatient's inpatient and medication information, and allows medical staff to operate the medication drawers 11. This module supports two operating modes: one is to control the simultaneous opening of all 40 medication drawers 11, and the other is to control the opening and closing of a single medication drawer 11. On the high-definition touch screen 3, the medication access module interface uses a three-layer information structure:

[0079] The first level displays the ward numbers for all wards. Clicking a ward number brings them to the second level. The second level displays the names of the patients assigned to each bed in the selected ward. Clicking a name brings them to the third level, which displays detailed hospitalization and medication information for the selected patient, including medication name, daily doses, dosage, and precautions. This three-level information structure allows medical staff to more intuitively and conveniently prescribe medications for each patient in each ward, reducing the possibility of operational errors.

[0080] The UV light strip control module is used to activate the UV light strip 12 in the medication storage device to ensure a hygienic and safe medication storage environment. Medical staff access the UV light strip control module from the medication management system interface and can use it to set the UV light strip 12 activation time, disinfection activation time, disinfection duration, and daily disinfection frequency.

[0081] After medical staff set the UV lamp's on / off settings, disinfection time, duration, and frequency on the medication management system interface, the system transmits these parameters to a microcontroller connected to a high-performance microcomputer 20. Based on pre-set control logic, the microcontroller periodically outputs corresponding level signals to control the UV lamp's on / off function. The UV lamp is set to turn on at a specific time each day for disinfection. When the disinfection time arrives, the microcontroller controls the GPIO pin to output a high level, energizing the control circuit and turning the UV lamp on. Once disinfection is complete, the microcontroller outputs a low level, turning the UV lamp off.

[0082] The inpatient medication management module connects to the medication gesture recognition system and the medicine drawer recognition system via an interface. It receives the inpatient medication taking results transmitted by the medication gesture recognition system and the empty result of the medicine drawer 11 transmitted by the medicine drawer recognition system, indicating that the inpatient has taken the medication. The inpatient medication management module updates the data to the database.

[0083] The medication management database runs on the relational database management software MySQL. It queries data based on business requests sent by the medication management system and returns the results to the management system. The database tables created in the medication management database include an inpatient information table, a medication information table, a medication delivery results table, a medication results table, and a medical staff information table. The fields in the inpatient information table are the inpatient number primary key, hospitalization number, name, gender, age, case information, medication information, length of stay, ward number, bed number, facial feature data, patient identification code, and allergy information. The fields in the medication information table are the medication information number primary key, drug name, number of medications taken per day, dosage per dose, and precautions. The fields in the medication delivery results table include the number primary key, hospitalization number, name, medication time, medication results foreign key, and medication information video. Medication results include the medication result number primary key, medication taken, and medication not taken. The fields in the medical staff information table are the medical staff number primary key, name, and facial feature data.

[0084] The medication posture recognition system runs in a high-performance micro host 20, deploys a trained medication posture recognition model, and extracts features from the captured medication information video to determine whether the hospitalized patient has taken the medication.

[0085] The medication posture recognition model is based on the IndexPose-RTDETR algorithm. First, the medication posture information video dataset is trained to obtain the training weight of the trained medication posture recognition model, and then the medication posture recognition model is deployed to the high-performance micro host 20.

[0086] Figure 8 This is a network framework diagram of the IndexPose-RTDETR algorithm according to an embodiment of the present invention, see Figure 8It can be seen that the IndexPose-RTDETR algorithm network framework provided by the embodiment of the present invention includes six parts: ConvNeXt backbone network, hybrid encoder, feature pyramid network, decoder, detection head, and posture judgment. The backbone network performs shallow feature extraction on medication posture images. The hybrid encoder performs deep feature extraction and cross-layer feature fusion on medication posture images. The hybrid encoder consists of a BIFI substructure and a CCFM substructure. A feature pyramid is used to enhance multi-scale posture feature extraction. Through top-to-bottom paths and lateral connections, it ensures comprehensive coverage of targets in different postures. The decoder selects key medication posture image features from the output of the hybrid encoder as the initial target query and generates accurate bounding boxes and confidence scores through iterative optimization. The detection head decodes the processed features into the final posture category and bounding box, generating medication action recognition detection frames that store action category, index number, detection frame coordinates, confidence score, and time information. Posture judgment uses the K-means clustering algorithm to cluster all medication action recognition detection frames. The clustering results are sorted by time information, and the action category and index number of each cluster center point are extracted. The sorted index numbers are used to determine whether the patient has completed the medication action sequence (0->1->2->3->4). If the index numbers are in order and there are no missing items, the patient is considered to have taken the medication; otherwise, the patient is considered to have not taken the medication.

[0087] The process of judging the medication posture of hospitalized patients is divided into two stages: IndexPose-RTDETR algorithm model training and IndexPose-RTDETR algorithm deployment and use.

[0088] IndexPose-RTDETR algorithm model training phase:

[0089] Step 1: First, a video of a hospitalized patient picking up medication is captured by the depth binocular 3D camera 10. A video of the patient raising their hand, placing the medication in their mouth, and swallowing the medication is captured by the facial recognition camera 7. The video is then processed by a high-performance microcomputer 20 into a fused video segment, which serves as a medication information video. Video frames are extracted from all medication information videos. The medication information video includes the patient picking up medication, raising their hand, placing the medication in their mouth, and swallowing the medication.

[0090] In step 2, the video frames were randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1. These sets were annotated with action categories categorized as picking up medication, raising hands, opening mouth, placing medication in mouth, and swallowing with mouth closed. The corresponding action categories were assigned an index number set {0: picking up medication; 1: raising hands; 2: opening mouth; 3: placing medication in mouth; 4: swallowing with mouth closed}. The training set was trained using the IndexPose-RTDETR algorithm, and the validation set was used to determine convergence. Finally, the test set was used to obtain the average precision and recall of the medication posture recognition model.

[0091] The backbone network of the IndexPose-RTDETR algorithm network model is ConvNeXt, which optimizes feature learning through a fully convolutional masked autoencoder and a global response normalization layer, reducing dependence on large-scale labeled data while enhancing feature diversity.

[0092] The BIFI substructure of the hybrid encoder introduces the Biformer attention mechanism to optimize and enhance the motion features. The Biformer attention mechanism first divides and projects the regions. Figure X ∈R H×W×C Divided into S 2 Regions, through linear projection to obtain query Q, key K and value V; then perform regional affinity calculation, through region-level query Q r and key K r Matrix multiplication of constructs affinity graph A r , and filter the top k regions, the formula is as follows:

[0093]

[0094] I r =topkIndex(A r )

[0095] The Biformer attention mechanism finally performs token attention and collects the key-value pairs K of the routing area. g and V g , performing dense matrix multiplication, where LCE(·) is the local context enhancement module, implemented using depthwise convolution, and the formula is as follows:

[0096] O=Attention(Q,K g ,V g )+LCE(V)

[0097] Figure 9The EUCB convolutional block network structure is introduced into the CCFM substructure of the hybrid encoder. This block performs cross-scale feature fusion on multi-scale action features to generate action image feature sequences containing rich semantic information. The EUCB convolutional block introduces a Shift-Channel-Mix operation, which enhances the expressive power of the medication action feature map through channel segmentation and spatial shifting. The mathematical formula for the EUCB convolutional block is as follows:

[0098]

[0099] In step 3, the posture judgment part clusters the stored medication action recognition detection frames, takes the action category, index number, and detection frame coordinates as feature vectors, uses the K-means clustering algorithm to cluster, extracts the action category and index number of each cluster center point, and sorts them by time information.

[0100] Step 4: Based on the index numbers of each cluster center, sort them by time to see if they form the order 0->1->2->3->4. If so, the medication posture recognition system determines that the patient has taken the medication. If the order is inconsistent, or if any index number is missing, the patient is determined not to have taken the medication.

[0101] Through the above steps, the detection boxes identified by the IndexPose-RTDETR object detection algorithm can be clustered, and the clustering results can be sorted by time information, so as to analyze the distribution and changes of the detection boxes in the time series.

[0102] IndexPose-RTDETR algorithm deployment and use phase:

[0103] The trained medication gesture recognition model is deployed to a high-performance microcomputer 20. During medication delivery, the deep binocular 3D camera 10 captures a video of the inpatient picking up the medication, while the facial recognition camera 7 captures a video of the inpatient raising their hand, placing the medication in their mouth, and swallowing it. The two segmented videos are fused together and input into the medication gesture recognition system for analysis. The output of the medication gesture recognition model is the inpatient's medication administration result.

[0104] The medication information video of the hospitalized patient is stored in real time in the memory 23 of the high-performance microcomputer 20, and the medication delivery result is recorded in the medication management system.

[0105] The annotations involved in the formulas for the above IndexPose-RTDETR algorithm model training and IndexPose-RTDETR algorithm deployment are as follows:

[0106] A r : represents the affinity graph, which is a graph of size S2 ×S 2 A matrix is used to describe the affinity relationship between different regions.

[0107] R: represents the set of real numbers, which is used to define the value range of the affinity graph in the real number space.

[0108] S: represents the square root of the number of regions into which the image is divided, that is, the image is divided into S 2 area.

[0109] I r : Represents the index set of the top k regions filtered out, which is used to determine which regions have higher importance or relevance in subsequent processing.

[0110] topkIndex: is an operation used to find the indexes of the top k regions with the largest affinity values from the affinity graph Ar to determine the location of the region of focus.

[0111] O: represents the final output of the attention mechanism, which combines the query, key-value pairs, and local context enhancement information for further processing or as a feature passed to the next layer of the network.

[0112] Attention: represents the attention calculation function, by querying Q, key-value pair K g and V g Perform dense matrix multiplication to obtain feature representation based on attention weights.

[0113] Q: Query, which is the vector used to match the key in the attention mechanism to determine the focus. Here, it is the query vector obtained by linear projection.

[0114] K g : represents the key of the collected routing area, which is used to perform matching calculations with the query Q to determine the degree of association between the features of different areas.

[0115] V g : Indicates the value of the collected routing area (Value), which contains the feature information of the area represented by the corresponding key. After the attention calculation, it will be weighted and summed according to the weight.

[0116] LCE: Local context enhancement module, implemented using deep convolution, is used to perform local feature enhancement on the original value V to add more context information to the output result.

[0117] V: represents the original value vector. After being processed by the local context enhancement module LCE, its result is added to the result of the attention calculation to form the final output O.

[0118] x: represents the input feature map, which is the input of the EUCB convolution block.

[0119] Conv 1×1 : Represents a 1×1 convolution operation, which is used to adjust the number of channels of the feature map, thereby reducing or increasing the dimensionality.

[0120] max(0,·): Represents the rectified linear unit (ReLU) function, which is used to introduce nonlinearity. When the input is greater than 0, it outputs directly, otherwise it outputs 0.

[0121] γ: represents the scaling factor, which is used to scale the normalized feature map and adjust the scale of the feature.

[0122] Represents a 3×3 depth convolution operation, which is used to extract features between channels and reduce the amount of computation.

[0123] Interpolation(x,2): Indicates that the input feature map x is interpolated and the size of the feature map is doubled for higher resolution feature processing.

[0124] μ: represents the mean, which is used for centralization in the normalization process to make the mean of the feature map 0.

[0125] σ 2 : represents the variance, which is used for scale adjustment during the normalization process to make the standard deviation of the feature map equal to 1.

[0126] ∈: represents a very small constant used to prevent division by zero and ensure the stability of numerical calculations.

[0127] β: represents the offset, which is used to translate the normalized feature map and adjust the feature bias.

[0128] The medication drawer recognition system is used to determine whether an inpatient has taken medication from the medication drawer 11. This system utilizes a trained medication drawer recognition model. Based on images captured by the medication storage device's deep binocular 3D camera 10, it determines whether the drawer 11 is empty and, therefore, whether medication has been removed by the inpatient. Upon detecting that the inpatient has taken medication, the high-performance microcomputer 20 activates the cylinder actuator to close the medication drawer 11.

[0129] The medication posture recognition model is based on the RTDETRV3 network model to identify whether the medicine drawer is empty.

[0130] The image analysis and processing system processes the raw image data transmitted by the image sensor 8 in real time to optimize the image quality and provide support for subsequent image analysis tasks: through noise reduction processing, random noise in the image is reduced to improve the clarity and visual effect of the image; through color correction, the color deviation of the image is corrected to make it closer to the real scene or meet specific color standards; through sharpening, the edges and details of the image are enhanced to make the image look clearer; through geometric correction, the geometric distortion of the image, such as perspective distortion or lens distortion, is corrected; through feature extraction, useful features are extracted from the image for further analysis or recognition.

[0131] Specifically, the image sensor in this patent is used to convert: (1) facial images of medical staff and hospitalized patients captured by the facial recognition camera, (2) images of patients taking medication captured by the facial recognition camera and the deep binocular 3D camera, (3) images of the presence of medication in the medication drawer captured by the deep binocular 3D camera, and (4) the patient identification code captured by the facial recognition camera into electrical signals, and then transmit the electrical signals to the image analysis and processing system for image feature extraction. The image analysis and processing system in the high-performance microcomputer 20 then feeds the images of the medication taking action into a gesture recognition algorithm model to analyze and determine whether the patient has taken the medication, and feeds the images of the interior of the medication drawer into the medication drawer recognition system to determine whether medication is still present in the drawer, and further combines the images to determine whether the patient has taken the medication.

[0132] The detection system functions are divided into battery detection and hydraulic gas tank detection.

[0133] Battery monitoring measures the robot's battery level in real time, estimating the power required for the planned route and ensuring sufficient power for medication delivery. If the battery is low, the robot will be left at the nurse's station to recharge. During off-hours, the robot automatically returns to the charging station for wireless charging, ensuring 24-hour uninterrupted delivery. The wireless charging device 30 is located directly below the center of the rear middle section of the robot's main structure 1.

[0134] The hydraulic gas tank detection is used to detect the pressure level of the hydraulic gas tank 31 and the gas content in the hydraulic gas tank 31 to ensure that the pressure level of the hydraulic gas tank 31 is within the set value range. If it is lower or higher than the set value, the hydraulic gas tank 31 needs to be replaced immediately; when the gas content in the hydraulic gas tank 31 is low, the high-performance micro host 20 will issue a prompt message to prompt medical staff to fill the hydraulic gas tank 31 with gas or replace the hydraulic gas tank 31 in time for continued use.

[0135] The electronic control system includes a power management system and a motion control system. The power management system is responsible for providing a stable and reliable power supply to the robot. The motion control system includes a sensor control program and a motion control program. The sensor control program uses sensors to obtain the robot's current status, such as position, posture, and speed. The motion control program, based on the desired trajectory and position information, uses motion control algorithms to calculate the robot's trajectory and speed.

[0136] The traveling device 25 includes two pairs of universal wheels 26 at the front and rear, a laser radar 19, and a set of traveling cameras 27. The two pairs of universal wheels 26 are mounted on the bottom of the traveling device 25. The laser radar 19 is located in the center above the front of the traveling device 25. A set of traveling cameras 27 are located on either side of the laser radar 19. The robot uses its built-in sensors (such as the laser radar 19 and traveling cameras 27) to scan the environment and construct or update indoor maps in real time. These maps not only include the location information of static obstacles (such as walls and beds), but also dynamically identify temporary obstacles (such as pedestrians and other mobile devices). Based on the constructed maps, the robot uses the A* algorithm, Dijkstra's algorithm, or other heuristic search algorithms to calculate the shortest or optimal path from the starting point to the destination. These algorithms consider various factors, such as path length, number of turns, and potential obstacles, to optimize travel efficiency. During travel, the robot continuously monitors environmental changes and dynamically adjusts its path as needed. Upon detecting an obstacle, the robot immediately initiates an obstacle avoidance program, selecting an appropriate strategy based on the type and location of the obstacle. For example, the robot will circumvent static obstacles, while it will predict the trajectory of moving obstacles (such as pedestrians) and avoid them in advance. In extreme cases, if the robot determines a collision is imminent, it will immediately activate the emergency braking mechanism, stopping the vehicle or quickly adjusting its direction to ensure safety.

[0137] The energy supply unit 28 includes four sets of 12V low-temperature lithium batteries 29, a wireless charging device 30, a hydraulic air tank 31, and a fixing bracket 32. The four sets of 12V low-temperature lithium batteries 29 are fixed to the bottom of the energy supply unit 28. A wireless charging device 30 is also installed behind the energy supply unit 28. The wireless charging device 30 is located in the center of the lower rear side of the energy supply unit 28 and can charge the 12V low-temperature lithium batteries 29. The air source is a hydraulic air tank 31, which is fixed to the energy supply unit 28 by a set of fixing brackets 32 and is used to store compressed air. The compressed air can be used as a power source to drive the cylinder expansion joint.

[0138] The present invention provides an application method of an intelligent medicine delivery and medication recognition robot for an inpatient department, comprising the following steps:

[0139] Medical staff log in after facial recognition verification through image acquisition devices. The medication management system is connected to the hospital inpatient management system through the data connection module, and the medication management system updates the inpatient patient information in real time.

[0140] According to the medication management system, medical staff use the high-definition touch screen 3 to batch open or individually open the medicine drawers 11, place the medicines according to the inpatient information and medication information, and finally close the medicine drawers 11.

[0141] The robot delivers medication three times daily, ward by ward and bed by bed according to a planned route. While the robot is delivering medication, its facial recognition camera 7 identifies the ward and bed numbers, transmits the captured digital image to a high-performance microcomputer 20, and then loads the corresponding inpatient information and displays it on a high-definition touchscreen display 3.

[0142] When the robot approaches a hospitalized patient, a tone prompting "After the drawer is opened, please face the screen while taking your medication!" plays through speaker 5, and the robot automatically adjusts its position until it is facing the patient. Facial recognition camera 7 then identifies the patient's facial information and transmits the captured facial image to high-performance microcomputer 20. Using a facial recognition algorithm, the patient's facial information is compared and verified with the facial feature data in the patient's information. The patient then scans the patient identification code worn on their wrist through facial recognition camera 7, which is then compared and verified with the patient identification code data in the patient's information. Only after both the patient's facial information and the patient identification code have been verified will medication drawer 11 open, ensuring that the wrong medication is not delivered to the patient.

[0143] The screen display device displays the medication information of the hospitalized patients, and the speaker 5 plays the name of the medicine taken, the number of times of taking the medicine per day, the dosage of each medicine, and precautions.

[0144] The robot's image acquisition device captures the patient's motion signature during medication administration, including picking up medication, raising their hand, placing the medication in their mouth, and swallowing. The medication gesture recognition system's medication gesture recognition model identifies whether the patient has taken their medication. Simultaneously, the medication drawer recognition system identifies whether the medication drawer 11 is empty, comprehensively determining that the patient has taken their medication. The medication management system records medication administration time. The memory 23 of the high-performance microcomputer 20 stores video footage of the patient's medication administration process. The medication administration results are recorded in the medication management system.

[0145] After the medication is taken, the cylinder retractor closes the medication drawer 11 of the current inpatient.

[0146] After the robot completes its first round of medicine delivery, if the facial recognition camera 7 does not identify an inpatient in the corresponding ward and bed, it will perform a second round of medicine delivery 15 minutes later. After the second round of medicine delivery, the robot returns to the nurse station and uploads the delivery results in batches to the hospital's inpatient management system.

[0147] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. An intelligent medicine delivery and medication recognition robot for inpatient departments, characterized by: It includes a main structure (1), a high-performance microcomputer (20), an energy supply device (28), and a traveling device (25); The main structure (1) is provided with a screen display device, an image acquisition device, and a medicine storage device; The screen display device is used for medical staff's operations and information interaction with hospitalized patients; Image acquisition devices are used to verify the identity of medical staff and determine the medication status of hospitalized patients; The medicine storage device comprises a medicine loading drawer (11) and a cylinder retractor, wherein a plurality of medicine loading drawers (11) are arranged in a matrix on a main structure (1); The cylinder telescopic device comprises a cylinder (14), a telescopic device (15) and a piston (16); the air pipe interface (18) is connected to the energy supply device (28); the piston (16), the cylinder (14) and the telescopic device (15) are connected in sequence; the gas in the piston (16) is transmitted to the cylinder (14); the cylinder (14) inflates the telescopic device (15), pushes the telescopic device (15) forward, drives the medicine loading drawer (11) to push out the main structure (1); one cylinder telescopic device corresponds to one medicine loading drawer (11); A high-performance micro host (20) connects various systems and modules in the robot and is used for data processing and control; The energy supply device (28) includes a power source and an air source. The power source supplies power to various systems and modules of the robot, and the air source provides driving power to the cylinder telescopic device.

2. The intelligent medicine delivery and medication recognition robot for inpatient departments according to claim 1, characterized in that: The screen display device comprises a high-definition touch screen (3), a loudspeaker (5), and a microphone (6), wherein the high-definition touch screen (3) is used for medical staff's operation work, the loudspeaker (5) is used for playing prompt information, and the microphone (6) is used for real-time communication between medical staff and hospitalized patients.

3. The intelligent medicine delivery and medication recognition robot for inpatient department according to claim 2 is characterized in that The image acquisition device comprises a face recognition camera (7), an image sensor (8), and a deep binocular 3D camera (10); the face recognition camera (7) is used to capture the action of hospitalized patients taking medicine; the deep binocular 3D camera (10) is used to capture the storage status of medicines in a medicine drawer (11); and the image sensor (8) is connected to the face recognition camera (7) and the deep binocular 3D camera (10) via a signal line.

4. The intelligent medicine delivery and medication recognition robot for inpatient departments according to claim 3, characterized in that: The traveling device (25) is the walking component of the robot, which is controlled by a high-performance micro host (20) and powered by a power supply. It includes a laser radar (19), a traveling camera (27), and two pairs of universal wheels (26) at the front and rear of the bottom of the robot. The robot uses the laser radar (19) and the traveling camera (27) to scan the environment and build or update the indoor map in real time.

5. The intelligent medicine delivery and medication recognition robot for inpatient departments according to claim 4, characterized in that: Ultraviolet light strips (12) are provided on both sides of the medicine loading drawer (11).

6. The method for using the intelligent medicine delivery and medication recognition robot in an inpatient department according to claim 5 is characterized in that: The steps include: Step 1: Medical staff log in after facial recognition verification through the image acquisition device. The medication management system is connected to the hospital inpatient management system through the data connection module, and the medication management system updates the inpatient information in real time; Step 2: The medical staff opens the medicine drawers (11) in batches or individually through the high-definition touch screen (3) according to the medication management system, places the medicines according to the inpatient information and medication information, and finally closes the medicine drawers (11); Step 3: The robot delivers medicine three times a day, ward by ward and bed by bed according to the planned route; When the robot is delivering medicine, the face recognition camera (7) identifies the ward number and bed number of the ward, transmits the captured digital image to the high-performance micro host (20), and then loads the corresponding inpatient information and displays it on the high-definition touch screen (3); Step 4: When the robot moves to the side of the inpatient, the speaker (5) plays a medication reminder sound, and the robot automatically adjusts its position until it is facing the inpatient; then the face recognition camera (7) recognizes the inpatient's facial information, transmits the collected inpatient facial image to the high-performance micro host (20), and compares and verifies the inpatient's facial information with the facial feature data of the inpatient information through the face recognition algorithm. The inpatient scans the patient identification code worn on the wrist on the face recognition camera (7), and compares and verifies it with the patient identification code data of the inpatient information; after the inpatient's facial information and the patient identification code are simultaneously verified, the medicine drawer (11) opens; Step 5: The screen display device displays the inpatient's medication information, and the speaker (5) plays the name of the medication, the number of times the medication is taken per day, the dosage of each medication, and precautions; Step 6: The robot's image acquisition device obtains the motion features of the inpatient's medication-taking process, including picking up the medication, raising the hand, placing the medication in the mouth, and swallowing; the medication posture recognition model of the medication posture recognition system recognizes whether the inpatient has taken the medication, and at the same time, the medication drawer recognition system recognizes that the medication drawer (11) is empty, and comprehensively judges that the inpatient has taken the medication; The medication management system records the medication taking time, the high-performance micro host (20) saves the medication information video of the inpatient medication process, and the medication results are recorded in the medication management system; Step 7: After taking the medicine, the cylinder retractor closes the medicine drawer (11) of the current inpatient; Step 8: After the robot completes the first round of medicine delivery, if the facial recognition camera (7) does not identify the inpatient in the corresponding ward and bed, it will perform the second round of medicine delivery after an interval of 15 minutes; after the second round of medicine delivery, it will return to the nurse station and upload the delivery results in batches to the hospital inpatient management system.

7. The method for using the intelligent medicine delivery and medication recognition robot for inpatient departments according to claim 6 is characterized by: In step 6, the robot's image acquisition device obtains the motion characteristics of the hospitalized patient during the medication-taking process, including two stages: IndexPose-RTDETR algorithm model training and IndexPose-RTDETR algorithm deployment and use; IndexPose-RTDETR algorithm model training segment: Step 1: First, collect videos of hospitalized patients picking up medicines shot by a deep binocular 3D camera (10), and videos of hospitalized patients raising their hands, placing them in their mouths, and swallowing them shot by a face recognition camera (7), and process them into a whole fusion video through a high-performance micro host (20) as a medication information video; extract video frame images from all medication information videos; Step 2: Divide the video frame images into a training set, a validation set, and a test set; annotate the training set, validation set, and test set with action categories, and set the index number set corresponding to the action category; train the training set using the IndexPose-RTDETR algorithm, and use the validation set to determine whether the IndexPose-RTDETR algorithm training has converged; finally, obtain the average precision and recall rate of the test medication posture recognition model using the test set; The backbone network of the IndexPose-RTDETR algorithm network model is ConvNeXt, which optimizes feature learning through a fully convolutional masked autoencoder and a global response normalization layer; The BIFI substructure of the hybrid encoder introduces the Biformer attention mechanism; the Biformer attention mechanism first divides and projects the region, and the image X∈R H×W×C Divided into S 2 Regions are linearly projected to obtain query Q, key K and value V; Then perform region affinity calculation, through region-level query Q r and key K r Matrix multiplication of constructs affinity graph A r , and filter the top k regions, the formula is as follows: I r =topkIndex(A r ) The Biformer attention mechanism finally performs token attention and collects the key-value pairs K of the routing area. g and V g , performing dense matrix multiplication, where LCE(·) is the local context enhancement module, implemented using depthwise convolution, and the formula is as follows: O=Attention(Q,K g ,V g )+LCE(V) The CCFM substructure of the hybrid encoder introduces the EUCB convolution block to perform cross-scale feature fusion on multi-scale action features to generate an action image feature sequence containing rich semantic information. The EUCB convolution block introduces the Shift-Channel-Mix operation, which enhances the expressiveness of the medication action feature map through channel segmentation and spatial offset. The mathematical formula of the EUCB convolution block is as follows: Step 3: The posture judgment part clusters the stored medication action recognition detection frames, uses the action category, index number, and detection frame coordinates as feature vectors, clusters them using the K-means clustering algorithm, extracts the action category and index number of each cluster center point, and sorts them by time information; Step 4: Based on the index numbers of the cluster centers, after sorting by time information, determine whether they form the order of 0->1->2->3->4. If so, the medication posture recognition system determines that the inpatient has taken the medication. If the order is inconsistent, or any index number is missing, the inpatient is determined to have not taken the medication. IndexPose-RTDETR algorithm deployment and use phase: The trained medication posture recognition model is deployed to a high-performance micro-host (20); when delivering medicine, a deep binocular 3D camera (10) captures a video of the hospitalized patient picking up the medicine, and a face recognition camera (7) captures a video of the hospitalized patient raising his hand, placing the medicine in his mouth, and swallowing the medicine. The two segmented videos are fused as a whole and input into the medication posture recognition system for analysis. The output result of the medication posture recognition model is the medication result of the hospitalized patient. The medication information video of the hospitalized patient is stored in real time in the memory (23) of the high-performance microcomputer (20), and the medication delivery result is recorded in the medication management system.

8. The method for using the intelligent medicine delivery and medication recognition robot for inpatient departments according to claim 6 is characterized by: The main modules of the medication management system include data connection module, system login module, medicine storage and access module, ultraviolet light strip control module, inpatient medication management module, and medication management database.

Images