Pharmacy intelligent robot and system thereof

Abstract

The invention discloses a pharmacy intelligent robot and a system thereof, relates to the technical field of robots, and is technically characterized by comprising a voice interaction module, an image recognition module, a navigation module and a mechanical arm control module to realize automatic medicine management and distribution. The voice interaction module utilizes an AI model to improve the voice recognition efficiency and accuracy, the image recognition module adopts PaddleOCR to perform medicine name recognition, a Baidu oneself-speaking AI model is used to analyze prescription information, the navigation module adopts an SLAM technology to realize instant positioning and map construction, and the mechanical arm control module controls a mechanical arm to grab and place medicines. The system is based on a robot operating system ROS, and the operation method comprises the steps of receiving user voice, recognizing medicine, navigating to the medicine position and grabbing and placing the medicine. By integrating various technologies, the efficiency and safety of medicine sales are improved, and meanwhile, the customer experience is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and in particular to an intelligent robot for pharmacies and a system thereof. Background Art

[0002] With the growth of the aging population and the expansion of demand for education and medical care, the service robot industry has ushered in new development opportunities and has become a highly anticipated field in the application of the robot market. Especially in the fields of education and public services, the demand for service robots has become the main driving force for their development. Smart pharmacies, as pharmacies that use artificial intelligence, big data, and the Internet of Things to automate the entire process of drug sales, storage, distribution, and information management, aim to improve the efficiency of drug sales, reduce costs, and improve quality and safety.

[0003] In China, although the development of smart pharmacies is still in its infancy, large pharmacy chains such as China Resources Pharmaceuticals and Dasanlin have begun to try to build smart pharmacies and automate processes by introducing smart equipment and management systems. Although smart pharmacies have shown positive effects worldwide, their development in China still faces some challenges. The technical maturity and penetration rate are not yet high. With the rapid increase in the number of drug types, smart pharmacies need more efficient information management and automation services to meet the challenges. In addition, the initial investment and maintenance costs may be high, and it is necessary to balance costs and benefits. Smart pharmacies need to integrate and optimize multiple technologies (such as AI, big data, and the Internet of Things) to achieve more efficient and accurate services. Although the emergence of smart pharmacy robots has improved the efficiency and safety of drug sales, it still needs further development and improvement in terms of technology integration, cost control, and market penetration.

[0004] Therefore, the present invention provides a pharmacy intelligent robot and a system thereof. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and to provide a drugstore intelligent robot and its system to solve the problems of low speech recognition efficiency and accuracy, inaccurate medication, and drugstore environment navigation in the prior art.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows: a drugstore intelligent robot system, comprising the following modules:

[0007] The voice interaction module is used to convert the user's input voice into text content and use the AI ​​model to improve the efficiency and accuracy of voice recognition;

[0008] Image recognition module, used to identify text on drug packaging to achieve accurate drug dispensing;

[0009] The navigation module includes global mapping, self-positioning, path planning, motion control, and environmental perception functions, which are used for the robot to navigate in the pharmacy;

[0010] The robotic arm control module is used to control the robotic arm to grab and place medicines.

[0011] Furthermore, the voice interaction module includes:

[0012] Speech recognition technology, which converts speech into text content;

[0013] AI models to improve speech recognition efficiency and accuracy;

[0014] The VITS speech generation model converts the text generated by the AI ​​model into speech to interact with users.

[0015] Furthermore, the image recognition module adopts the PaddleOCR module, which uses the Baidu Wenxin Yiyan AI model, and has natural language processing capabilities to analyze the patient's prescription information and remind the patient of the dosage and precautions of different drugs.

[0016] Furthermore, the navigation module uses SLAM technology to perform real-time positioning and map construction.

[0017] Furthermore, the robotic arm control module comprises:

[0018] Gripper control instructions, used to control the opening and closing of the gripper;

[0019] Posture control instructions, used to control the robot arm to reach specific spatial coordinates;

[0020] Servo control instructions are used to control each servo of the robotic arm.

[0021] Another object of the present invention is to provide a pharmacy intelligent robot, including the above-mentioned pharmacy intelligent robot system.

[0022] Another object of the present invention is to provide an operating method of a drugstore intelligent robot, the operating method comprising the following steps:

[0023] S1: receiving user instructions through the voice interaction module;

[0024] S2: Identify drugs using image recognition module;

[0025] S3: Use the navigation module to navigate the robot to the location of the medicine;

[0026] S4: Grab and place the medicine through the robotic arm control module.

[0027] Compared with the prior art, the beneficial effects of this solution are as follows: the intelligent robot for drug stores of the present invention can realize automated drug management and distribution by integrating voice interaction module, image recognition module, navigation module and mechanical arm control module. The voice interaction module enables the robot to understand the voice commands of customers and provide a more humanized service experience. The addition of the image recognition module enables the robot to accurately recognize the text on the drug packaging, thereby achieving accurate drug dispensing and reducing human errors; the SLAM technology of the navigation module of the present invention enables the robot to realize autonomous positioning and path planning in a complex drugstore environment, improving the efficiency and accuracy of drug distribution, and the mechanical arm control module gives the robot the ability to grab and place drugs, realizing the automated process of drugs from the shelf to the hands of customers.

[0028] The intelligent robot and system for drug stores of the present invention not only improve the efficiency of drug sales and reduce the cost of drug sales, but also improve the quality and safety of drug sales through intelligent technology. At the same time, the system enhances customer experience, reduces operating costs, improves the image of drug stores and intelligent health management. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a diagram of the architecture of the intelligent robot for pharmacies in an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a process of a user having a voice conversation with a drugstore intelligent robot in an embodiment of the present invention;

[0031] Figure 3 Schematic diagram of a map-building and navigation module of a drugstore intelligent robot in an embodiment of the present invention;

[0032] Figure 4 Schematic diagrams of two methods of hand-eye calibration performed by a robotic arm in an embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of the hand-eye calibration process of the robotic arm in an embodiment of the present invention;

[0034] Figure 6 This is a picture of the robot in the embodiment of the present invention navigating to the grasping point and performing text recognition;

[0035] Figure 7 is a flow chart of the robot identifying the target medicine box in an embodiment of the present invention;

[0036] Figure 8 Schematic diagram of the process of the robot grasping the target drug in the fixed grasping mode in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below in conjunction with the embodiments.

[0039] Embodiment 1: A drugstore intelligent robot and its system, as shown in the attached Figure 1 As shown, it includes: a voice interaction module, which is used to convert the voice input by the user into text content and use the AI ​​model to improve the efficiency and accuracy of voice recognition;

[0040] The intelligent robot for drugstores of the present invention uses speech recognition technology to convert the speech input by the user into text content, and uses an AI model to improve the efficiency and accuracy of speech recognition. And with the help of a method of unifying audio data, the errors caused by the speaking habits of different people are reduced to improve the agility of the system. Multiple rounds of dialogue are conducted with the AI ​​model to generate a natural language response model, and the response process is completed in cooperation with the speech recognition model. Then the VITS speech generation model is used to convert the text generated by the AI ​​model into speech to interact with the patient.

[0041] Image recognition technology is used to accurately dispense medicines by identifying the text on the drug packaging. At the same time, the patient's prescription information can be analyzed based on the powerful AI model to remind the patient of the dosage and precautions of different drugs. At the same time, the Baidu Wenxin Yiyan AI model is used, which has powerful natural language processing capabilities, can communicate like humans, and make intelligent replies based on the content of the conversation. Calling the AI ​​model interface in the pharmacy smart robot can intelligently and accurately push medicines to patients, and at the same time provide dietary recommendations and precautions in life. Figure 2 The figure shows the process of a user having a voice conversation with a robot.

[0042] The mapping and navigation module includes global mapping, self-positioning, path planning, motion control and environmental perception functions, which are used for robot navigation in pharmacies. Figure 3As shown; the navigation module uses SLAM technology for real-time positioning and map construction. SLAM (real-time positioning and mapping technology) plays an important role in robot positioning and navigation in indoor environments and is widely used in various types of robots, including sweeping robots. Among many SLAM algorithms, this embodiment uses Gmapping, which is a laser-based SLAM algorithm. Due to its wide application and integration in ROS (Robot Operating System), it has become one of the most used SLAM algorithms in the field of mobile robots. The specific implementation process is as follows:

[0043] First, place the robot on the floor in the area where you want to build a map. Next, log in to the robot system through the remote software and open a terminal to start the robot's basic functions, the lidar SLAM algorithm, and the RViz visualization tool. In RViz, you can use the mouse wheel to drag, zoom in and out, and flip the map to adjust the map window to the appropriate position. Then start keyboard control and use commands to control the robot's movement to complete the map construction. Finally, execute the script to save the map to save the completed map for subsequent use.

[0044] At present, the robot arm used in the present invention is the xArm robot arm of Huaner. The robot arm control of the present invention is controlled in Json format, which is easy for users to understand and adopts the value:key method. There are three control methods:

[0045] 1. Gripper gripper control. Since the gripper does not participate in the posture solution in the robot arm, a separate instruction is used here for control. The control instruction is as follows: {"abotxArmCmd":2,"Gripper":100,"time":1000}, abotxArmCmd is the command set, 2 is the gripper control, Gripper represents the gripper control opening and closing size, the gripper control range of the xArm robot arm is 0-700 values, if the input value is 0, it means the gripper is closed, 700 means it is open, time represents the speed, and the following 1000 is milliseconds, that is, it takes 1000ms to execute this action.

[0046] 2. Posture control mode. Posture control mode is to let the robot run to the x, y, z spatial coordinates of the space, that is, give a spatial point, and the robot automatically calculates and reaches the relevant spatial point. The instructions are as follows: {"abotxArmCmd":1,"Pose":{"x":100,"y":100,"z":100,"rotary":90},"time":1000}, abotxArmCmd is the command set, 1 is the posture control command, Pose is the posture instruction set, the base x, y, z represents the robot relative to the center origin of the robot as the coordinate, normalized to the relevant spatial coordinates, rotary represents the angle of rotation of the gripper, the value range is 0-270 degrees, time represents the speed, and the following 1000 is milliseconds, that is, it takes 1000ms to execute this action. If the x, y, z given by the user exceeds the range of the robot's normalization, the driver board buzzer will make a beeping sound to tell the user that this value exceeds the range of motion of the robot.

[0047] 3. Control each servo. Our gripper has been controlled separately in the first line. Here we only control other related servos. The value range of each servo is 0-1000, and the corresponding mapping angle is 0-270 degrees. The instructions are as follows: {"abotxArmCmd":3,"Trajectory":{"id2":500,"id3":500,"id4":500,"id5":500,"id6":500},"time":1000}, abotxArmCmd is the command set, 3 is the servo trajectory control command, Trajectory is the trajectory set of the servo, id2-id6 is the control step number of each servo, here the id sequence is the gripper id1 to the bottom seat servo id6, time represents the speed, and the following 1000 is milliseconds, that is, it takes 1000ms to execute this action.

[0048] In this embodiment, the present invention realizes the text recognition and grasping process of medicines by a robot. First, the robot's servo needs to be set so that its camera faces downwards in order to capture the medicine box, which is achieved by starting the chassis connection. After completion, the servo is adjusted to an appropriate position to ensure that the camera can obtain depth data, but it cannot be set to be completely vertical, otherwise the camera cannot obtain depth information.

[0049] First, hand-eye calibration is required. The Eye To Hand method is used here. The camera is not installed on the robot arm and does not move with the movement of the robot arm. Figure 4 shown.

[0050] The specific implementation steps for correcting the relative position relationship between the camera and the robotic arm are as follows:

[0051] Set the calibration mode: First, set the system to calibration mode by modifying the relevant parameters in the startup file to make it enter the calibration mode.

[0052] Place the medicine box and start the recognition test: Place the medicine box in the recognition area of ​​the camera and start the medicine box text recognition test program. This step is crucial because it is necessary to ensure that the camera can normally recognize the text on the medicine box and obtain the correct medicine box information.

[0053] Start the calibration of the robot and camera: After confirming that the medicine box is correctly identified, start the calibration program of the robot and camera and enter the calibration mode. This indicates that the system is ready for hand-eye calibration.

[0054] Start dynamic parameter adjustment: By running the dynamic parameter adjustment tool, the position of the robot arm is dynamically adjusted so that it can accurately grab the medicine box. At this time, the real viewing angle of the robot arm is as shown in the attached figure. Figure 5 For specific models, in order to prevent the servo from over-rotating, the position needs to be adjusted to a suitable value to ensure that the robotic arm can grab the medicine box downwards, and fine-tune to obtain the best grabbing position.

[0055] Record coordinate values: During the calibration process, you need to record the coordinate values ​​of the target object relative to the camera and the coordinate values ​​of the position where the robot arm needs to move to ensure that the robot arm can accurately grasp the medicine box.

[0056] Fixed crawl:

[0057] 1. Set up fixed grasping mode: First, adjust the system configuration to enable fixed grasping mode. This involves modifying the startup file, setting the calibration mode to disabled, while the fixed grasping mode remains enabled. At the same time, the data obtained from the previous hand-eye calibration is sequentially input into the calibration data array so that the system can perform accurate grasping operations based on these coordinates.

[0058] 2. Fixed grab test: When performing a fixed grab test, place the medicine box in the recognition area, and then start the medicine box text recognition program and the fixed grab program. Observe whether the robot arm can grab the medicine box normally. If the grab operation is abnormal, you need to re-calibrate the hand-eye or check whether the input coordinate value is incorrect.

[0059] Mobile crawling:

[0060] 1. Set the mobile grabbing mode: In order to achieve mobile grabbing, the system needs to be configured in mobile grabbing mode. This includes modifying the startup file, setting the fixed grabbing mode to disabled, and defining the coordinate order of each grabbing point and placement point.

[0061] 2. Environment map construction: Before actually using the mobile grabbing function, you need to build an environment map first. Place the vehicle at the starting position, start the underlying connection, and then build the map. After the map is built, fill in the corresponding coordinates in the parameter file according to the positional relationship between the starting point and the grabbing point and placement point.

[0062] 3. Start the navigation and grasping task: After the environment map is built, start the navigation program so that the vehicle can navigate to the predetermined grasping point. At the same time, start the target detection program to identify the grasping target object, and start the grasping task program so that the robot arm can grasp the target object and navigate to the placement point to put it down. This series of operations constitutes a complete mobile grasping process, including navigation to the grasping point, identification of the target object, grasping by the robot arm, navigation to the placement point, and putting down the target object.

[0063] The above is the process of the robot identifying and grabbing the medicine box. The specific technical details are as follows.

[0064] 1. Introduction of text recognition technology: Introduce text recognition technology and use PaddleOCR to recognize drug names. The robot can recognize text information on drug packaging from the images captured by the camera. Specifically, first determine all text areas, and sort the text areas in descending order based on the characteristics of the larger drug name area, so that other irrelevant text can be excluded and the target medicine box can be located faster.

[0065] 2. Receiving and processing voice commands: When the user has a voice conversation with the robot and asks for a certain medicine, for example, "Please get me a medicine for xxx", the robot's voice interaction module will convert the user's voice command into text and store the target medicine name in the global variable of ROS, as shown in the attached figure. Figure 6 Shown is a robot identifying the name of a drug.

[0066] 3. Work of the text recognition node: The text recognition node monitors the target drug name in the ROS global variable and searches for matching drug text information in the image of the robot camera. Once the target drug is recognized, the node triggers the grab command.

[0067] 4. Execute the grabbing action: After receiving the grabbing instruction, the robot will execute the corresponding grabbing action. If it is a fixed grabbing mode, the robot will directly move the mechanical arm to the preset coordinate position for grabbing. If it is a mobile grabbing mode, it is necessary to add a navigation process. The robot will first navigate to the location of the target drug and then perform the grabbing, as shown in the attached figure. Figure 7 The robot is shown grabbing the target drug.

[0068] 5. Navigation and placement: In mobile grabbing mode, after the robot grabs the medicine, it will use the navigation function again to navigate to the designated placement point. After arriving at the placement point, the robot will put down the grabbed medicine, completing the entire grabbing and placement task.

[0069] 6. Mission completed: After the robot has finished grabbing and placing the medicine, it will notify the user through the voice interaction module that the medicine has been placed in the designated location, or provide feedback through other means (such as screen display).

[0070] Through the above steps, the robot can realize the full process automation from receiving voice commands to drug identification, grabbing, navigation and placement, which greatly improves the work efficiency of pharmacies and reduces human errors.

[0071] The above specific embodiments are merely explanations of the present invention and are not limitations of the present invention. After reading this specification, those skilled in the art may make modifications to the embodiments without any creative contribution as needed. However, such modifications are protected by the patent law as long as they are within the scope of the claims of the present invention.

Claims

1. A drugstore intelligent robot system, characterized by: Includes the following modules: The voice interaction module is used to convert the user's input voice into text content and use the AI ​​model to improve the efficiency and accuracy of voice recognition; Image recognition module, used to identify text on drug packaging to achieve accurate drug dispensing; The navigation module includes global mapping, self-positioning, path planning, motion control, and environmental perception functions, which are used for the robot to navigate in the pharmacy; The robotic arm control module is used to control the robotic arm to grab and place medicines.

2. The intelligent robot system for drugstores as claimed in claim 1, characterized in that: The voice interaction module includes: Speech recognition technology converts speech into text content; AI models to improve speech recognition efficiency and accuracy; The VITS speech generation model converts the text generated by the AI ​​model into speech to interact with users.

3. The intelligent robot system for drugstores as claimed in claim 1, characterized in that: The image recognition module uses the PaddleOCR module to recognize drug names, and with the help of the Baidu Wenxin Yiyan AI model, it has natural language processing capabilities to analyze the patient's prescription information and remind the patient of the dosage and precautions of different drugs.

4. The intelligent robot system for drugstores as claimed in claim 1, characterized in that: The navigation module uses SLAM technology for real-time positioning and map construction.

5. The intelligent robot system for drug stores as claimed in claim 1, characterized in that: The robotic arm control module comprises: Gripper control instructions, used to control the opening and closing of the gripper; Posture control instructions, used to control the robot arm to reach specific spatial coordinates; Servo control instructions are used to control each servo of the robotic arm.

6. A drugstore intelligent robot, characterized by: A pharmacy intelligent robot system comprising any one of claims 1 to 5.

7. A method for operating the intelligent robot for drugstores as claimed in claim 6, characterized in that: The operation method comprises the following steps: S1: receiving user instructions through the voice interaction module; S2: Identify drugs using image recognition module; S3: Use the navigation module to navigate the robot to the location of the medicine; S4: Grab and place the medicine through the robotic arm control module.

Images