A rgbd robot nursing system and method based on deep learning

Abstract

The present invention provides a RGBD robot nursing system and method based on deep learning, the system includes a first RGBD camera, a second RGBD camera, a mechanical arm and a controller; the method includes face multi-modal information extraction, item multi-modal Information extraction and control of the mechanical arm, multi-modal information extraction of the face to obtain the three-dimensional coordinates of the mouth, multi-modal information extraction of the item to obtain the three-dimensional coordinates of the item, the controller machine controls the mechanical arm to grab the item according to the three-dimensional coordinates of the item, and then According to the three-dimensional coordinates of the face and mouth, the robotic arm is controlled to deliver the items to the patient's mouth. Using the deep learning method, it is more robust than the traditional technology, can adapt to different environments, and is more suitable for application in public service scenarios. At the same time, it uses multi-modal face pose and key point estimation, making RGBD robots The nursing system service is more precise.

Description

technical field [0001] The invention belongs to the field of medical devices, and in particular relates to a deep learning-based RGBD robot nursing system and method. Background technique [0002] Since 2012, artificial intelligence has ushered in a new round of product application boom with the breakthrough of deep learning algorithms. Under the background of insufficient supply of medical resources and serious imbalance in the distribution of medical resources in my country, artificial intelligence has been implemented in various segments of medical and health, covering various application scenarios of the entire industrial chain. [0003] At the same time, with the development of the country, the society is gradually aging, and the demand for medical care, nursing and rehabilitation is increasing. Due to the serious imbalance in the ratio of doctors to patients in the domestic medical system. Many patients in the ward who have difficulty moving and have no autonomy need ...

AI Technical Summary

Problems solved by technology

Since this invention does not have a multi-degree-of-freedom robotic arm, it is suitable for entertainment for the elderly. For scenes like hospitals, it is difficult to help patients feed water and medicine.

Images