Intelligent obstacle avoidance and path planning method and system for medical auxiliary robot

Abstract

The invention discloses an intelligent obstacle avoidance and path planning method and system for a medical auxiliary robot, and the method comprises the steps: obtaining multi-modal data from the medical auxiliary robot, and constructing a three-dimensional environment perception model according to the multi-modal data; identifying according to the three-dimensional environment perception model to obtain a character type and an equipment condition, and performing intelligent obstacle avoidance according to the character type, the equipment condition and a preset obstacle avoidance priority rule; performing crowd behavior prediction according to the multi-modal data and a preset social force model, and performing intelligent obstacle avoidance according to a crowd behavior prediction result; and calculating an optimal path according to the three-dimensional environment perception model, and performing path planning according to the optimal path. The method can improve the execution efficiency and flexibility of the medical auxiliary robot in a complex hospital environment, and can be widely applied to the technical field of path planning.

Description

Technical Field

[0001] The present application relates to the field of path planning technology, and in particular to an intelligent obstacle avoidance and path planning method and system for a medical assistance robot. Background Art

[0002] With the development of intelligent medical industry, medical assistance robots are more and more widely used in hospitals. They can not only reduce the work pressure of medical staff, but also improve the medical experience of patients. Common functions of medical assistance robots include welcoming reception, patient guidance, registration appointment, drug delivery, etc. However, the complexity of the hospital environment, especially the dynamic changes of patients, medical staff, equipment and emergency scenes, brings huge challenges to robot navigation and obstacle avoidance.

[0003] Existing medical-assisted robots mainly rely on simple infrared, ultrasonic and other sensors to perceive the surrounding environment, and avoid collisions through rule-based obstacle avoidance algorithms. However, this traditional obstacle avoidance method is often difficult to cope with when faced with complex and changeable environments such as hospitals. For example, when faced with fast-moving medical staff, fighting children or slow-moving elderly people, the response speed and accuracy of traditional obstacle avoidance systems are limited, which can easily lead to problems such as untimely path planning and obstacle avoidance failure. In addition, temporary obstacles (such as trolleys and stretchers) often appear in hospital scenes, which puts higher demands on the robot's dynamic obstacle avoidance capabilities. Therefore, the existing medical-assisted robot obstacle avoidance methods lack the ability to flexibly handle special scenarios (such as first aid and guidance for the disabled), and cannot provide safer and more timely avoidance for key groups in different scenarios. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an intelligent obstacle avoidance and path planning method and system for a medical-assistive robot, which can improve the execution efficiency of the medical-assistive robot in complex environments.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides an intelligent obstacle avoidance and path planning method for a medical assistance robot, comprising the following steps:

[0006] Acquire multimodal data from the medical assistance robot, and construct a three-dimensional environment perception model based on the multimodal data;

[0007] Recognize the person type and device status according to the three-dimensional environment perception model, and then perform intelligent obstacle avoidance according to the person type, the device status and a preset obstacle avoidance priority rule;

[0008] Predicting crowd behavior based on the multimodal data and a preset social force model, and then performing intelligent obstacle avoidance based on the crowd behavior prediction results;

[0009] An optimal path is calculated according to the three-dimensional environment perception model, and then path planning is performed according to the optimal path.

[0010] In some embodiments, the multimodal data includes distance data, visual data, perception data, and temperature data, and the acquiring of the multimodal data from the medical assistance robot and the building of a three-dimensional environment perception model according to the multimodal data specifically include:

[0011] Acquiring the distance data by using a laser radar of the medical assistance robot;

[0012] Acquiring the visual data through a camera of the medical assistance robot;

[0013] Acquiring the perception data through an ultrasonic sensor of the medical assistance robot;

[0014] Acquiring the temperature data through the infrared sensor of the medical assistance robot;

[0015] The distance data, the visual data, the perception data and the temperature data are fused to obtain the three-dimensional environment perception model.

[0016] In some embodiments, the identifying of the person type and the equipment status according to the three-dimensional environment perception model, and then performing intelligent obstacle avoidance according to the person type, the equipment status and the preset obstacle avoidance priority rules, specifically includes:

[0017] Setting the obstacle avoidance priority rule;

[0018] Identify the types of people including at least emergency patients, elderly people and children according to the three-dimensional environment perception model;

[0019] Obtaining the equipment status of the medical auxiliary equipment according to the three-dimensional environment perception model;

[0020] The medical assistance robot performs intelligent obstacle avoidance according to the person type, the equipment condition and the obstacle avoidance priority rule.

[0021] In some embodiments, the predicting of crowd behavior based on the multimodal data and a preset social force model, and then performing intelligent obstacle avoidance based on the crowd behavior prediction results, specifically includes:

[0022] Identify the types of people including at least emergency patients, elderly people and children according to the three-dimensional environment perception model;

[0023] Inputting the distance data, the visual data and the character type into the social force model to predict crowd behavior, and obtaining the crowd behavior prediction result;

[0024] The medical assistance robot is used to intelligently avoid obstacles based on the crowd behavior prediction results.

[0025] In some embodiments, calculating the optimal path according to the three-dimensional environment perception model, and then performing path planning according to the optimal path, specifically includes:

[0026] Determine the path cost function;

[0027] Obtaining obstacle information according to the three-dimensional environment perception model recognition;

[0028] The optimal path is calculated by the D*Lite dynamic path planning algorithm according to the obstacle information and the path cost function;

[0029] The driving path of the medical assistance robot is replanned according to the optimal path.

[0030] In some embodiments, the intelligent obstacle avoidance and path planning method further includes:

[0031] Assessing the risk level of each area of ​​the target hospital according to the three-dimensional environmental perception model;

[0032] The speed and obstacle avoidance sensitivity of the medical assistance robot are adjusted according to the risk level.

[0033] In some embodiments, the social force model is:

[0034]

[0035] Among them, F i represents the force acting on the i-th character type, f ij represents the social force exerted by others on the i-th character type, f io Represents the force exerted by the surrounding obstacles on the i-th character type.

[0036] To achieve the above objectives, another aspect of the embodiments of the present application provides an intelligent obstacle avoidance and path planning system for a medical assistance robot, comprising:

[0037] A multimodal perception module, used to obtain multimodal data from the medical assistance robot and construct a three-dimensional environment perception model based on the multimodal data;

[0038] A priority judgment module, used to identify the type of person and the condition of equipment according to the three-dimensional environment perception model, and then perform intelligent obstacle avoidance according to the type of person, the condition of equipment and preset obstacle avoidance priority rules;

[0039] A behavior prediction module, used to predict crowd behavior based on the multimodal data and a preset social force model, and then perform intelligent obstacle avoidance based on the crowd behavior prediction results;

[0040] The path planning module is used to calculate the optimal path according to the three-dimensional environment perception model, and then perform path planning according to the optimal path.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the intelligent obstacle avoidance and path planning method of the medical-assisted robot as described above is realized.

[0042] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the intelligent obstacle avoidance and path planning method of the medical-assisted robot as described above.

[0043] The beneficial effects of the present invention are as follows: the intelligent obstacle avoidance and path planning method and system of the medical assistance robot of the present invention obtains multimodal data from the medical assistance robot, and then constructs a three-dimensional environment perception model according to the multimodal data, identifies the type of person and the condition of the equipment according to the three-dimensional environment perception model, and then performs intelligent obstacle avoidance according to the type of person, the condition of the equipment and the preset obstacle avoidance priority rules, predicts crowd behavior according to the multimodal data and the preset social force model, and then performs intelligent obstacle avoidance according to the crowd behavior prediction results, and finally calculates the optimal path according to the three-dimensional environment perception model, and then performs path planning according to the optimal path. The present invention combines multimodal data fusion, priority obstacle avoidance, crowd behavior prediction and dynamic path planning technology to improve the execution efficiency and flexibility of the medical assistance robot in a complex hospital environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flowchart of the steps of an intelligent obstacle avoidance and path planning method for a medical assistance robot provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of the structure of an intelligent obstacle avoidance and path planning system for a medical assistance robot provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0049] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0050] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0051] With the development of intelligent medical industry, medical assistance robots are more and more widely used in hospitals. They can not only reduce the work pressure of medical staff, but also improve the medical experience of patients. Common functions of medical assistance robots include welcoming reception, patient guidance, registration appointment, drug delivery, etc. However, the complexity of the hospital environment, especially the dynamic changes of patients, medical staff, equipment and emergency scenes, brings huge challenges to robot navigation and obstacle avoidance.

[0052] Existing intelligent obstacle avoidance methods for medical assistance robots in hospital environments have the following main shortcomings:

[0053] Insufficient single sensor data: Existing medical assistance robots mainly rely on ultrasonic or infrared sensors for obstacle detection. Although these sensors can sense obstacles within a certain range, they are prone to blind spots when facing complex hospital scenes (such as multiple people moving, equipment push, and dynamic obstacles). The single sensor data makes it difficult to accurately judge the shape, speed, and direction of movement of obstacles in dense crowds, resulting in untimely or inaccurate obstacle avoidance responses.

[0054] Limited dynamic obstacle avoidance capabilities: In hospital environments, medical staff and patients often move quickly, and the environment changes frequently, with frequent temporary obstacles such as carts and stretchers. Traditional rule-based obstacle avoidance algorithms can only deal with static or slow-moving obstacles, and are difficult to adapt to complex obstacle avoidance requirements in dynamic environments, resulting in possible collisions between medical assistance robots and moving medical staff, patients, etc.

[0055] Lack of situational awareness and priority judgment: Existing technologies cannot effectively distinguish the priorities of different objects in the environment. When facing some special objects, the robot cannot intelligently adjust its obstacle avoidance strategy, resulting in slow avoidance response to important scenes (such as emergency passages).

[0056] Inflexible path planning: Existing obstacle avoidance systems often use static path planning and lack the ability to adjust to dynamic environments in real time. In a complex and changing environment like a hospital, medical assistance robots need to frequently replan their paths. Traditional path planning algorithms have a slow response speed and are prone to stagnation and detours, reducing the working efficiency of medical assistance robots in high-density environments.

[0057] Lack of prediction of crowd behavior: There are often special situations in hospitals, such as children fighting and elderly people moving slowly. Traditional obstacle avoidance algorithms cannot effectively predict the behavior patterns of these people, resulting in the inability of medical assistance robots to make avoidance decisions in advance, posing potential safety hazards.

[0058] To this end, an embodiment of the present invention proposes an intelligent obstacle avoidance and path planning method for a medical assistance robot, which obtains multimodal data from the medical assistance robot, and then constructs a three-dimensional environment perception model based on the multimodal data, identifies the type of person and the equipment status according to the three-dimensional environment perception model, and then performs intelligent obstacle avoidance according to the type of person, the equipment status and the preset obstacle avoidance priority rules, predicts crowd behavior according to the multimodal data and the preset social force model, and then performs intelligent obstacle avoidance according to the crowd behavior prediction results, and finally calculates the optimal path according to the three-dimensional environment perception model, and then performs path planning according to the optimal path. The present invention combines multimodal data fusion, priority obstacle avoidance, crowd behavior prediction and dynamic path planning technology to improve the execution efficiency and flexibility of medical assistance robots in complex hospital environments.

[0059] Reference Figure 1 , Figure 1 The present invention provides a flowchart of a method for intelligent obstacle avoidance and path planning of a medical assistance robot. The present invention provides a method for intelligent obstacle avoidance and path planning of a medical assistance robot, and the method includes steps S101 to S104:

[0060] S101, acquiring multimodal data from the medical assistance robot, and constructing a three-dimensional environment perception model according to the multimodal data;

[0061] Specifically, in an embodiment of the present invention, a medical assistance robot (hereinafter referred to as a robot) includes a sensor system, a computing platform, a power and motion control system, a positioning system, and a wireless communication system. The sensor system is composed of a laser radar, a camera, an ultrasonic and an infrared sensor, and is used to collect multimodal data in the surrounding environment in real time, so that the robot can accurately perceive the distance, shape, and motion state of obstacles and other information, and adapt to complex hospital scenes. The computing platform includes an edge computing device and a GPU acceleration module, the edge computing device is used to process multi-sensor data, and the GPU acceleration module is used to accelerate computer vision and deep learning tasks, so that the robot can complete image recognition, path planning, and obstacle avoidance decisions in real time. The power and motion control system is driven by an electric motor to provide power support for the robot to move smoothly and push the wheelchair. Combined with the suspension and shock absorption system, it maintains stable operation under different road conditions in the hospital, especially when pushing the wheelchair or taking medicine, and maintains flexible obstacle avoidance. The positioning system is used to achieve outdoor and indoor positioning through GPS and an inertial measurement unit (IMU) so that the robot can accurately locate and navigate. The wireless communication system is used to synchronize patient information and appointment registration information with the hospital information system in real time, and supports multi-robot collaborative work.

[0062] As an optional implementation, the multimodal data includes distance data, visual data, perception data, and temperature data. The step of acquiring the multimodal data from the medical assistance robot and constructing a three-dimensional environment perception model according to the multimodal data can be specifically divided into the following steps S1011 to S1015:

[0063] S1011. Acquire distance data through a laser radar of a medical assistance robot;

[0064] S1012, obtaining visual data through a camera of the medical assistance robot;

[0065] S1013, acquiring perception data through an ultrasonic sensor of the medical assistance robot;

[0066] S1014, obtaining temperature data through an infrared sensor of the medical assistance robot;

[0067] S1015: Fuse the distance data, visual data, perception data, and temperature data to obtain a three-dimensional environment perception model.

[0068] In some optional embodiments, multimodal data is first obtained through laser radar, camera, ultrasonic and infrared sensors. The laser radar is used to detect the distance and contour information of obstacles to obtain distance data; the camera is responsible for capturing images for visual recognition to obtain visual data; the ultrasonic and infrared sensors provide close distance and temperature perception information to obtain perception data and temperature data. The data of different sensors are integrated into a unified three-dimensional environment perception model through a multimodal data fusion algorithm.

[0069] For example, assume that the distance data acquired by the laser radar is D LiDAR , the visual data obtained by the camera is D camera , the perception data obtained by the ultrasonic sensor is D ultrasound , the temperature data obtained by the infrared sensor is D infrared , then the fused three-dimensional environment perception model M env It can be expressed as the following formula:

[0070] M env =f(D LiDAR ,D camera ,D ultrasound ,D infrared )

[0071] Here, f(·) represents the multimodal data fusion function, which is used to align the time and space of each sensor data and integrate the information of each sensor to generate an accurate 3D environment perception model.

[0072] S102, identifying the person type and the equipment status according to the three-dimensional environment perception model, and then performing intelligent obstacle avoidance according to the person type, the equipment status and the preset obstacle avoidance priority rules;

[0073] As an optional implementation, the person type and device status are obtained according to the three-dimensional environment perception model, and then the step of performing intelligent obstacle avoidance according to the person type, device status and preset obstacle avoidance priority rules can be specifically divided into the following steps S1021 to S1024:

[0074] S1021. Setting obstacle avoidance priority rules;

[0075] Specifically, the obstacle avoidance priority rule set in the embodiment of the present invention is expressed by the following formula:

[0076] P obstacle =w1·R emergency +w2·R elderly +w3·R child+w4·R other

[0077] Among them, P obstacle represents the priority avoidance weight of the medical assistance robot to the current obstacle, R emergency , R elderly , R child Respectively represent the recognition results of emergency patients, the elderly and children, R other represents the weights of other obstacles, w1, w2, w3 and w4 represent pre-set priority weight parameters under different scenarios, and the priority weight parameters can be set according to actual needs and are not limited here.

[0078] S1022, identifying at least the types of people including emergency patients, elderly people and children according to the three-dimensional environment perception model;

[0079] S1023. Obtaining the equipment status of the medical auxiliary equipment according to the three-dimensional environment perception model identification;

[0080] S1024. Perform intelligent obstacle avoidance of the medical assistance robot according to the person type, equipment condition, and obstacle avoidance priority rules.

[0081] In some optional embodiments, situational analysis is performed through the sensor data in the three-dimensional environmental perception model to identify special groups such as emergency patients, the elderly, children, and emergency patients, as well as the equipment conditions of medical auxiliary equipment, and assign different obstacle avoidance priorities. Among them, medical auxiliary equipment may include stretchers, carts, wheelchairs, mobile beds, monitors, ventilators, cardiopulmonary resuscitation machines, etc. According to the identified person type and equipment conditions, a dynamically adjusted obstacle avoidance strategy is adopted. For example, when an emergency patient is detected, the robot will trigger the emergency obstacle avoidance mode, give priority to avoidance and open the way for medical staff in time.

[0082] S103, predicting crowd behavior based on multimodal data and a preset social force model, and then performing intelligent obstacle avoidance based on the crowd behavior prediction results;

[0083] As an optional implementation, crowd behavior prediction is performed based on the multimodal data and the preset social force model, and then intelligent obstacle avoidance is performed based on the crowd behavior prediction result. Specifically, the step can be divided into the following steps S1031 to S1033:

[0084] S1031, identifying at least the types of people including emergency patients, elderly people and children according to the three-dimensional environment perception model;

[0085] S1032, inputting the distance data, visual data and character type into the social force model to predict crowd behavior, and obtaining crowd behavior prediction results;

[0086] S1033. Perform intelligent obstacle avoidance of the medical assistance robot based on crowd behavior prediction results.

[0087] In some optional embodiments, the social force model is used to predict the behavior of people in the hospital, especially special groups such as emergency patients, the elderly and children. Based on the distance data collected by the lidar and the visual data collected by the camera, the movement trajectory of the crowd is predicted, and obstacle avoidance decisions are made in advance. For example, when the behavior of children fighting is identified, the social force model predicts its movement trend based on the speed, direction, density and other information of the crowd, and plans the best obstacle avoidance route for the robot.

[0088] As an optional implementation, the social force model is:

[0089]

[0090] Among them, F i represents the force acting on the i-th character type, f ij represents the social force exerted by others on the i-th character type, f io Represents the force exerted by the surrounding obstacles on the i-th character type.

[0091] S104: Calculate the optimal path according to the three-dimensional environment perception model, and then perform path planning based on the optimal path.

[0092] As an optional implementation, the step of calculating the optimal path according to the three-dimensional environment perception model and then performing path planning according to the optimal path can be specifically divided into the following steps S1041 to S1044:

[0093] S1041, determining a path cost function;

[0094] S1042, obtaining obstacle information according to the three-dimensional environment perception model recognition;

[0095] S1043, calculating the optimal path according to obstacle information and path cost function using the D*Lite dynamic path planning algorithm;

[0096] S1044. Replan the driving path of the medical assistance robot according to the optimal path.

[0097] In some optional embodiments, the D*Lite dynamic path planning algorithm is used to achieve flexible navigation of the robot in a complex dynamic environment. During the initial path planning, the optimal path is calculated according to the environment model Menv. When new dynamic obstacle information (such as a moving patient or cart) is detected, the path is further replanned by the D*Lite algorithm so that the robot can avoid the new obstacle.

[0098] The goal of path planning is to minimize the path cost function C(p), which is defined as:

[0099]

[0100] Among them, c(d i ) represents the distance cost of the i-th segment in the path, Δo i represents the cost of the change of obstacles encountered on the i-th path, and α is a weight factor used to control the robot's avoidance sensitivity to dynamic obstacles. Through this path cost function, the embodiment of the present invention can maintain efficient obstacle avoidance capabilities while adjusting the path in real time.

[0101] As an optional implementation, the intelligent obstacle avoidance and path planning method further includes the following steps S105 and S106:

[0102] S105, assessing the risk level of each area of ​​the target hospital according to the three-dimensional environmental perception model;

[0103] S106. Adjust the speed and obstacle avoidance sensitivity of the medical assistance robot according to the risk level.

[0104] In some optional embodiments, based on the risk assessment mechanism, the risk level of different areas of the hospital is assessed (such as the high priority of the emergency channel). Through the real-time feedback of the risk level data, the robot's obstacle avoidance sensitivity and travel speed are dynamically adjusted. For example, in the emergency area, the robot speed is automatically reduced and the robot's obstacle avoidance reaction sensitivity is increased; while in the general ward area, the robot can be kept running at a normal speed to improve work efficiency.

[0105] In order to better understand the solution of the embodiment of the present invention, the intelligent obstacle avoidance and path planning method of the medical assistance robot provided by the embodiment of the present invention is described below through specific application examples.

[0106] 1) Preliminary preparation and system initialization

[0107] First, the robot connects data with the hospital's management system to obtain the day's registration information, the list of scheduled patients, and the distribution of related departments. The robot is located at the designated starting position of the outpatient reception area, the sensor system (lidar, camera, ultrasonic sensor, infrared sensor) is turned on, and the computing platform is started. First, the robot uses the lidar to perform preliminary three-dimensional environmental perception modeling of the outpatient reception area, generate basic information about the current environment, and preliminarily identify nearby obstacles.

[0108] 2) Environmental perception and multimodal data fusion

[0109] The robot acquires data from the robot's lidar, camera, ultrasonic and infrared sensors in real time to build a dynamic three-dimensional environmental perception model. The lidar is responsible for measuring the distance of obstacles and the outline of objects in the ward, and the camera captures images to identify the movement of patients and medical staff. Ultrasonic sensors assist in detecting close-range obstacles so that the robot can avoid collisions in crowded areas. Infrared sensors supplement temperature data at night and in low-light environments. All sensor data is integrated into a unified three-dimensional environmental perception model through a fusion algorithm for subsequent path planning and obstacle avoidance.

[0110] 3) Situation identification and priority determination

[0111] Based on the visual data collected by the camera, the robot can identify the patient type, movement speed and equipment status. The robot uses a situational recognition algorithm to distinguish between emergency patients, the elderly, children and medical staff, and adjusts the obstacle avoidance priority according to the degree of urgency. For example, when an emergency patient is detected passing through, the robot immediately triggers the high-priority obstacle avoidance mode and adjusts the path planning to make way for the emergency passage. For the elderly and disabled, the robot's travel speed is adjusted to ensure a larger safety distance to avoid collisions.

[0112] 4) Dynamic path planning and task execution

[0113] Based on the three-dimensional environment perception model, the D*Lite dynamic path planning algorithm is used for real-time path planning. The robot can continuously adjust its route according to the dynamic changes of patients and medical staff to ensure the optimal path and no obstacles. When dynamic changes are detected in a certain area (such as patient movement or cart passing by), the path is immediately replanned to ensure that the robot bypasses obstacles and continues to perform the task.

[0114] 5) Behavior simulation and prediction

[0115] The social force model is used to predict children's behavior and avoid collisions caused by their sudden behavior. The camera captures the children's movements in real time, and the behavior prediction algorithm analyzes their movement trajectory, makes obstacle avoidance decisions in advance, and adjusts the path planning. At the same time, for the elderly who move slowly, their movement speed is predicted and the travel speed is dynamically adjusted to ensure that the avoidance space is safe enough.

[0116] 6) Real-time risk assessment and feedback adjustment

[0117] At different locations in the experimental area, dynamic adjustments are made in real time based on the risk level of the current area. In high-risk areas such as emergency passages and doctor's offices, the robot's travel speed is reduced and the obstacle avoidance sensitivity is increased; in ordinary waiting areas, the robot resumes normal speed. Throughout the process, sensors continuously receive environmental data and feed it back to the computing platform to dynamically adjust the robot's speed and path.

[0118] 7) Mission completion and data recording

[0119] When the robot completes a task (such as safely delivering a patient to the registration window or department), it records the execution status of the current task, the obstacle avoidance path selection process, and sensor data. The robot returns to its initial position and prepares to perform the next task. After each task is completed, the sensor data, path planning, and obstacle avoidance strategy in the process are analyzed and optimized to provide a basis for improvement for subsequent tasks.

[0120] The above is a description of the intelligent obstacle avoidance and path planning method for a medical assistance robot according to an embodiment of the present invention. It can be appreciated that, compared with the obstacle avoidance method for a medical assistance robot in the prior art, the embodiment of the present invention has the following advantages:

[0121] 1. Through multimodal data fusion, it can provide more comprehensive and accurate environmental perception capabilities, so that medical assistance robots can accurately identify static and dynamic obstacles in complex and changing environments;

[0122] Second, dynamic path planning is performed through the D*Lite dynamic path planning algorithm, so that the robot can quickly and accurately re-plan its path in the hospital to avoid moving medical staff, patients and equipment, and improve the robot's reaction speed and obstacle avoidance efficiency in dynamic environments;

[0123] 3. According to the type of person, equipment status and emergency situation, the priority of obstacle avoidance strategy is adjusted so that the robot can give way to special groups such as emergency patients and the disabled, improving safety and the ability to respond to emergencies;

[0124] Fourth, crowd behavior prediction is performed through social force models, so that the robot can identify and predict the fighting behavior of children or the slow movement of the elderly, make obstacle avoidance decisions in advance, and thus operate safely in complex hospital scenarios.

[0125] Reference Figure 2 The embodiment of the present invention further provides an intelligent obstacle avoidance and path planning system for a medical assistance robot, comprising:

[0126] A multimodal perception module is used to obtain multimodal data from the medical assistance robot and build a three-dimensional environmental perception model based on the multimodal data;

[0127] The priority judgment module is used to identify the type of person and the status of the equipment according to the three-dimensional environment perception model, and then perform intelligent obstacle avoidance according to the type of person, the status of the equipment and the preset obstacle avoidance priority rules;

[0128] The behavior prediction module is used to predict crowd behavior based on multimodal data and a preset social force model, and then perform intelligent obstacle avoidance based on the crowd behavior prediction results;

[0129] The path planning module is used to calculate the optimal path based on the three-dimensional environment perception model, and then perform path planning based on the optimal path.

[0130] Furthermore, the intelligent obstacle avoidance and path planning system of the medical assistance robot proposed in the embodiment of the present invention also includes a risk assessment and feedback module. The operation of the intelligent obstacle avoidance and path planning method of the medical assistance robot in the embodiment of the present invention depends on the close integration of software and hardware, so that the robot can achieve effective obstacle avoidance and path planning in a complex hospital environment. The following describes the overall process of the combination of software and hardware:

[0131] 1. Multimodal perception:

[0132] After the robot is started, the hardware sensor system starts working first, including lidar, camera, ultrasonic and infrared sensors. These sensors collect information from the hospital environment in real time. The lidar provides information about the distance and shape of obstacles, the camera is responsible for image capture and identification of people, equipment, etc., and the ultrasonic and infrared sensors are used for close-range obstacle detection and temperature perception. The collected multimodal data is transmitted to the computing platform through the hardware for preliminary processing and fusion.

[0133] 2. Multimodal data fusion:

[0134] After multimodal data collection is completed, the software's multimodal perception module integrates data from lidar, cameras, ultrasound, and infrared sensors. The fused data forms a three-dimensional environmental perception model that contains detailed information about dynamic and static obstacles. This model provides the basis for subsequent obstacle avoidance and path planning, allowing the robot to accurately perceive the complex environment of the hospital.

[0135] 3. Situation identification and priority judgment:

[0136] The fused 3D environmental perception data enters the priority judgment module, and combined with computer vision algorithms, the robot identifies the types of people in the hospital (such as emergency patients, the elderly, and children) and the conditions of equipment (such as stretchers and carts). The robot dynamically adjusts the obstacle avoidance strategy based on the preset obstacle avoidance priority rules to ensure that priority is given to emergency patients, elderly people with limited mobility, etc.

[0137] 4. Behavior simulation and prediction:

[0138] In some complex scenarios (such as children playing and elderly people walking slowly), the behavior prediction module predicts the behavior trajectory of the crowd through the visual and distance data collected by the sensor. The robot's movement strategy is adjusted in real time according to the predicted results to avoid possible obstacles in advance. This process needs to be completed through software algorithms, while the hardware sensor system continuously updates the crowd behavior information.

[0139] 5. Real-time dynamic path planning:

[0140] When the situation recognition is completed, the software's path planning module will use the D*Lite dynamic path planning algorithm to plan the best path for the robot based on the current 3D environment perception model. If new dynamic obstacle information is detected (such as moving patients, carts, etc.), the path will be updated in real time to enable the robot to flexibly bypass these obstacles. At this time, the hardware computing platform will continue to perform high-performance computing tasks, support fast path adjustments, and avoid collisions.

[0141] 6. Risk assessment and feedback adjustment:

[0142] When the robot is running in different areas of the hospital (such as emergency passages, general wards, etc.), the risk assessment and feedback module dynamically adjusts the robot's speed and obstacle avoidance sensitivity according to the risk level of the area. The system will feed back the environmental change information obtained in real time by the sensor to the hardware power and motion control system to adjust the robot's travel speed and obstacle avoidance behavior in a timely manner. For example, in the emergency passage, the robot will reduce its speed and increase the sensitivity of its obstacle avoidance reaction; while in the general ward area, the robot can maintain a normal moving speed.

[0143] 7. Action Execution:

[0144] When all path planning, obstacle avoidance strategies, and behavior predictions are completed, the motion control system is responsible for converting the robot's instructions into actual actions. The hardware's power system adjusts the robot's speed and direction based on the feedback information provided by the sensors and the path planning results, and in some cases, performs stop or detour operations. At the same time, the robot will also make fine adjustments based on the hardware's sensor feedback to ensure that the actual movement conforms to the predetermined plan.

[0145] 8. Task completion and real-time feedback:

[0146] When the robot completes an obstacle avoidance and path adjustment task, the system will feed back the completion information to the control center and record the current environmental data, decision-making process, etc. If the hospital scene changes, the sensor will collect data again and trigger a new round of path planning and obstacle avoidance decisions.

[0147] The contents of the above-mentioned medical-assisting robot's intelligent obstacle avoidance and path planning method embodiments are applicable to the intelligent obstacle avoidance and path planning system embodiments of this medical-assisting robot. The functions specifically implemented by the intelligent obstacle avoidance and path planning system embodiments of this medical-assisting robot are the same as those in the above-mentioned medical-assisting robot's intelligent obstacle avoidance and path planning method embodiments, and the beneficial effects achieved are also the same as those achieved by the above-mentioned medical-assisting robot's intelligent obstacle avoidance and path planning method embodiments.

[0148] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and when the program is executed by the processor, the intelligent obstacle avoidance and path planning method of the medical assistance robot is realized. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0149] like Figure 3 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Figure 3 , an embodiment of the present invention provides an electronic device, including:

[0150] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0151] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the intelligent obstacle avoidance and path planning method of the medical assistance robot in the embodiment of the present invention;

[0152] Input / output interface 1003, used to implement information input and output;

[0153] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0154] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0155] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0156] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned intelligent obstacle avoidance and path planning method of the medical-assisted robot.

[0157] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0159] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0160] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0161] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0162] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0163] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0164] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0165] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. An intelligent obstacle avoidance and path planning method for a medical assistance robot, characterized in that: The following steps are involved: Acquire multimodal data from the medical assistance robot, and construct a three-dimensional environment perception model based on the multimodal data; Recognize the person type and device status according to the three-dimensional environment perception model, and then perform intelligent obstacle avoidance according to the person type, the device status and a preset obstacle avoidance priority rule; Predicting crowd behavior based on the multimodal data and a preset social force model, and then performing intelligent obstacle avoidance based on the crowd behavior prediction results; An optimal path is calculated according to the three-dimensional environment perception model, and then path planning is performed according to the optimal path.

2. The intelligent obstacle avoidance and path planning method of a medical assistive robot according to claim 1, characterized in that: The multimodal data includes distance data, visual data, perception data and temperature data. The acquiring of the multimodal data from the medical assistance robot and constructing a three-dimensional environment perception model according to the multimodal data specifically includes: Acquiring the distance data by using a laser radar of the medical assistance robot; Acquiring the visual data through a camera of the medical assistance robot; Acquiring the perception data through an ultrasonic sensor of the medical assistance robot; Acquiring the temperature data through the infrared sensor of the medical assistance robot; The distance data, the visual data, the perception data and the temperature data are fused to obtain the three-dimensional environment perception model.

3. The intelligent obstacle avoidance and path planning method of a medical assistive robot according to claim 1, characterized in that: The identifying the person type and the equipment status according to the three-dimensional environment perception model, and then performing intelligent obstacle avoidance according to the person type, the equipment status and the preset obstacle avoidance priority rule, specifically includes: Setting the obstacle avoidance priority rule; Identify the types of people including at least emergency patients, elderly people and children according to the three-dimensional environment perception model; Obtaining the equipment status of the medical auxiliary equipment according to the three-dimensional environment perception model; The medical assistance robot performs intelligent obstacle avoidance according to the person type, the equipment condition and the obstacle avoidance priority rule.

4. The intelligent obstacle avoidance and path planning method of a medical assistive robot according to claim 2, characterized in that: The method of predicting crowd behavior based on the multimodal data and a preset social force model, and then performing intelligent obstacle avoidance based on the crowd behavior prediction results, specifically includes: Identify the types of people including at least emergency patients, elderly people and children according to the three-dimensional environment perception model; Inputting the distance data, the visual data and the character type into the social force model to perform crowd behavior prediction, and obtaining the crowd behavior prediction result; The medical assistance robot is used to intelligently avoid obstacles based on the crowd behavior prediction results.

5. The intelligent obstacle avoidance and path planning method of a medical assistive robot according to claim 1, characterized in that: The calculating the optimal path according to the three-dimensional environment perception model, and then performing path planning according to the optimal path, specifically includes: Determine the path cost function; Obtaining obstacle information according to the three-dimensional environment perception model recognition; The optimal path is calculated by the D*Lite dynamic path planning algorithm according to the obstacle information and the path cost function; The driving path of the medical assistance robot is replanned according to the optimal path.

6. The intelligent obstacle avoidance and path planning method of a medical assistive robot according to claim 1, characterized in that: The intelligent obstacle avoidance and path planning method also includes: Assessing the risk level of each area of ​​the target hospital according to the three-dimensional environmental perception model; The speed and obstacle avoidance sensitivity of the medical assistance robot are adjusted according to the risk level.

7. An intelligent obstacle avoidance and path planning method for a medical assistive robot according to any one of claims 1 to 6, characterized in that: The social force model is: Among them, F i represents the force acting on the i-th character type, f ij represents the social force exerted by others on the i-th character type, f io Represents the force exerted by the surrounding obstacles on the i-th character type.

8. An intelligent obstacle avoidance and path planning system for a medical assistance robot, characterized in that: include: A multimodal perception module, used to obtain multimodal data from the medical assistance robot and construct a three-dimensional environment perception model based on the multimodal data; A priority judgment module, used to identify the type of person and the condition of equipment according to the three-dimensional environment perception model, and then perform intelligent obstacle avoidance according to the type of person, the condition of equipment and preset obstacle avoidance priority rules; A behavior prediction module, used to predict crowd behavior based on the multimodal data and a preset social force model, and then perform intelligent obstacle avoidance based on the crowd behavior prediction results; The path planning module is used to calculate the optimal path according to the three-dimensional environment perception model, and then perform path planning according to the optimal path.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the intelligent obstacle avoidance and path planning method for a medical-assistive robot as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the intelligent obstacle avoidance and path planning method for a medical-assistive robot as described in any one of claims 1 to 7.

Images