Intelligent manipulator for nursing and method

Abstract

The invention provides an intelligent manipulator for nursing and a method. A task instruction of the intelligent manipulator is received; determining a feature matrix of all articles in the working space based on an intelligent visual perception sensor, and determining a spatial deviation degree of a nursing tool in the working space according to geometric features of the nursing tool required for executing the task instruction and the feature matrix; determining a three-dimensional pose space of each nursing part through the visual point cloud data of the nursing target, and further obtaining a compensation coefficient of pose change in each three-dimensional pose space when the intelligent manipulator operates the nursing area; and determining a dynamic posture sequence of the intelligent manipulator according to the spatial deviation degree and all the compensation coefficients, and dynamically adjusting the nursing path of the intelligent manipulator based on the dynamic posture sequence. According to the scheme, the nursing tool and the nursing area can be recognized through the artificial intelligence visual technology, dynamic path planning is carried out, and therefore the path adaptability of the intelligent manipulator in the nursing task is improved.

Description

Technical Field

[0001] The present application relates to the technical field of robot program control, and more specifically, to an intelligent robot and method for nursing. Background Art

[0002] With the rapid development of artificial intelligence technology, the application of intelligent robots in the field of medical care is gradually increasing, especially in auxiliary nursing tasks. Intelligent robots for nursing are a kind of equipment specially used to assist nursing staff to complete various nursing tasks; and artificial intelligence vision is a technology that combines computer vision and artificial intelligence, which aims to enable machines to understand and analyze information extracted from visual data and make corresponding intelligent decisions; combining artificial intelligence vision technology with intelligent robots for nursing can use computer vision for real-time environmental perception and decision-making, assisting intelligent robots for nursing to complete various nursing tasks, and providing accurate and efficient services for patients and caregivers.

[0003] Although existing intelligent robotic arms for nursing have certain flexibility and intelligent functions, when performing nursing tasks that require high-precision operations (such as wound cleaning and injection assistance), it is often difficult to accurately identify the spatial positions of nursing tools and nursing areas and dynamically adjust the operation path according to the spatial position. This causes positioning deviations in the intelligent robotic arms for nursing during operation, thereby causing harm to patients. Therefore, how to use artificial intelligence vision technology to identify nursing tools and nursing areas and perform dynamic path planning to improve the path adaptability of intelligent robotic arms in nursing tasks has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides an intelligent manipulator and method for nursing, which can identify nursing tools and nursing areas and perform dynamic path planning through artificial intelligence vision technology, thereby improving the path adaptability of the intelligent manipulator in nursing tasks.

[0005] In a first aspect, the present application provides a method for controlling an intelligent manipulator for nursing, comprising: Receive task instructions when the intelligent manipulator performs nursing operations on nursing targets; Extract features of all items in the workspace corresponding to the nursing operation based on the intelligent visual perception sensor to obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the nursing tools in the workspace according to the geometric features of the nursing tools required to execute the task instructions and the feature matrix; Obtain visual point cloud data of the nursing target, determine the three-dimensional posture space of each nursing part on the nursing target through the visual point cloud data, and then perform posture calibration on the nursing area that executes the task instruction in each three-dimensional posture space, and obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; The position changes of the intelligent manipulator when executing the task instruction are fused according to the spatial deviation and all the compensation coefficients to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction, and the nursing path of the intelligent manipulator is dynamically adjusted based on the dynamic posture sequence.

[0006] In some embodiments, the feature matrix of all objects in the workspace corresponding to the nursing operation is obtained by extracting features based on the intelligent visual perception sensor, specifically including: Acquire workspace images of nursing operations; Performing object detection on the workspace image to obtain all auxiliary objects in the workspace; For each auxiliary item, determine a spatial feature vector of the auxiliary item; Performing dimensionality reduction compression on the spatial feature vector to obtain a feature vector of the auxiliary item, and then obtaining a feature vector of each auxiliary item; Determine the feature matrix of all objects in the workspace based on all feature vectors.

[0007] In some embodiments, determining the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix specifically includes: Determine the geometric features of nursing tools required to perform task instructions; Performing confidence matching on the geometric features and the feature matrix to obtain feature confidence between the nursing tool and each auxiliary item in the workspace; The spatial deviation of the nursing tool in the workspace is determined based on all feature confidences and the spatial position of the intelligent manipulator.

[0008] In some embodiments, determining the three-dimensional position space of each nursing part on the nursing target through the visual point cloud data specifically includes: Determine a three-dimensional surface model of a nursing target according to the visual point cloud data; Divide the three-dimensional surface model into regions to obtain all nursing parts in the nursing target; For each nursing part on the nursing target, the three-dimensional posture space of the nursing part is determined according to the partition information of the nursing part in the visual point cloud data, and then the three-dimensional posture space of each nursing part on the nursing target is obtained.

[0009] In some embodiments, the posture calibration of the nursing area executing the task instruction is performed in each three-dimensional posture space, and the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area specifically includes: Determine the posture characteristics of the nursing area for executing task instructions; For each three-dimensional posture space, the posture feature is aligned with the three-dimensional posture space to obtain a posture deviation between the care area and the three-dimensional posture space; Based on the posture deviation, a compensation coefficient of posture change in the three-dimensional posture space when the intelligent manipulator operates the care area is determined, and then the compensation coefficient of posture change in each three-dimensional posture space when the intelligent manipulator operates the care area is obtained.

[0010] In some embodiments, the deviation fusion is performed on the posture change of the intelligent manipulator when executing the task instruction according to the spatial deviation degree and all compensation coefficients to obtain the dynamic posture sequence of the intelligent manipulator executing the task instruction, which specifically includes: Determine the spatial error between the intelligent manipulator and the nursing area based on all compensation coefficients; The spatial deviation and the spatial error are fused to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction.

[0011] In some embodiments, the visual point cloud data is three-dimensional spatial data of the care target (ie, the body of the target user) in the workspace, including the geometric shape, position information, and posture information of the care target in the three-dimensional space.

[0012] In a second aspect, the present application provides an intelligent robot arm for nursing, comprising: The acquisition module is used to receive the task instructions when the intelligent manipulator performs nursing operations on the nursing target; A processing module is used to extract features of all items in the workspace corresponding to the nursing operation based on an intelligent visual perception sensor, obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix; The processing module is used to obtain visual point cloud data of the nursing target, determine the three-dimensional posture space of each nursing part on the nursing target through the visual point cloud data, and then perform posture calibration on the nursing area that executes the task instruction in each three-dimensional posture space, and obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; The execution module is used to perform deviation fusion on the posture changes of the intelligent manipulator when executing the task instruction according to the spatial deviation and all compensation coefficients, obtain the dynamic posture sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic posture sequence.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned intelligent robotic arm control method for nursing.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned intelligent robot arm control method for nursing.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: In the intelligent manipulator and method for nursing provided in the present application, first, a task instruction is received when the intelligent manipulator performs a nursing operation on a nursing target; based on the intelligent visual perception sensor, feature extraction is performed on all items in the workspace corresponding to the nursing operation to obtain a feature matrix of all items in the workspace, and the spatial deviation of the nursing tools in the workspace is determined according to the geometric features of the nursing tools required to execute the task instruction and the feature matrix; the visual point cloud data of the nursing target is obtained, and the three-dimensional posture space of each nursing part on the nursing target is determined through the visual point cloud data, and then the posture of the nursing area that executes the task instruction is calibrated in each three-dimensional posture space to obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; the posture change of the intelligent manipulator when executing the task instruction is deviation-fused according to the spatial deviation and all the compensation coefficients to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction, and the nursing path of the intelligent manipulator is dynamically adjusted based on the dynamic posture sequence.

[0016] It can be seen that the present application dynamically adjusts the nursing path of the intelligent manipulator based on the dynamic posture sequence; first, determining the spatial deviation can obtain an indicator to measure the deviation between the actual position of the nursing tool in the workspace and the ideal task position. The determination of the spatial deviation helps to evaluate whether the nursing tool is in the correct operating position, thereby ensuring that the intelligent manipulator can successfully obtain the nursing tool and complete the nursing task; then, determining the compensation coefficient can obtain the cost value required by the intelligent manipulator to offset the error caused by the position deviation and posture deviation of the nursing area when performing the nursing task. The determination of the compensation coefficient helps the intelligent manipulator to perform dynamic compensation when performing the task, thereby ensuring Its operation can be adjusted according to the actual position and posture of the nursing target; finally, determining the dynamic posture sequence can obtain a sequence composed of joint change trajectories generated by the end effector of the intelligent manipulator in the process of performing nursing tasks. The dynamic posture sequence reflects how the intelligent manipulator continuously adjusts its own posture during the task execution to adapt to task requirements and environmental changes. The determination of the dynamic posture sequence can avoid the situation where the intelligent manipulator makes the wrong operation area and causes harm to the patient; in summary, based on the above scheme, intelligent vision technology can be used to identify nursing tools and nursing areas and perform dynamic path planning, thereby improving the path adaptability of the intelligent manipulator in nursing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a method for controlling an intelligent manipulator for nursing care according to some embodiments of the present application; Figure 2 is an exemplary flow chart of determining a feature matrix according to some embodiments of the present application; Figure 3 is a partial structural diagram of an intelligent robot for nursing according to some embodiments of the present application; Figure 4 is a schematic diagram of the structure of an intelligent robot for nursing according to some embodiments of the present application; Figure 5 It is a diagram of the internal structure of a computer device for implementing a method for controlling an intelligent manipulator for nursing according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1 , which is an exemplary flow chart of a method for controlling an intelligent manipulator for nursing according to some embodiments of the present application. The method 100 for controlling an intelligent manipulator for nursing mainly includes the following steps: In step 101, a task instruction is received when the intelligent robot performs a nursing operation on a nursing target.

[0020] It should be noted that in the present application, task instructions are specific nursing operation instructions that the intelligent robot arm needs to perform, such as: using disinfectant to clean the patient's wound or using disposable wet wipes to wipe the patient's face, etc.; in specific implementation, obtaining the task instructions that the intelligent robot arm needs to perform can be achieved in the following way, namely: the task instructions issued by the caregiver can be received through the voice input sensor (for example: array microphone) in the intelligent robot arm.

[0021] In step 102, feature extraction is performed on all objects in the workspace corresponding to the nursing operation based on the intelligent visual perception sensor to obtain a feature matrix of all objects in the workspace, and the spatial deviation of the nursing tools in the workspace is determined based on the geometric features of the nursing tools required to execute the task instructions and the feature matrix.

[0022] In some embodiments, reference Figure 2 , which is an exemplary flow chart of determining a feature matrix according to some embodiments of the present application. In the present application, feature extraction is performed on all items in the workspace corresponding to the nursing operation based on an intelligent visual perception sensor, and the feature matrix of all items in the workspace can be obtained by the following steps: First, in step 1021, a workspace image of a nursing operation is acquired; Next, in step 1022, object detection is performed on the workspace image to obtain all auxiliary objects in the workspace; Then, in step 1023, for each auxiliary item, a spatial feature vector of the auxiliary item is determined; Next, in step 1024, the spatial feature vector is compressed by dimensionality reduction to obtain a feature vector of the auxiliary item, and then a feature vector of each auxiliary item is obtained; Finally, in step 1025, a feature matrix of all objects in the workspace is determined based on all feature vectors.

[0023] It should be noted that the smart visual perception sensor (such as the HuskyLens smart visual perception sensor) is an image sensor with integrated artificial intelligence processing capabilities. The smart visual perception sensor has 7 built-in functions, including: face recognition, object tracking, object recognition, line tracking, color recognition, label recognition and object classification, and can perform high-speed edge AI processing, image processing and AI signal processing within the sensor unit; by performing AI processing inside the smart visual sensor, the amount of data transmission is reduced, which helps to protect privacy, reduce security risks and communication costs. In this application, the smart visual perception sensor is used to perform object recognition on all items in the workspace corresponding to the nursing operation, and the data processing unit in the smart visual perception sensor is used to extract features of all items.

[0024] In addition, it should be noted that in the present application, the workspace image is an image that displays the visual information and spatial information of all objects in the workspace, wherein the workspace is the physical space (for example, a ward) in which the intelligent manipulator is located when performing nursing tasks; in specific implementation, obtaining the workspace image set for nursing operations can be achieved in the following manner, namely: the depth camera integrated in the intelligent visual perception sensor on the intelligent manipulator can be used to photograph the workspace to obtain a workspace image.

[0025] In specific implementation, object detection is performed on the workspace image to obtain all auxiliary objects in the workspace, which can be achieved in the following manner, namely: an existing object detection algorithm (for example, an object detection algorithm based on a masked area convolutional neural network) can be used to detect objects on the workspace image and generate a segmented image of each object, and the detected objects are used as auxiliary objects and each auxiliary object is assigned a unique number, for example: No. 0, No. 1, No. 2, etc., thereby obtaining all auxiliary objects in the workspace; wherein the auxiliary objects are related objects in the workspace that can assist the intelligent manipulator to complete nursing tasks, such as: handkerchiefs, syringes, etc.

[0026] It should be noted that, in the present application, the spatial feature vector is a multidimensional feature vector that measures the position, shape and direction of the auxiliary items in the workspace in three-dimensional space, which helps the intelligent manipulator understand the distribution of the auxiliary items in space; in specific implementation, the spatial feature vector of the auxiliary item can be determined in the following way, namely: the position feature and geometric feature of the auxiliary item can be obtained in the segmented image of the auxiliary item through the existing image analysis method (for example: a plane fitting algorithm based on the least squares method), and then the position feature and the geometric feature are integrated into a multidimensional vector, and the multidimensional vector is used as the spatial feature vector of the auxiliary item; wherein, the position feature is a feature vector representing the position coordinates of the auxiliary item in the workspace, and the geometric feature is a feature vector measuring the shape and size attributes of the auxiliary item.

[0027] In specific implementation, the spatial feature vector is compressed by dimensionality reduction to obtain the feature vector of the auxiliary item. This can be achieved in the following manner, namely: the existing dimensionality reduction method (for example, principal component analysis method) can be used to reduce the dimensionality of the spatial feature vector, and the feature vector obtained after dimensionality reduction and compression is used as the feature vector of the auxiliary item; wherein the feature vector is a low-dimensional feature vector that quantifies the position, shape and orientation attributes of the auxiliary object in the workspace, and includes position features, geometric features and orientation features. This feature vector facilitates the intelligent manipulator to determine whether the auxiliary object is the tool required to execute the task instructions, which can help the intelligent manipulator simplify the execution process of the nursing task and improve computing efficiency, ensuring that the intelligent manipulator can respond quickly.

[0028] It should be noted that in the present application, the feature matrix represents a data structure that helps the intelligent manipulator match the features of nursing tools and auxiliary items and determine the spatial features of the nursing tools. The feature matrix helps the intelligent manipulator determine the spatial deviation of the nursing tools in the workspace based on the decision results, and at the same time helps the intelligent manipulator understand the spatial layout of all auxiliary items and adjust the actions and paths according to the spatial layout; in specific implementation, the feature matrix of all items in the workspace based on all feature vectors can be achieved in the following way, namely: the feature vectors of all auxiliary items can be arranged and integrated from small to large according to the numbers of the auxiliary items to obtain a matrix, and the matrix is ​​used as the feature matrix of the items in the workspace.

[0029] In some embodiments, determining the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix can be implemented by the following steps: Determine the geometric features of nursing tools required to perform task instructions; Performing confidence matching on the geometric features and the feature matrix to obtain feature confidence between the nursing tool and each auxiliary item in the workspace; The spatial deviation of the nursing tool in the workspace is determined based on all feature confidences and the spatial position of the intelligent manipulator.

[0030] It should be noted that, in the present application, the geometric features are feature vectors that measure the shape and size attributes of auxiliary items; in specific implementation, the geometric features of the nursing tools required to execute task instructions can be determined in the following manner, namely: the existing natural language processing technology (for example: named entity recognition technology) can be used to perform entity recognition on the task instructions to obtain the nursing tools required to execute the task instructions, and then the geometric features of the nursing tools are obtained from the tool database of the intelligent manipulator; wherein the tool database is a database system that stores specific information of nursing tools that can be operated by the intelligent manipulator, including the geometric features, operating characteristics and usage scenarios of the nursing tools.

[0031] In specific implementation, the geometric features are confidence matched with the feature matrix to obtain the feature confidence between the nursing tool and each auxiliary item in the workspace. This can be achieved in the following manner, namely: for each auxiliary item in the workspace, the geometric features of the auxiliary items can be obtained in the feature matrix, and the existing shape matching algorithm (for example: a shape matching algorithm based on the Hasdorff distance) is used to calculate the matching degree between the geometric features and the geometric features, and the matching degree is used as the feature confidence between the nursing tool and the auxiliary item. The feature confidence between the nursing tool and each auxiliary item in the workspace can be obtained in the above manner; wherein, the feature confidence is a numerical indicator that measures the degree of matching between the geometric features of the nursing tool and the geometric features of the auxiliary items in the workspace, reflecting the degree of similarity between the two in terms of shape and size attributes. The feature confidence can be used to determine the matching relationship between the nursing tool and the items in the workspace, so that the intelligent manipulator can make accurate operation decisions.

[0032] It should be noted that in the present application, the spatial deviation is a parameter that measures the deviation between the actual position of the nursing tool in the workspace and the ideal task position. The spatial deviation can be used to determine whether the nursing tool is in the correct operating position to ensure that the intelligent manipulator can successfully acquire the nursing tool and complete the nursing task. In specific implementation, the spatial deviation of the nursing tool in the workspace is determined based on all feature confidences and the spatial position of the intelligent manipulator, that is: first, the maximum value of all feature confidences can be selected, and the position feature of the auxiliary item corresponding to the maximum value can be obtained in the feature matrix of the above step, and then the spatial position of the end effector of the intelligent manipulator is obtained in the motion controller of the intelligent manipulator. Subsequently, the square root of the sum of the squares of the differences between the position feature and the spatial position in each direction (i.e., horizontal component, vertical direction and depth direction) is used as the spatial deviation of the nursing tool in the workspace.

[0033] In step 103, the visual point cloud data of the care target is obtained, and the three-dimensional posture space of each care part on the care target is determined through the visual point cloud data, and then the posture of the care area executing the task instruction is calibrated in each three-dimensional posture space to obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the care area.

[0034] It should be noted that in the present application, the visual point cloud data is the three-dimensional spatial data of the care target (i.e., the body of the target user) in the workspace, including the geometric shape, position information, and posture information of the care target in the three-dimensional space; in specific implementation, the visual point cloud data of the care target can be obtained in the following manner, namely: the care target can be dynamically tracked and the visual point cloud data can be captured through the built-in three-dimensional imaging device of the intelligent manipulator (for example: lidar, structured light sensor, and stereo vision camera, etc.).

[0035] In some embodiments, determining the three-dimensional position space of each nursing part on the nursing target through the visual point cloud data can be achieved by the following steps: Determine a three-dimensional surface model of a nursing target according to the visual point cloud data; Divide the three-dimensional surface model into regions to obtain all nursing parts in the nursing target; For each nursing part on the nursing target, the three-dimensional posture space of the nursing part is determined according to the partition information of the nursing part in the visual point cloud data, and then the three-dimensional posture space of each nursing part on the nursing target is obtained.

[0036] In specific implementation, determining the three-dimensional surface model of the nursing target based on the visual point cloud data can be achieved in the following manner, namely: the three-dimensional surface model of the nursing target can be constructed through existing point cloud processing technology (for example, Poisson reconstruction or voxel grid filtering); wherein the three-dimensional surface model is a three-dimensional grid model that describes the external contour, shape and structure of the nursing target, and the three-dimensional surface model helps the intelligent manipulator understand the geometric structure of the human body, and provides a basis for posture correction and path planning for subsequent nursing operations.

[0037] It should be noted that in the present application, the nursing parts are functional areas in the nursing target with specific nursing needs, such as: head, upper limbs, lower limbs, chest, back and other areas; in specific implementation, the three-dimensional surface model is divided into regions, and all the nursing parts in the nursing target can be obtained in the following way, namely: a region boundary detection algorithm (for example: region growing algorithm) can be used to divide the three-dimensional surface model into regions according to the anatomical structure of the nursing target to obtain all the nursing parts in the nursing target.

[0038] It should be noted that, in the present application, the three-dimensional pose space is a coordinate domain that represents the position and posture of the care part of the care target in the three-dimensional space, reflecting the geometric distribution and dynamic changes of each part of the care target in the space; in specific implementation, the three-dimensional pose space of the care part can be determined according to the partition information of the care part in the visual point cloud data. It can be implemented in the following way, namely: the partition information of the care part can be first extracted from the visual point cloud data, and then the existing three-dimensional pose estimation algorithm (for example: iterative nearest point algorithm or deep learning-based pose estimation network) is applied to perform pose estimation on the partition information to obtain the spatial position, direction and posture change of the care part, and the spatial position, direction and posture change are mapped to the three-dimensional space coordinate system of the workspace to obtain the three-dimensional pose space of the care part.

[0039] In some embodiments, the posture calibration of the nursing area executing the task instruction is performed in each three-dimensional posture space, and the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area can be achieved by the following steps: Determine the posture characteristics of the nursing area for executing task instructions; For each three-dimensional posture space, the posture feature is aligned with the three-dimensional posture space to obtain a posture deviation between the care area and the three-dimensional posture space; Based on the posture deviation, a compensation coefficient of posture change in the three-dimensional posture space when the intelligent manipulator operates the care area is determined, and then the compensation coefficient of posture change in each three-dimensional posture space when the intelligent manipulator operates the care area is obtained.

[0040] It should be noted that, in the present application, the posture feature is a feature that quantifies the spatial position and posture of the care area in the workspace; in specific implementation, the posture feature of the care area that executes the task instruction can be determined in the following manner, namely: the existing natural language processing technology (for example, named entity recognition technology) can be used to perform entity recognition on the task instruction to obtain the care area that executes the task instruction, and an image segmentation algorithm (for example, a point cloud segmentation algorithm) can be used to extract the regional image of the care area from the workspace image obtained in the above steps, and then the existing image analysis method (for example, a plane fitting algorithm based on the least squares method) can be used to obtain the position coordinates and rotation angle of the care area in the three-dimensional space in the regional image, and the set of the position coordinates and the rotation angle is used as the posture feature of the care area.

[0041] In specific implementation, the posture features are posture-aligned with the three-dimensional posture space, and the posture deviation between the care area and the three-dimensional posture space can be obtained in the following way, namely: a posture alignment method (for example, a rigid transformation algorithm) can be used to find the best match between the posture features of the care area and the three-dimensional posture space and calculate the position error and the posture error, and then the vector composed of the position error and the posture error is used as the posture deviation between the care area and the three-dimensional posture space; wherein the posture deviation represents the difference in position and orientation between the actual posture of the care area and the expected posture in the three-dimensional posture space.

[0042] It should be noted that in the present application, the compensation coefficient is the cost value required to offset the error caused by the position deviation and posture deviation of the nursing area when the intelligent manipulator performs the nursing task. The compensation coefficient helps the intelligent manipulator to perform dynamic compensation when performing the task, ensuring that its operation can be accurately adjusted according to the actual position and posture of the nursing target; in specific implementation, the compensation coefficient of the posture change of the intelligent manipulator in the three-dimensional posture space when operating the nursing area is determined based on the posture deviation and the spatial posture of the intelligent manipulator. The following method can be used, namely: the inverse of the average value of the position error and posture error in the posture deviation can be used as the compensation coefficient for the posture change of the intelligent manipulator in the three-dimensional posture space when operating the nursing area.

[0043] In step 104, deviation fusion is performed on the posture changes of the intelligent manipulator when executing the task instruction according to the spatial deviation and all compensation coefficients to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction, and the nursing path of the intelligent manipulator is dynamically adjusted based on the dynamic posture sequence.

[0044] In some embodiments, the deviation fusion is performed on the posture change of the intelligent manipulator when executing the task instruction according to the spatial deviation degree and all compensation coefficients to obtain the dynamic posture sequence of the intelligent manipulator executing the task instruction, which can be achieved by the following steps: Determine the spatial error between the intelligent manipulator and the nursing area based on all compensation coefficients; The spatial deviation and the spatial error are fused to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction.

[0045] In specific implementation, the spatial error between the intelligent manipulator and the care area can be determined based on all the compensation coefficients in the following manner, namely: the minimum value among all the compensation coefficients can be first selected as the minimum compensation coefficient, and the three-dimensional posture space corresponding to the minimum compensation coefficient can be used as the target posture of the care area, and then the spatial posture of the end effector of the intelligent manipulator (that is, the position coordinates and posture of the end effector of the intelligent manipulator in the three-dimensional space) is obtained in the motion controller of the intelligent manipulator, and the square root of the sum of the squares of the differences between the spatial position in the target posture and the position coordinates in the spatial posture in each direction (that is, the horizontal component, the vertical direction and the depth direction) is used as the spatial error between the intelligent manipulator and the care area; wherein the spatial error is an indicator to measure the deviation between the actual position of the care area in the workspace and the ideal task position.

[0046] It should be noted that, in the present application, the dynamic posture sequence is a sequence composed of joint change trajectories generated by the end effector of the intelligent manipulator in the process of performing nursing tasks. The dynamic posture sequence reflects how the intelligent manipulator continuously adjusts its own posture during the task execution to adapt to task requirements and environmental changes; in specific implementation, the spatial deviation and the spatial error are fused to obtain the dynamic posture sequence of the intelligent manipulator to perform task instructions. It can be implemented in the following way, namely: a kinematic model (for example: an inverse kinematics model) can be loaded, and the spatial deviation and the spatial error are used as the target position deviation in the input parameters of the kinematic model respectively, the kinematic model is executed twice in succession, and the sequence composed of the two joint change trajectories output by the kinematic model is used as the dynamic posture sequence of the intelligent manipulator to perform task instructions.

[0047] In specific implementation, the dynamic adjustment of the nursing path of the intelligent manipulator based on the dynamic posture sequence can be achieved in the following way, namely: the existing dynamic path planning algorithm (for example: A-star search algorithm, dynamic A-star search algorithm, etc.) can be used to generate the optimal nursing path for the intelligent manipulator to perform nursing tasks in the workspace according to the dynamic posture sequence.

[0048] In another aspect of the present application, in some embodiments, the present application provides an intelligent manipulator for nursing, referring to Figure 3 , which is a partial detail diagram of the intelligent manipulator for nursing according to some embodiments of the present application, such as Figure 3As shown in the figure, the part indicated by number 1 is the forearm structure of the intelligent manipulator, which plays the role of supporting and connecting the hand; the part indicated by number 2 is the spherical connector of the rotary joint, which is used to realize the rotation and flexible movement of the palm of the intelligent manipulator and enhance the freedom of movement of the manipulator; the part indicated by number 3 is the palm body of the manipulator, which provides operation support and sensor installation position for the entire manipulator; the part indicated by number 4 is the thumb joint, which can realize flexible grasping and operation, and cooperate with other fingers to complete delicate tasks; the part indicated by number 6 is the sensor embedded in the palm, which is used to monitor the position, posture or contact pressure of the manipulator in real time and provide feedback; the part indicated by number 7 is the execution module installed on the forearm, which is used to issue execution instructions according to the data of the processing module and control the movement of the manipulator.

[0049] In addition, in another aspect of the present application, in some embodiments, the present application provides an intelligent manipulator for nursing, referring to Figure 4 , which is a schematic diagram of the structure of an intelligent manipulator for nursing according to some embodiments of the present application, the intelligent manipulator for nursing 400 includes: a collection module 401, a processing module 402 and an execution module 403, which are described as follows: The acquisition module 401 in this application is mainly used to receive task instructions when the intelligent manipulator performs nursing operations on the nursing target; Processing module 402, in this application, the processing module 402 is mainly used to extract features of all items in the workspace corresponding to the nursing operation based on the intelligent visual perception sensor, obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix; It should be noted that the processing module 402 in the present application is also used to obtain visual point cloud data of the nursing target, determine the three-dimensional posture space of each nursing part on the nursing target through the visual point cloud data, and then perform posture calibration on the nursing area that executes the task instruction in each three-dimensional posture space, and obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; Execution module 403. In the present application, execution module 403 is mainly used to perform deviation fusion on the posture changes of the intelligent manipulator when executing the task instruction according to the spatial deviation and all compensation coefficients, obtain a dynamic posture sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic posture sequence.

[0050] Each module in the above-mentioned intelligent robot for nursing care can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0051] In addition, in one embodiment, the present application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of the control method of the intelligent manipulator for nursing. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a control method of an intelligent manipulator for nursing is implemented.

[0052] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0053] In one embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the above-mentioned embodiment of the intelligent nursing robot control method are implemented.

[0054] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned embodiment of the intelligent nursing robot control method.

[0055] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned embodiment of the control method of the intelligent manipulator for nursing.

[0056] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0057] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for controlling an intelligent manipulator for nursing, characterized in that: The steps include: Receive task instructions when the intelligent manipulator performs nursing operations on nursing targets; Extract features of all items in the workspace corresponding to the nursing operation based on the intelligent visual perception sensor to obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the nursing tools in the workspace according to the geometric features of the nursing tools required to execute the task instructions and the feature matrix; Obtain visual point cloud data of the nursing target, determine the three-dimensional posture space of each nursing part on the nursing target through the visual point cloud data, and then perform posture calibration on the nursing area that executes the task instruction in each three-dimensional posture space, and obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; The position changes of the intelligent manipulator when executing the task instruction are fused according to the spatial deviation and all the compensation coefficients to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction, and the nursing path of the intelligent manipulator is dynamically adjusted based on the dynamic posture sequence.

2. The method according to claim 1, characterized in that Based on the intelligent visual perception sensor, the features of all items in the workspace corresponding to the nursing operation are extracted, and the feature matrix of all items in the workspace is obtained, which specifically includes: Acquire workspace images of nursing operations; Performing object detection on the workspace image to obtain all auxiliary objects in the workspace; For each auxiliary item, determine a spatial feature vector of the auxiliary item; Performing dimensionality reduction compression on the spatial feature vector to obtain a feature vector of the auxiliary item, and then obtaining a feature vector of each auxiliary item; Determine the feature matrix of all objects in the workspace based on all feature vectors.

3. The method according to claim 1, characterized in that Determining the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix specifically includes: Determine the geometric features of nursing tools required to perform task instructions; Performing confidence matching on the geometric features and the feature matrix to obtain feature confidence between the nursing tool and each auxiliary item in the workspace; The spatial deviation of the nursing tool in the workspace is determined based on all feature confidences and the spatial position of the intelligent manipulator.

4. The method according to claim 1, characterized in that Determining the three-dimensional position space of each nursing part on the nursing target through the visual point cloud data specifically includes: Determine a three-dimensional surface model of a nursing target according to the visual point cloud data; Divide the three-dimensional surface model into regions to obtain all nursing parts in the nursing target; For each nursing part on the nursing target, the three-dimensional posture space of the nursing part is determined according to the partition information of the nursing part in the visual point cloud data, and then the three-dimensional posture space of each nursing part on the nursing target is obtained.

5. The method according to claim 1, characterized in that The posture of the nursing area that executes the task instruction is calibrated in each three-dimensional posture space, and the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area is obtained, which specifically includes: Determine the posture characteristics of the nursing area for executing task instructions; For each three-dimensional posture space, the posture feature is aligned with the three-dimensional posture space to obtain a posture deviation between the care area and the three-dimensional posture space; Based on the posture deviation, a compensation coefficient of posture change in the three-dimensional posture space when the intelligent manipulator operates the care area is determined, and then the compensation coefficient of posture change in each three-dimensional posture space when the intelligent manipulator operates the care area is obtained.

6. The method according to claim 1, characterized in that According to the spatial deviation and all compensation coefficients, the position and posture changes of the intelligent manipulator when executing the task instruction are fused to obtain the dynamic posture sequence of the intelligent manipulator executing the task instruction, which specifically includes: Determine the spatial error between the intelligent manipulator and the nursing area based on all compensation coefficients; The spatial deviation and the spatial error are fused to obtain a dynamic posture sequence of the intelligent manipulator executing the task instruction.

7. The method according to claim 1, characterized in that The visual point cloud data is three-dimensional space data of the care target (ie, the body of the target user) in the workspace, including the geometric shape, position information and posture information of the care target in the three-dimensional space.

8. An intelligent manipulator for nursing, characterized in that: include: The acquisition module is used to receive the task instructions when the intelligent manipulator performs nursing operations on the nursing target; A processing module is used to extract features of all items in the workspace corresponding to the nursing operation based on an intelligent visual perception sensor, obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the nursing tool in the workspace according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix; The processing module is used to obtain visual point cloud data of the nursing target, determine the three-dimensional posture space of each nursing part on the nursing target through the visual point cloud data, and then perform posture calibration on the nursing area that executes the task instruction in each three-dimensional posture space, and obtain the compensation coefficient of the posture change in each three-dimensional posture space when the intelligent manipulator operates the nursing area; The execution module is used to perform deviation fusion on the posture changes of the intelligent manipulator when executing the task instruction according to the spatial deviation and all compensation coefficients, obtain the dynamic posture sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic posture sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for controlling the intelligent robotic arm for nursing described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for controlling an intelligent nursing robot as described in any one of claims 1 to 7 are implemented.

Images