An intelligent manipulator and method for nursing

Abstract

The present application provides an intelligent manipulator and method for nursing. By receiving task instructions of the intelligent manipulator; determining the feature matrix of all items in the working space based on an intelligent vision perception sensor, and determining the spatial deviation degree of the nursing tool in the working space according to the geometric features of the nursing tool required for executing the task instructions and the feature matrix; determining the three-dimensional pose space of each nursing part through the visual point cloud data of the nursing target, and further obtaining the compensation coefficient of the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the nursing area; determining the dynamic pose sequence of the intelligent manipulator according to the spatial deviation degree and all the compensation coefficients, and dynamically adjusting the nursing path of the intelligent manipulator based on the dynamic pose sequence. By adopting the solution of the present application, the nursing tool and the nursing area can be identified through artificial intelligence vision technology and dynamic path planning can be performed, thereby improving the path adaptability of the intelligent manipulator in nursing tasks.

Description

Technical Field

[0001] This application relates to the technical field of manipulator program control. More specifically, this application relates to an intelligent manipulator and method for nursing care. Background Art

[0002] With the rapid development of artificial intelligence technology, the application of intelligent manipulators in the field of medical care is gradually increasing. Especially in the aspect of assisting nursing tasks, the intelligent manipulator for nursing care is a device specifically used to assist nursing staff in completing various nursing tasks. Artificial intelligence vision is a technology that combines computer vision and artificial intelligence, aiming to enable machines to understand and analyze the information extracted from visual data and make corresponding intelligent decisions. By integrating artificial intelligence vision technology into the intelligent manipulator for nursing care, real-time environmental perception and decision-making can be carried out using computer vision to assist the intelligent manipulator for nursing care in completing various nursing tasks, providing accurate and efficient services for patients and nursing staff.

[0003] Although the existing intelligent manipulators for nursing care have certain flexibility and intelligent functions, for nursing tasks that require high-precision operations (such as wound cleaning and injection assistance), when performing nursing operations, it is often difficult to accurately identify the spatial positions of nursing tools and nursing areas and dynamically adjust the operation path according to the spatial positions, resulting in positioning deviations during the operation of the intelligent manipulator for nursing care, thus causing harm to patients. Therefore, how to identify nursing tools and nursing areas through artificial intelligence vision technology and perform dynamic path planning, so as to improve the path adaptability of intelligent manipulators in nursing tasks has become a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides an intelligent manipulator and method for nursing care, which can identify nursing tools and nursing areas through artificial intelligence vision technology and perform dynamic path planning, thereby improving the path adaptability of intelligent manipulators in nursing tasks.

[0005] In a first aspect, this application provides an intelligent manipulator control method for nursing care, including:

[0006] Receiving a task instruction when the intelligent manipulator performs a nursing operation on a nursing target;

[0007] Based on the intelligent vision perception sensor, extracting features of all items in the working space corresponding to the nursing operation to obtain a feature matrix of all items in the working space, and determining the spatial deviation degree of the nursing tool in the working space according to the geometric features of the nursing tool required for executing the task instruction and the feature matrix;

[0008] Obtain the visual point cloud data of the nursing target, determine the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data, and then calibrate the pose of the nursing area where the task instruction is executed in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the nursing area;

[0009] According to the spatial deviation degree and all the compensation coefficients, perform deviation fusion on the pose change of the intelligent manipulator when executing the task instruction to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic pose sequence.

[0010] In some embodiments, based on the intelligent vision perception sensor, extract the features of all items in the working space corresponding to the nursing operation, and obtaining the feature matrix of all items in the working space specifically includes:

[0011] Obtain the working space image of the nursing operation;

[0012] Perform item detection on the working space image to obtain all auxiliary items in the working space;

[0013] For each auxiliary item, determine the spatial feature vector of the auxiliary item;

[0014] Reduce the dimension and compress the spatial feature vector to obtain the feature vector of the auxiliary item, and then obtain the feature vector of each auxiliary item;

[0015] Determine the feature matrix of all items in the working space based on all the feature vectors.

[0016] In some embodiments, determining the spatial deviation degree of the nursing tool in the working space according to the geometric features of the nursing tool required for executing the task instruction and the feature matrix specifically includes:

[0017] Determine the geometric features of the nursing tool required for executing the task instruction;

[0018] Perform confidence matching between the geometric features and the feature matrix to obtain the feature confidence degree between the nursing tool and each auxiliary item in the working space;

[0019] Determine the spatial deviation degree of the nursing tool in the working space according to all the feature confidence degrees and the spatial position of the intelligent manipulator.

[0020] In some embodiments, determining the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data specifically includes:

[0021] Determine the three-dimensional surface model of the nursing target according to the visual point cloud data;

[0022] Divide the three-dimensional surface model to obtain all the care parts in the care target;

[0023] For each care part on the care target, determine the three-dimensional pose space of the care part according to the partition information of the care part in the visual point cloud data, and then obtain the three-dimensional pose spaces of all the care parts on the care target.

[0024] In some embodiments, performing pose calibration on the care area for executing the task instruction in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the care area specifically includes:

[0025] Determine the pose characteristics of the care area for executing the task instruction;

[0026] For each three-dimensional pose space, perform pose registration on the pose characteristics and the three-dimensional pose space to obtain the pose deviation between the care area and the three-dimensional pose space;

[0027] Based on the pose deviation, determine the compensation coefficient of the pose change of the intelligent manipulator in the three-dimensional pose space when operating on the care area, and then obtain the compensation coefficients of the pose changes of the intelligent manipulator in each three-dimensional pose space when operating on the care area.

[0028] In some embodiments, performing deviation fusion on the pose change of the intelligent manipulator when executing the task instruction according to the spatial deviation degree and all the compensation coefficients to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction specifically includes:

[0029] Determine the spatial error between the intelligent manipulator and the care area according to all the compensation coefficients;

[0030] Fuse the spatial deviation degree and the spatial error to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction.

[0031] In some embodiments, the visual point cloud data is the three-dimensional space data of the care target (i.e., the body of the target user) in the working space, including the geometric shape, position information, and pose information of the care target in the three-dimensional space.

[0032] In a second aspect, the present application provides an intelligent manipulator for nursing, including:

[0033] An acquisition module, configured to receive the task instruction when the intelligent manipulator performs a nursing operation on the care target;

[0034] A processing module, configured to extract features of all items in the workspace corresponding to the care operation based on an intelligent vision perception sensor, obtain a feature matrix of all items in the workspace, and determine the spatial deviation of the care tool in the workspace according to the geometric features of the care tool required for the task execution instruction and the feature matrix;

[0035] The processing module is configured to obtain visual point cloud data of the care target, determine the three-dimensional pose space of each care part on the care target through the visual point cloud data, and then perform pose calibration on the care area for executing the task instruction in each three-dimensional pose space, so as to obtain a compensation coefficient for the pose change of the intelligent manipulator when operating on the care area in each three-dimensional pose space;

[0036] An execution module, configured to perform deviation fusion on the pose change of the intelligent manipulator when executing the task instruction according to the spatial deviation and all the compensation coefficients, obtain a dynamic pose sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the care path of the intelligent manipulator based on the dynamic pose sequence.

[0037] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned control method for the intelligent manipulator for care.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned control method for the intelligent manipulator for care is implemented.

[0039] The technical solution provided by the disclosed embodiments of the present application has the following beneficial effects:

[0040] In the intelligent robotic arm and method for nursing provided by this application, first, a task instruction is received when the intelligent robotic arm performs a nursing operation on a nursing target; based on the intelligent vision perception sensor, feature extraction is performed on all items in the working space corresponding to the nursing operation to obtain the feature matrix of all items in the working space, and the spatial deviation of the nursing tool in the working space is determined according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix; the visual point cloud data of the nursing target is obtained, and the three-dimensional pose space of each nursing part on the nursing target is determined through the visual point cloud data. Furthermore, pose calibration is performed on the nursing area for executing the task instruction in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent robotic arm when operating on the nursing area in each three-dimensional pose space; according to the spatial deviation and all the compensation coefficients, deviation fusion is performed on the pose change of the intelligent robotic arm when executing the task instruction to obtain the dynamic pose sequence of the intelligent robotic arm when executing the task instruction, and the nursing path of the intelligent robotic arm is dynamically adjusted based on the dynamic pose sequence.

[0041] It can be seen that this application dynamically adjusts the nursing path of the intelligent robotic arm based on the dynamic pose sequence; first, determining the spatial deviation can obtain an index for measuring the deviation between the actual position of the nursing tool in the working space and the ideal task position. The determination of the spatial deviation helps to evaluate whether the nursing tool is in the correct operating position, so as to ensure that the intelligent robotic arm can successfully obtain the nursing tool and complete the nursing task; then, determining the compensation coefficient can obtain the cost value required for the intelligent robotic arm to offset the errors caused by the position deviation and pose deviation of the nursing area when performing the nursing task. The determination of the compensation coefficient helps the intelligent robotic arm to perform dynamic compensation when executing the task, so as to ensure that its operation can be adjusted according to the actual position and pose of the nursing target; finally, determining the dynamic pose sequence can obtain the sequence composed of the joint change trajectories generated by the end effector of the intelligent robotic arm during the execution of the nursing task. This dynamic pose sequence reflects how the intelligent robotic arm continuously adjusts its own posture during the task execution to adapt to the task requirements and environmental changes. The determination of the dynamic pose sequence can avoid the situation that the intelligent robotic arm makes a mistake in the operating area and causes harm to the patient; in summary, based on the above solution, the intelligent vision technology can be used to identify the nursing tool and the nursing area and perform dynamic path planning, so as to improve the path adaptability of the intelligent robotic arm in the nursing task. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an exemplary flowchart of a method for controlling an intelligent robotic arm for nursing according to some embodiments of this application;

[0043] Figure 2 is an exemplary flowchart of determining a feature matrix according to some embodiments of this application;

[0044] Figure 3 It is a partial structure diagram of an intelligent manipulator for nursing shown in some embodiments of the present application;

[0045] Figure 4 It is a schematic structural diagram of an intelligent manipulator for nursing shown in some embodiments of the present application;

[0046] Figure 5 It is an internal structure diagram of a computer device for implementing the control method of an intelligent manipulator for nursing shown in some embodiments of the present application. Specific embodiments

[0047] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0048] Refer to Figure 1 , this figure is an exemplary flowchart of a control method for an intelligent manipulator for nursing shown in some embodiments of the present application. The control method 100 of the intelligent manipulator for nursing mainly includes the following steps:

[0049] In step 101, receive the task instruction when the intelligent manipulator performs a nursing operation on the nursing target.

[0050] It should be noted that in the present application, the task instruction is a specific nursing operation instruction that the intelligent manipulator needs to execute. For example: cleaning the patient's wound with disinfectant or wiping the patient's face with a disposable wet wipe, etc.; specifically, when implemented, obtaining the task instruction that the intelligent manipulator needs to execute can be achieved by the following method, that is: the task instruction issued by the caregiver can be received through the voice input sensor (for example: an array microphone) in the intelligent manipulator.

[0051] In step 102, based on the intelligent vision perception sensor, extract the features of all items in the working space corresponding to the nursing operation, obtain the feature matrix of all items in the working space, and determine the spatial deviation of the nursing tool in the working space according to the geometric features of the nursing tool required to execute the task instruction and the feature matrix.

[0052] In some embodiments, refer to Figure 2 , this figure is an exemplary flowchart of determining the feature matrix shown in some embodiments of the present application. In the present application, extracting the features of all items in the working space corresponding to the nursing operation based on the intelligent vision perception sensor and obtaining the feature matrix of all items in the working space can be achieved by the following steps:

[0053] First, in step 1021, obtain the working space image of the nursing operation;

[0054] Secondly, in step 1022, object detection is performed on the workspace image to obtain all auxiliary objects in the workspace;

[0055] Then, in step 1023, for each auxiliary object, the spatial feature vector of the auxiliary object is determined;

[0056] Next, in step 1024, the spatial feature vector is dimensionally reduced and compressed to obtain the feature vector of the auxiliary object, and thus the feature vector of each auxiliary object is obtained;

[0057] Finally, in step 1025, the feature matrix of all objects in the workspace is determined based on all the feature vectors.

[0058] It should be noted that an intelligent vision perception sensor (such as the HuskyLens intelligent vision perception sensor) is an image sensor integrated with artificial intelligence processing capabilities. This intelligent vision perception sensor has 7 built-in functions, including: face recognition, object tracking, object recognition, line tracking, color recognition, label recognition, and object classification, and can perform high-speed edge AI processing, image processing, and AI signal processing within the sensor unit; by performing AI processing inside the intelligent vision sensor, the amount of data transmission is reduced, which helps to protect privacy, reduce security risks, and communication costs. In this application, the intelligent vision perception sensor is used to perform object recognition on all objects in the workspace corresponding to the nursing operation, and the data processing unit in the intelligent vision perception sensor is used to extract features from all objects.

[0059] In addition, it should be noted that in this application, the workspace image is an image that displays the visual information and spatial information of all objects in the workspace. Among them, the workspace is the physical space (such as a ward) where the intelligent manipulator performs nursing tasks; specifically, when implemented, the workspace image set for the nursing operation can be obtained in the following way, that is: the depth camera integrated in the intelligent vision perception sensor on the intelligent manipulator can be used to capture the workspace where it is located to obtain the workspace image.

[0060] Specifically, when implemented, object detection is performed on the workspace image to obtain all auxiliary objects in the workspace can be achieved in the following way, that is: an existing object detection algorithm (such as an object detection algorithm based on a mask region convolutional neural network) can be used to detect the objects on the workspace image and generate a segmentation image for each object, and the detected objects are used as auxiliary objects and a unique number is assigned to each auxiliary object, for example: No. 0, No. 1, No. 2, etc., so as to obtain all auxiliary objects in the workspace; among them, the auxiliary objects are related objects in the workspace that can assist the intelligent manipulator to complete the nursing task, for example: a towel, a syringe, etc.

[0061] It should be noted that in this application, the spatial feature vector is a multi-dimensional feature vector that measures the position, shape, and orientation of auxiliary items in the workspace. It helps the intelligent manipulator understand the distribution of auxiliary items in space. Specifically, when implemented, the spatial feature vector of the auxiliary item can be determined in the following way: that is, the position feature and geometric feature of the auxiliary item can be obtained from the segmented image of the auxiliary item through existing image analysis methods (for example, the plane fitting algorithm based on the least squares method), and then the position feature and geometric feature are integrated into a multi-dimensional vector, and this multi-dimensional vector is used as the spatial feature vector of the auxiliary item. Among them, the position feature is a feature vector representing the position coordinates of the auxiliary item in the workspace, and the geometric feature is a feature vector that measures the shape and size attributes of the auxiliary item.

[0062] Specifically, when implemented, the dimensionality reduction and compression of the spatial feature vector to obtain the feature vector of the auxiliary item can be achieved in the following way: that is, existing dimensionality reduction methods (for example, the principal component analysis method) can be used to perform dimensionality reduction and compression on the spatial feature vector, and the feature vector obtained after dimensionality reduction and compression is used as the feature vector of the auxiliary item. Among them, the feature vector is a low-dimensional feature vector that quantifies the position, shape, and orientation attributes of the auxiliary object in the workspace, including position features, geometric features, and orientation features. This feature vector is convenient for the intelligent manipulator to determine whether the auxiliary object is the tool required to execute the task instruction, and can help the intelligent manipulator simplify the execution process of the nursing task and improve the calculation efficiency, ensuring that the intelligent manipulator can respond quickly.

[0063] It should be noted that in this application, the feature matrix is a data structure that helps the intelligent manipulator match the features of the nursing tool and the auxiliary item and make a decision on the spatial features of the nursing tool. This feature matrix helps the intelligent manipulator determine the spatial deviation of the nursing tool in the workspace according to the decision result, and at the same time can help the intelligent manipulator understand the spatial layout of all auxiliary items and adjust the actions and paths according to the spatial layout. Specifically, when implemented, the feature matrix of all items in the workspace can be determined based on all the feature vectors in the following way: that is, the feature vectors of all auxiliary items can be arranged and integrated in ascending order according to the numbers of the auxiliary items to obtain a matrix, and this matrix is used as the feature matrix of the items in the workspace.

[0064] In some embodiments, the spatial deviation of the nursing tool in the workspace can be determined according to the geometric feature of the nursing tool required to execute the task instruction and the feature matrix by the following steps:

[0065] Determine the geometric feature of the nursing tool required to execute the task instruction;

[0066] Perform a confidence matching between the geometric features and the feature matrix to obtain the feature confidence between the care tool and each auxiliary item in the working space;

[0067] Determine the spatial deviation of the care tool in the working space based on all the feature confidences and the spatial position of the intelligent manipulator.

[0068] It should be noted that in this application, the geometric feature is a feature vector that measures the shape and size attributes of the auxiliary item; in specific implementation, the geometric feature of the care tool required to execute the task instruction can be implemented in the following way, that is: first, use the existing natural language processing technology (for example: named entity recognition technology) to perform entity recognition on the task instruction to obtain the care tool required to execute the task instruction, and then obtain the geometric feature of the care tool in the tool database of the intelligent manipulator; among them, the tool database is a database system that stores the specific information of the care tools that the intelligent manipulator can operate, including the geometric features, operation characteristics and usage scenarios of the care tools.

[0069] In specific implementation, performing a confidence matching between the geometric features and the feature matrix to obtain the feature confidence between the care tool and each auxiliary item in the working space can be implemented in the following way, that is: for each auxiliary item in the working space, the geometric feature of the auxiliary item can be obtained in the feature matrix, and the existing shape matching algorithm (for example: shape matching algorithm based on Hausdorff distance) can be used to calculate the matching degree between this geometric feature and this geometric feature, and use this matching degree as the feature confidence between the care tool and the auxiliary item. Through the above method, the feature confidence between the care tool and each auxiliary item in the working space can be obtained; among them, the feature confidence is a numerical index that measures the matching degree between the geometric feature of the care tool and the geometric feature of the auxiliary item in the working space, reflecting the similarity degree in terms of shape and size attributes, etc. This feature confidence can be used to determine the matching relationship between the care tool and the items in the working space, so that the intelligent manipulator can make accurate operation decisions.

[0070] It should be noted that in this application, the spatial deviation degree is a parameter that measures the deviation between the actual position of the nursing tool in the working space and the ideal task position. This spatial deviation degree can be used to determine whether the nursing tool is in the correct operating position to ensure that the intelligent manipulator can successfully acquire the nursing tool and complete the nursing task. Specifically, when implemented, the spatial deviation degree of the nursing tool in the working space is determined according to all the feature confidence degrees and the spatial position of the intelligent manipulator, that is: First, the maximum value among all the feature confidence degrees can be selected, and the position feature of the auxiliary item corresponding to this maximum value is obtained in the feature matrix of the above steps. Then, the spatial position of the end effector of the intelligent manipulator is obtained in the motion controller of the intelligent manipulator. Subsequently, the square root of the sum of the squares of the component differences of this position feature and this spatial position in each direction (i.e., the horizontal direction component, the vertical direction, and the depth direction) is used as the spatial deviation degree of the nursing tool in the working space.

[0071] In step 103, the visual point cloud data of the nursing target is acquired, and the three-dimensional pose space of each nursing part on the nursing target is determined through the visual point cloud data. Furthermore, the pose calibration of the nursing area for executing the task instruction is performed in each three-dimensional pose space, and the compensation coefficient of the pose change of the intelligent manipulator when operating on the nursing area in each three-dimensional pose space is obtained.

[0072] It should be noted that in this application, the visual point cloud data is the three-dimensional space data of the nursing target (i.e., the body of the target user) in the working space, including the geometric shape, position information, and attitude information of the nursing target in the three-dimensional space. Specifically, when implemented, the visual point cloud data of the nursing target can be acquired in the following manner, that is: The three-dimensional imaging device (such as: lidar, structured light sensor, and stereo vision camera, etc.) built in the intelligent manipulator can be used to dynamically track the nursing target and capture the visual point cloud data.

[0073] In some embodiments, the three-dimensional pose space of each nursing part on the nursing target can be determined through the visual point cloud data by the following steps:

[0074] Determine the three-dimensional surface model of the nursing target according to the visual point cloud data;

[0075] Perform regional division on the three-dimensional surface model to obtain all the nursing parts in the nursing target;

[0076] For each nursing part on the nursing target, determine the three-dimensional pose space of the nursing part according to the partition information of the nursing part in the visual point cloud data, and then obtain the three-dimensional pose space of each nursing part on the nursing target.

[0077] In specific implementation, the three-dimensional surface model of the care target determined according to the visual point cloud data can be implemented in the following manner, that is: the three-dimensional surface model of the care target can be constructed by existing point cloud processing technologies (such as Poisson reconstruction or voxel grid filtering); the three-dimensional surface model is a three-dimensional grid model that describes the external contour, shape and structure of the care target, and this three-dimensional surface model helps the intelligent manipulator understand the geometric structure of the human body and provides a basis for attitude correction and path planning during subsequent execution of care operations.

[0078] It should be noted that in this application, the care part is a functional area with specific care needs in the care target, such as: areas of the head, upper limbs, lower limbs, chest, back, etc.; in specific implementation, the three-dimensional surface model can be divided into regions to obtain all care parts in the care target in the following manner, that is: the region boundary detection algorithm (such as the region growing algorithm) can be used to divide the three-dimensional surface model according to the anatomical structure of the care target to obtain all care parts in the care target.

[0079] It should be noted that in this application, the three-dimensional pose space is a coordinate domain that represents the position and pose of the care part of the care target in the three-dimensional space, reflecting the geometric distribution and dynamic changes of each part of the care target in space; in specific implementation, the three-dimensional pose space of the care part can be determined according to the partition information of the care part in the visual point cloud data in the following manner, that is: the partition information of the care part can be extracted from the visual point cloud data first, and then the existing three-dimensional pose estimation algorithm (such as the iterative closest point algorithm or the pose estimation network based on deep learning) can be applied to estimate the pose of the partition information to obtain the spatial position, direction and pose change of the care part, and map the spatial position, direction and pose change to the three-dimensional space coordinate system of the working space to obtain the three-dimensional pose space of the care part.

[0080] In some embodiments, the pose calibration of the care area for executing the task instruction in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the care area can be implemented by the following steps:

[0081] Determine the pose characteristics of the care area for executing the task instruction;

[0082] For each three-dimensional pose space, register the pose characteristics with the three-dimensional pose space to obtain the pose deviation between the care area and the three-dimensional pose space;

[0083] Based on the pose deviation, determine the compensation coefficient of the pose change of the intelligent manipulator in the three-dimensional pose space when operating on the care area, and then obtain the compensation coefficient of the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the care area.

[0084] It should be noted that in this application, the pose feature is a feature that quantifies the spatial position and attitude of the nursing area in the workspace. When specifically implemented, the pose feature of the nursing area for executing the task instruction can be implemented in the following manner, that is: First, the existing natural language processing technology (such as: named entity recognition technology) can be used to perform entity recognition on the task instruction to obtain the nursing area for executing the task instruction, and an image segmentation algorithm (such as: point cloud segmentation algorithm) can be used to extract the regional image of the nursing area from the workspace image obtained in the above step. Then, the position coordinates and rotation angle of the nursing area in the three-dimensional space can be obtained in the regional image through the existing image analysis method (such as: plane fitting algorithm based on the least square method), and the set of the position coordinates and the rotation angle is used as the pose feature of the nursing area.

[0085] When specifically implemented, the pose registration of the pose feature with the three-dimensional pose space to obtain the pose deviation between the nursing area and the three-dimensional pose space can be implemented in the following manner, that is: A pose registration method (such as: rigid transformation algorithm) can be used to find the best match between the pose feature of the nursing area and the three-dimensional pose space and calculate the position error and attitude error, and then the vector composed of the position error and the attitude error is used as the pose deviation between the nursing area and the three-dimensional pose space; wherein, the pose deviation represents the difference in position and orientation between the actual pose of the nursing area and the expected pose in the three-dimensional pose space.

[0086] It should be noted that in this application, the compensation coefficient is the cost value required for the intelligent manipulator to offset the error caused by the position deviation and attitude deviation of the nursing area when performing the nursing task. This compensation coefficient helps the intelligent manipulator to perform dynamic compensation during the task execution to ensure that its operation can be precisely adjusted according to the actual position and attitude of the nursing target. When specifically implemented, based on the pose deviation and the spatial pose of the intelligent manipulator, the compensation coefficient for the pose change of the intelligent manipulator when operating on the nursing area in the three-dimensional pose space can be implemented in the following manner, that is: The reciprocal of the average value of the position error and the attitude error in the pose deviation can be used as the compensation coefficient for the pose change of the intelligent manipulator when operating on the nursing area in the three-dimensional pose space.

[0087] In step 104, according to the spatial deviation degree and all the compensation coefficients, the pose change of the intelligent manipulator when executing the task instruction is subjected to deviation fusion to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction, and the nursing path of the intelligent manipulator is dynamically adjusted based on the dynamic pose sequence.

[0088] In some embodiments, the deviation fusion of the pose change of the intelligent manipulator when executing a task instruction according to the spatial deviation degree and all compensation coefficients to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction can be implemented by the following steps:

[0089] Determine the spatial error between the intelligent manipulator and the nursing area according to all compensation coefficients;

[0090] Fuse the spatial deviation degree and the spatial error to further obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction.

[0091] Specifically, when implemented, determining the spatial error between the intelligent manipulator and the nursing area according to all compensation coefficients can be implemented in the following manner, that is: first select the minimum value among all compensation coefficients as the minimum compensation coefficient, and use the three-dimensional pose space corresponding to this minimum compensation coefficient as the target pose of the nursing area. Then, obtain the spatial pose of the end effector of the intelligent manipulator in the motion controller of the intelligent manipulator (that is, the position coordinates and pose of the end effector of the intelligent manipulator in the three-dimensional space), and take the square root of the sum of the squares of the component differences of the spatial position in the target pose and the position coordinates in this spatial pose in each direction (that is, the horizontal direction component, the vertical direction, and the depth direction) as the spatial error between the intelligent manipulator and the nursing area; wherein, the spatial error is an index for measuring the deviation between the actual position of the nursing area in the working space and the ideal task position.

[0092] It should be noted that in this application, the dynamic pose sequence is a sequence composed of the joint change trajectories generated by the end effector of the intelligent manipulator during the execution of the nursing task. This dynamic pose sequence reflects how the intelligent manipulator continuously adjusts its own posture during the task execution to adapt to the task requirements and environmental changes; specifically, when implemented, fusing the spatial deviation degree and the spatial error to further obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction can be implemented in the following manner, that is: a kinematic model (for example: inverse kinematic model) can be loaded, and the spatial deviation degree and the spatial error are respectively used as the target position deviations in the input parameters of the kinematic model. The kinematic model is executed twice successively, and the sequence composed of the two joint change trajectories output by the kinematic model is used as the dynamic pose sequence of the intelligent manipulator when executing the task instruction.

[0093] Specifically, when implemented, dynamically adjusting the nursing path of the intelligent manipulator based on the dynamic pose sequence can be implemented in the following manner, that is: existing dynamic path planning algorithms (for example: A* search algorithm, dynamic A* search algorithm, etc.) can be used to generate the optimal nursing path for the intelligent manipulator to execute the nursing task in the working space according to this dynamic pose sequence.

[0094] On the other hand of the present application, in some embodiments, the present application provides an intelligent manipulator for nursing, refer to Figure 3 , this figure is a partial detail view of the intelligent manipulator for nursing shown in some embodiments of the present application, such as Figure 3 shown, the part indicated by reference numeral 1 is the forearm structure of the intelligent manipulator, which plays a role in supporting and connecting the hand; the part indicated by reference numeral 2 is the rotary joint spherical connector, which is used to realize the rotation and flexible movement of the palm of the intelligent manipulator, and enhance the movement freedom of the manipulator; the part indicated by reference numeral 3 is the main body of the palm of the manipulator, which provides operation support and sensor installation positions for the entire manipulator; the part indicated by reference numeral 4 is the thumb joint, which can realize flexible grasping and operation, and cooperate with other fingers to complete fine tasks; the part indicated by reference numeral 6 is the sensor embedded in the palm, which is used to monitor the position, posture or contact pressure of the manipulator in real time and provide feedback; the part indicated by reference numeral 7 is the execution module installed on the forearm, which is used to issue execution instructions according to the data of the processing module and control the movement of the manipulator.

[0095] In addition, on the other hand of the present application, in some embodiments, the present application provides an intelligent manipulator for nursing, refer to Figure 4 , this figure is a schematic structural view of the intelligent manipulator for nursing shown in some embodiments of the present application. The intelligent manipulator for nursing 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are described as follows:

[0096] Acquisition module 401, in the present application, the acquisition module 401 is mainly used to receive task instructions when the intelligent manipulator performs nursing operations on the nursing target;

[0097] Processing module 402, in the present application, the processing module 402 is mainly used to extract features of all items in the working space corresponding to the nursing operation based on the intelligent vision perception sensor, obtain the feature matrix of all items in the working space, and determine the spatial deviation degree of the nursing tool in the working space according to the geometric features of the nursing tool required for executing the task instruction and the feature matrix;

[0098] It should be noted that in the present application, the processing module 402 is further used to obtain the visual point cloud data of the nursing target, determine the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data, and then perform pose calibration on the nursing area for executing the task instruction in each three-dimensional pose space, so as to obtain the compensation coefficient of the pose change of the intelligent manipulator when operating on the nursing area in each three-dimensional pose space;

[0099] Execution module 403. In this application, the execution module 403 is mainly used to perform deviation fusion on the pose change of the intelligent manipulator when executing a task instruction according to the spatial deviation degree and all compensation coefficients, obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic pose sequence.

[0100] Each module in the above intelligent manipulator for nursing can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0101] In addition, in one embodiment, this application provides a computer device, which can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of the control method of the intelligent manipulator for nursing. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a control method of an intelligent manipulator for nursing.

[0102] Those skilled in the art can understand that Figure 5 the structure shown in

[0103] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0104] In one embodiment, there is also provided a computer device, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the above embodiment of the control method of the intelligent manipulator for nursing.

[0105] In one embodiment, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions that are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the embodiment of the above-mentioned intelligent manipulator control method for nursing.

[0106] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0107] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0108] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An intelligent manipulator control method for nursing, characterized in that, It includes the following steps: Receive the task instructions when the intelligent manipulator performs nursing operations on the nursing target; Based on the intelligent vision perception sensor, extract the features of all items in the working space corresponding to the nursing operation to obtain the feature matrix of all items in the working space, and determine the spatial deviation of the nursing tool in the working space according to the geometric features of the nursing tool required to execute the task instructions and the feature matrix; Obtain the visual point cloud data of the nursing target, determine the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data, and then perform pose calibration on the nursing area for executing the task instructions in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent manipulator when operating on the nursing area in each three-dimensional pose space; According to the spatial deviation and all compensation coefficients, perform deviation fusion on the pose change of the intelligent manipulator when executing the task instructions to obtain the dynamic pose sequence of the intelligent manipulator for executing the task instructions, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic pose sequence.

2. The method according to claim 1, characterized in that, Based on the intelligent vision perception sensor, extracting the features of all items in the working space corresponding to the nursing operation to obtain the feature matrix of all items in the working space specifically includes: Obtain the working space image of the nursing operation; Perform item detection on the working space image to obtain all auxiliary items in the working space; For each auxiliary item, determine the spatial feature vector of the auxiliary item; Reduce the dimensionality and compress the spatial feature vector to obtain the feature vector of the auxiliary item, and then obtain the feature vector of each auxiliary item; Based on all the feature vectors, determine the feature matrix of all items in the working space.

3. The method according to claim 1, characterized in that Determine the spatial deviation of the nursing tool in the working space according to the geometric features of the nursing tool required to execute the task instructions and the feature matrix specifically includes: Determine the geometric features of the nursing tool required to execute the task instructions; Perform confidence matching between the geometric features and the feature matrix to obtain the feature confidence between the nursing tool and each auxiliary item in the working space; Determine the spatial deviation of the nursing tool in the working space according to all the feature confidences and the spatial position of the intelligent manipulator.

4. The method according to claim 1, characterized in that Determine the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data specifically includes: Determine the three-dimensional surface model of the nursing target according to the visual point cloud data; Perform regional division on the three-dimensional surface model to obtain all nursing parts in the nursing target; For each nursing part on the nursing target, determine the three-dimensional pose space of the nursing part according to the partition information of the nursing part in the visual point cloud data, and then obtain the three-dimensional pose space of each nursing part on the nursing target.

5. The method according to claim 1, characterized in that, Perform pose calibration on the nursing area for executing the task instructions in each three-dimensional pose space to obtain the compensation coefficient of the pose change of the intelligent manipulator when operating on the nursing area in each three-dimensional pose space specifically includes: Determine the pose characteristics of the nursing area for executing the task instructions; For each three-dimensional pose space, perform pose registration between the pose characteristics and the three-dimensional pose space to obtain the pose deviation between the nursing area and the three-dimensional pose space; Based on the pose deviation, determine the compensation coefficient for the pose change of the intelligent manipulator in the three-dimensional pose space when operating on the nursing area, and then obtain the compensation coefficients for the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the nursing area.

6. The method according to claim 1, wherein Perform deviation fusion on the pose change of the intelligent manipulator when executing the task instruction according to the spatial deviation degree and all the compensation coefficients to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction, specifically including: Determine the spatial error between the intelligent manipulator and the nursing area according to all the compensation coefficients; Fuse the spatial deviation degree and the spatial error, and then obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction.

7. The method according to claim 1, characterized in that The visual point cloud data is the three-dimensional spatial data of the nursing target (i.e., the body of the target user) in the working space, including the geometric shape, position information, and pose information of the nursing target in the three-dimensional space.

8. An intelligent manipulator for nursing, characterized in that, Include: An acquisition module, configured to receive the task instruction when the intelligent manipulator performs a nursing operation on the nursing target; A processing module, configured to perform feature extraction on all items in the working space corresponding to the nursing operation based on the intelligent vision perception sensor to obtain the feature matrix of all items in the working space, and determine the spatial deviation degree of the nursing tool in the working space according to the geometric features of the nursing tool required for executing the task instruction and the feature matrix; The processing module is configured to obtain the visual point cloud data of the nursing target, determine the three-dimensional pose space of each nursing part on the nursing target through the visual point cloud data, and then perform pose calibration on the nursing area where the task instruction is executed in each three-dimensional pose space to obtain the compensation coefficients for the pose change of the intelligent manipulator in each three-dimensional pose space when operating on the nursing area; An execution module, configured to perform deviation fusion on the pose change of the intelligent manipulator when executing the task instruction according to the spatial deviation degree and all the compensation coefficients to obtain the dynamic pose sequence of the intelligent manipulator when executing the task instruction, and dynamically adjust the nursing path of the intelligent manipulator based on the dynamic pose sequence.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the intelligent manipulator control method for nursing described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the intelligent manipulator control method for nursing described in any one of claims 1 to 7 are implemented.

Images