A method and device for human-computer coordination spatial judgment based on the density of nearest neighbors in point clouds

Abstract

This invention relates to a method and apparatus for determining human-machine coordination space based on the density of nearest neighbor points in point clouds. The method involves establishing a human upper limb model and a rehabilitation robot model; generating a first point cloud for the human upper limb workspace and a second point cloud for the rehabilitation robot's robotic arm workspace using the Monte Carlo method; filtering the first and second point clouds to obtain a third and fourth point cloud; mixing the third and fourth point clouds to obtain a fifth composite point cloud; calculating the relative density of each point in the third, fourth, and fifth composite point clouds based on the k-nearest neighbor algorithm; and finding the first, second, and third nearest neighbors of the motion input point to obtain the first, second, and third relative densities to determine whether the motion input point is within the human-machine coordination space. This invention's method for determining human-machine coordination space can quickly generate human-machine coordination space range information from point clouds with high accuracy, and uses the relative density information of the point clouds to determine the position of the input point within the human-machine coordination space range. It also features fast calculation speed and accurate positioning of the human-machine coordination space range.

Description

Technical Field

[0001] This invention relates to the field of rehabilitation robot kinematics technology, and in particular to a method and device for human-machine coordination spatial judgment based on the density of nearest neighbor points in point clouds. Background Technology

[0002] The human upper limbs possess strong manipulative abilities and complex coordination, capable of performing a series of actions and combinations such as grasping, releasing, pushing, and pulling, and are an important component of overall human motor ability. With the accelerating aging of the population and increasing life expectancy, the number of patients suffering from limb disabilities due to muscle weakness, cardiovascular diseases, and other conditions is increasing. Currently, effective treatments for limb disabilities involve rehabilitation of damaged nerves and muscles through exercise, thereby achieving limb recovery. However, due to the complex structure of the upper limb joints, rehabilitation requires long-term, meticulous, and repetitive rehabilitation training to achieve therapeutic effects. In recent years, with the rapid development of robotic hardware and control technology, and the shortage of professional medical rehabilitation and nursing personnel for such diseases, rehabilitation robots have gradually become an important solution to this problem.

[0003] In most cases, robots are connected to the patient's body and work closely with them in the same workspace. Therefore, the safety of human-robot interaction in rehabilitation robots is a critical issue. As a rehabilitation training device, unsafe control of rehabilitation robots can cause training injuries and secondary harm to patients. Numerous such incidents have occurred in recent years, making the safety of human-robot interaction a hot research topic in academia and industry. Currently, some exoskeleton-type upper limb rehabilitation robots are designed with a simulation of the mapping relationship between the robot's workspace and the human upper limb's workspace, ensuring overall adaptability and safety in their mechanical structure design. However, some rehabilitation robots, especially traction-type robots, have higher adaptability, making it difficult to improve the safety of human-robot interaction from a mechanical structure design perspective. Ensuring that the robot's end effector moves within a safe human-robot coordination space for rehabilitation training poses significant safety risks when patients use these robots for active or mediated rehabilitation training. In the existing technology, rehabilitation robots also lack methods for determining the spatial range of the robot's end effector at the control level. This poses a significant safety hazard to the machine interaction of traction robots, hindering the development and application of such rehabilitation robots.

[0004] Determining the human-robot coordination space can be equated to determining the range of the collaborative space between two robotic arms. Common computational methods involve solving for the collaborative space of each arm separately using numerical inverse kinematics or geometric inverse kinematics. However, traditional inverse kinematics methods are often limited by the number of degrees of freedom of the robotic arms, the speed of inverse kinematics solutions, and the tendency to produce singular solutions. Therefore, some researchers have applied optimization algorithms to improve upon the shortcomings of traditional inverse kinematics methods. However, these methods, when determining whether the robot's end effector is within the human-robot coordination space, are slow to compute and prone to errors, thus failing to meet the real-time requirements of a safety control system for rapid spatial determination.

[0005] Therefore, there is an urgent need for a method that can quickly and accurately determine the relative spatial range of the human-machine coordination space where the end-effector of an upper limb rehabilitation robot is located, so as to provide a kinematic basis for safety control. Summary of the Invention

[0006] The purpose of this invention is to provide a method and device for determining human-machine coordination space based on the density of nearest neighbors in point clouds, so as to accurately and quickly determine the range of human-machine coordination space of upper limb rehabilitation robots according to rehabilitation patients of different body types, provide a kinematic basis for the safe control of rehabilitation robots, and thus improve the safety of upper limb rehabilitation robots at the control level.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide a human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud, specifically including:

[0008] A human upper limb model was created using a depth camera, and a rehabilitation robot model was also created.

[0009] Based on the human upper limb model and the rehabilitation robot model, the Monte Carlo method is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace.

[0010] Filtering the first and second point clouds yields a third and fourth point cloud with uniform density and size.

[0011] Based on the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot, the third point cloud and the fourth point cloud are mixed to obtain the fifth synthetic point cloud with changes in point cloud density gradient.

[0012] The relative density of each point in the third, fourth, and fifth composite point clouds is calculated based on the k-nearest neighbor algorithm.

[0013] In calculating the third point cloud, the fourth point cloud, and the fifth composite point cloud, the motion input point is substituted into the calculation. The k-nearest neighbor algorithm is used to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively.

[0014] Determine whether the motion input point is within the human-machine coordination space based on the first relative density, the second relative density, and the third relative density.

[0015] Preferably, the relative density of each point in the third, fourth, and fifth composite point clouds is calculated based on the k-nearest neighbor algorithm, specifically including:

[0016] Traverse each point in the third, fourth, and fifth composite point clouds, search for the neighboring points of the current point according to the k-nearest neighbor algorithm, and use the reciprocal of the distance between the current point and the nearest neighbor as the relative density of the current point.

[0017] Preferably, for a point p in the point cloud to be calculated, p represents the current point, q represents any other point in the point cloud where p is located, N is the amount of data in the point cloud to be calculated, dis(p,q) represents the distance between point p and point q, and d p Let p represent the minimum distance between point p and other points in the point cloud to be calculated. Then, the relative density of the current point is:

[0018] .

[0019] Preferably, determining whether the motion input point is within the human-machine coordination space based on the first relative density, the second relative density, and the third relative density specifically includes:

[0020] If the third relative density is greater than the second relative density, it means that the motion input point is within the human-machine coordination space; otherwise, it is determined that the motion input point is not within the human-machine coordination space.

[0021] Preferably, the first and second point clouds are filtered using a voxel center filter, specifically including:

[0022] Calculate the minimum bounding box side length based on the first point cloud and the second point cloud respectively;

[0023] Based on the octree structure, three-dimensional voxel grids of the first point cloud and the second point cloud are constructed respectively.

[0024] The non-empty voxels in the first and second point clouds are traversed separately. The center of the non-empty voxels is calculated based on the number of rows and columns of the non-empty voxels, resulting in the third and fourth point clouds with uniform density and consistent size.

[0025] Preferably, the center of the non-empty voxel is calculated based on the number of rows and columns of the non-empty voxel, specifically including:

[0026] The center point of each voxel is used to approximate all other points within the voxel. For any point within a non-empty voxel, the minimum coordinate x of the voxel is calculated based on its row, column, layer, and the minimum coordinate x of its voxel. min y min z min The center of the voxel is calculated to be C. center (x center ,y center ,z center )for:

[0027] ,

[0028] Where r is the set voxel side length.

[0029] Preferably, the generation of the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot's robotic arm workspace specifically includes:

[0030] The forward kinematic equations of the human upper limb are obtained by solving the human upper limb model, which is specifically the human upper limb DH parameter.

[0031] The forward kinematic equations of the rehabilitation robot are obtained by solving the model of the rehabilitation robot, and the model of the rehabilitation robot is the DH parameter of the rehabilitation robot.

[0032] Random values ​​are randomly selected from the upper and lower limits of each joint in the human upper limb and the rehabilitation robot for traversal;

[0033] By substituting the random values ​​of the human upper limb joints into the forward kinematics equations of the human upper limb, and by substituting the random values ​​of the rehabilitation robot joints into the forward kinematics equations of the rehabilitation robot, the set of position coordinate points of the upper limb and the set of position coordinate points of the rehabilitation robot end effector are obtained.

[0034] Preferably, values ​​are randomly selected from the upper and lower limits of each joint in the human upper limb and rehabilitation robot for iteration, specifically including: ,

[0035] g p,limb Let g be a random value obtained for the angle of the p-th joint of the human upper limb. q,robot Let g be a random value obtained for the q-th joint angle of the rehabilitation robot. p,min G represents the minimum angle of the p-th joint. q,min Let g represent the minimum angle of the q-th joint. p,max G represents the maximum angle of the p-th joint. q,maxThis represents the maximum angle of the q-th joint, where RAND is a random number between 0 and 1.

[0036] Preferably, a human upper limb model is created based on a depth camera, specifically including:

[0037] Hand-eye calibration was performed on the depth camera and the rehabilitation robot to obtain the coordinate transformation relationship between the pixel coordinate system and the rehabilitation robot coordinate system. The depth camera was used to capture the joint coordinates of the human upper limb.

[0038] By using a depth camera to take multiple photos of the upper limbs of the rehabilitation patients, the joint data and three-dimensional coordinates of the upper limbs of the rehabilitation patients were obtained, and the lengths of the shoulder girdle, humerus and forearm of the upper limbs were calculated.

[0039] The simplified kinetic chain of the upper limb of the rehabilitation patient was obtained. Based on the length of the scapular girdle, humerus and forearm of the human upper limb and the limit range of motion of the joints of the human upper limb, the simplified DH parameters of the human upper limb were obtained.

[0040] The present invention also provides a human-machine coordinated spatial judgment device based on the density of nearest neighbors in a point cloud, comprising:

[0041] The modeling module is used to create human upper limb models and rehabilitation robot models based on depth cameras.

[0042] The point cloud generation module is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace based on the human upper limb model and the rehabilitation robot model using the Monte Carlo method.

[0043] The filtering module is used to filter the first and second point clouds to obtain a third and fourth point cloud with uniform density and consistent size.

[0044] The synthesis module is used to combine the third and fourth point clouds to obtain a fifth synthesized point cloud with varying point cloud density gradients, based on the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot.

[0045] The density calculation module is used to calculate the relative density of each point in the third point cloud, the fourth point cloud, and the fifth composite point cloud based on the k-nearest neighbor algorithm.

[0046] The search and judgment module is used to substitute the motion input point into the calculation of the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively, and use the k-nearest neighbor algorithm to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively. Based on the first relative density, the second relative density, and the third relative density, it is determined whether the motion input point is within the human-machine coordination space.

[0047] Compared to existing technologies, the present invention provides a method and apparatus for determining human-machine coordination space based on the density of nearest neighbors in point clouds. By utilizing point clouds to determine the range of the human-machine coordination space at the upper limb structural level within a spatial range, it improves the speed and accuracy of solving for the human-machine coordination space range, thereby enhancing the operational safety of rehabilitation robots and meeting current rehabilitation training requirements. It directly utilizes the variation characteristics of point cloud density in the human-machine coordination space to quickly determine whether the motion input point is within the human-machine coordination space based on the relative density value. It uses the attribute information of the point cloud to locate the range of the human-machine coordination space, avoiding the use of complex algorithms for position analysis and judgment in three-dimensional space. The entire judgment process has low computational load and high speed. Furthermore, the present invention can establish corresponding upper limb models for rehabilitation patients of different body types, thereby obtaining accurate corresponding human-machine coordination spaces. This provides a kinematic basis for the safe control of rehabilitation robots, helping developers improve the safety of upper limb rehabilitation robots from a control perspective.

[0048] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0049] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0050] Figure 1 This is a flowchart of the human-computer coordination spatial judgment method based on the density of nearest neighbors in point clouds according to the present invention;

[0051] Figure 2 This is a schematic diagram of the structure of each workspace in this invention;

[0052] Figure 3 These are simulation diagrams of the various workspaces of this invention;

[0053] Figure 4 This is a schematic diagram of the coordinate systems of the present invention;

[0054] Figure 5 A schematic diagram showing the actual human joints corresponding to the joint coordinate points captured by the depth camera.

[0055] Figure 6 Schematic diagram of human joints and rehabilitation robot joints;

[0056] Figure 7This is a schematic diagram of the first point cloud before voxel filtering;

[0057] Figure 8 This is a schematic diagram of the third point cloud after the first point cloud voxel filtering;

[0058] Figure 9 This is a schematic diagram of the second point cloud before voxel filtering;

[0059] Figure 10 This is a schematic diagram of the fourth point cloud after the second point cloud voxel filtering;

[0060] Figure 11 This is a schematic diagram of the fifth composite point cloud;

[0061] Figure 12 This is a schematic diagram showing the relative density of each point in the fifth composite point cloud;

[0062] Figure 13 A schematic diagram showing the spatial relationship between the motion input point and its nearest neighbors;

[0063] Figure 14 This is a schematic diagram of a human-machine coordination spatial judgment device based on the density of nearest neighbor points in a point cloud, provided as an embodiment of the present invention. Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of the present invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0065] The first embodiment of the present invention relates to a human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud, the flowchart of which is shown below. Figure 1 As shown, this embodiment uses a six-axis traction rehabilitation robot developed based on the Aubo i5, with a Microsoft Kinect v2 depth camera, and the rehabilitation subject is an adult male with a height of 175cm. When the rehabilitation robot does not use the human-machine coordination space judgment method provided in this invention for active rehabilitation tasks, it cannot quickly and accurately obtain the range of the human-machine coordination space, and therefore cannot quickly determine whether the motion input point is within the upper limb's activity space. This results in a situation where the assistance range exceeds the human upper limb's working space, posing a significant safety hazard to the rehabilitation patient. Figure 2 , Figure 3As shown, the workspace of this invention includes a robotic arm workspace and a human upper limb workspace, with the intersection of the two forming a human-machine coordination space. Figure 2 This is a schematic diagram of the workspace structure. Figure 3 This is a schematic diagram of the workspace simulation.

[0066] A human upper limb model was created using a depth camera, and a rehabilitation robot model was then developed.

[0067] The human upper limb model specifically refers to the DH parameters of the human upper limb, while the rehabilitation robot model refers to the DH parameters of the rehabilitation robot. These parameters can be obtained by consulting the rehabilitation robot's manual.

[0068] Creating a human upper limb model using a depth camera specifically includes:

[0069] Hand-eye calibration was performed on the depth camera and the rehabilitation robot to obtain the coordinate transformation relationship between the pixel coordinate system and the rehabilitation robot coordinate system. The depth camera was used to capture the joint coordinates of the human upper limb.

[0070] Each coordinate system is as follows Figure 4 As shown, the pose descriptions of other local coordinate systems within the system relative to the camera coordinate system {C}, x0y0z0, are as follows: (Rehabilitation robot base coordinate system) , Robotic arm end-effector coordinate system , and the baseline coordinate system of the rehabilitation subjects , .

[0071] By using a depth camera to take multiple photos of the upper limbs of the rehabilitation patients, the joint data and three-dimensional coordinates of the upper limbs of the rehabilitation patients were obtained, and the lengths of the shoulder girdle, humerus and forearm of the upper limbs were calculated.

[0072] like Figure 5 As shown, the joint data of the human upper limb specifically includes the coordinates of the spine-scapular joint, shoulder joint, elbow joint, wrist joint, and palm: coordinates of the spine-scapular joint J1(i1,j1,k1), shoulder joint J2(i2,j2,k2), elbow joint J3(i3,j3,k3), wrist joint J4(i4,j4,k4), and palm J5(i5,j5,k5). The length L of the human upper limb scapular girdle is then calculated. s Length of humerus L u Forearm length L f And the length L from wrist to palm h :

[0073]

[0074] The simplified kinetic chain of the upper limb of the rehabilitation patient was obtained. Based on the length of the scapular girdle, humerus and forearm of the human upper limb and the limit range of motion of the joints of the human upper limb, the simplified DH parameters of the human upper limb were obtained.

[0075] To obtain the simplified kinetic chain of the upper limb of the rehabilitation patient, specifically, the simplified kinetic chain of the upper limb is obtained by ignoring the internal / external rotation degrees of freedom of the elbow joint and the motion joints. Since the human joints captured by the depth camera can only correspond to a few rotational joints in the upper limb, and cannot capture the motion joints, a simplified kinetic chain related to upper limb movement is obtained by ignoring the above information.

[0076] Based on the simplified kinetic chain of the upper limb, the upper and lower limits of each joint in the human upper limb and the rehabilitation robot are obtained. The joint positions of the human upper limb and the rehabilitation robot are as follows: Figure 6 As shown. The upper and lower limits of the angles of each joint in the human upper limb are as follows: {g 1,max =0°, g 2,max =0°, g 3,max =-10°, g 4,max =-60°, g 5,max =-60°, g 6,max =0°, g 7,max =0°}

[0077] {g 1,min =0°, g 2,min =0°, g 3,min =170°, g 4,min =170°, g 5,min =90°, g 6,min =90°, g 7,min =0°},

[0078] Among them, {g n,max |n=1,2,3,…,f}、{g m,min |m=1,2,3,…,f} represent the upper limit of the angle of the nth joint and the lower limit of the angle of the mth joint, respectively.

[0079] Due to individual differences in joints and muscles, the spatial structure of the upper limbs varies from patient to patient. Using a uniform human anatomy model for active rehabilitation training can easily lead to secondary injuries during the procedure due to discrepancies between the model and the actual patient's upper limb space, significantly reducing the precision of safety control and operational safety. The method provided in this invention utilizes a depth camera to establish a human upper limb structural model, improving the system's adaptability.

[0080] Based on the human upper limb model and the rehabilitation robot model, the Monte Carlo method is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace.

[0081] Specifically, random values ​​are traversed within the range of motion of each joint of the human upper limb and the rehabilitation robot. The forward kinematic equations are solved based on the human upper limb model and the rehabilitation robot model. The coordinate positions of the human upper limb end and the rehabilitation robot end are calculated based on the random values ​​and the forward kinematic equations, generating the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace.

[0082] Furthermore, generating the first and second point clouds specifically includes:

[0083] Based on the human upper limb model, the forward kinematic equations of the human upper limb are obtained:

[0084] ;

[0085] Where the subscript of g indicates the nth joint, and g1-g7 represent the biological angles of joints 1-7 respectively: g1 represents the sternoclavicular joint 1, g2 represents the sternoclavicular joint 2, g3 represents the shoulder joint 1, g4 represents the shoulder joint 2, g5 represents the shoulder joint 3, g6 represents the elbow joint, and g7 represents the wrist joint. The transformation matrix is... The transformation matrix represents the biological angles of the corresponding joints of the human upper limbs.

[0086] The forward kinematic equations of the rehabilitation robot are obtained by solving the model of the rehabilitation robot:

[0087] ;

[0088] f represents the degrees of freedom of the rehabilitation robot, f=1,2,3,…,7, g f This represents the biological angle of the f-th joint. Let f represent the transformation matrix of the f-th joint, where the transformation matrix is ​​a function of the biological angles of the f-th joint. Let be the transformation matrix.

[0089] Randomly select values ​​from the upper and lower limits of the joint angles of the human upper limb and the rehabilitation robot and iterate through them: ,

[0090] g p,limb Let g be a random value obtained for the angle of the p-th joint of the human upper limb. q,robot Let g be a random value obtained for the q-th joint angle of the rehabilitation robot. p,min G represents the minimum angle of the p-th joint.q,min Let g represent the minimum angle of the q-th joint. p,max G represents the maximum angle of the p-th joint. q,max This represents the maximum angle of the q-th joint, where RAND is a random number between 0 and 1.

[0091] The random values ​​of the human upper limb joints are substituted into the forward kinematics equation of the human upper limb, and the random values ​​of the rehabilitation robot joints are substituted into the forward kinematics equation of the rehabilitation robot to obtain the set of position coordinate points of the upper limb and the set of position coordinate points of the rehabilitation robot end effector.

[0092] The set of location coordinate points constitutes the workspace point cloud, that is, the first point cloud P of the human upper limb workspace. i,limb The second point cloud P in the workspace of the robotic arm i,robot : ,

[0093] Among them, i limb i robot These represent the data volume of the first point cloud and the data volume of the second point cloud, respectively. limb =i robot .

[0094] After generating the first and second point clouds, filtering is performed on the first and second point clouds to obtain the third and fourth point clouds with uniform density and consistent size.

[0095] Filtering is achieved using a voxel-centered filter. The third point cloud is the filtered data from the first point cloud, and the fourth point cloud is the filtered data from the second point cloud. Figure 7-10 As shown. Filtering the first and second point clouds specifically includes:

[0096] Calculate the minimum bounding box side lengths for the first and second point clouds respectively.

[0097] The minimum bounding box side length is calculated by finding the maximum value x of the X, Y, and Z axes in the point cloud coordinate set. max y max z max With the minimum value x min y min z min The difference is used to calculate the side length l of the minimum bounding box. x l y l z ;

[0098] Based on the octree structure, three-dimensional voxel grids of the first point cloud and the second point cloud are constructed respectively.

[0099] Specifically, the voxel grids of the first and second point clouds have the same side length, r. limb =r robot This ensures that the density of the two sets of point clouds output in the subsequent process is consistent.

[0100] The non-empty voxels in the first and second point clouds are traversed separately. The center of the non-empty voxels is calculated based on the number of rows and columns of the non-empty voxels, resulting in the third and fourth point clouds with uniform density and consistent size.

[0101] Specifically, for any point within a non-empty voxel, the minimum coordinate x of the voxel is calculated based on the row, column, and layer numbers (row, col, lay) of the voxel. min y min z min The center of the voxel is found to be C. center (x center ,y center ,z center )for: ,

[0102] Where r is the set voxel side length.

[0103] By traversing all non-empty voxels in the first and second point clouds respectively, and using the center point of each voxel to approximate all other points within the voxel, a third point cloud P with uniform density and size is finally obtained. k,limb And the fourth point cloud P k,robot k limb k robot These represent the data volumes of the third and fourth point clouds, respectively.

[0104]

[0105] Based on the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot, the third and fourth point clouds are mixed to obtain the fifth synthetic point cloud with changes in point cloud density gradient.

[0106] Based on the pose relationship between the rehabilitation subject's base coordinate system {G} and the camera coordinate system {C} The pose relationship between the base coordinate system {B} of the rehabilitation robot and the camera coordinate system {G} By performing coordinate translation on all points in both sets of point clouds, a fifth composite point cloud with varying point cloud density gradients is obtained. For example... Figure 11 As shown, the denser part of the fifth synthetic point cloud is the human-machine coordination space.

[0107] The relative density of each point in the third, fourth, and fifth composite point clouds was calculated based on the k-nearest neighbor algorithm.

[0108] For each point in the third, fourth, and fifth composite point clouds, the k-nearest neighbor algorithm is used to search for the neighboring points of the current point. The reciprocal of the distance between the current point and its nearest neighbor is used as the relative density of the current point. Specifically, for a point p in the point cloud to be calculated, p represents the current point, q represents any other point in the point cloud to be calculated where p is located, N is the amount of data in the point cloud to be calculated, dis(p,q) represents the distance between point p and point q, and d p Let p represent the minimum distance between point p and other points in the point cloud to be calculated. Then, the relative density of the current point is:

[0109] .

[0110] In a uniformly distributed point cloud d p This is a constant value, meaning that the relative density ρ of all points within the point cloud can be represented by the same value, 1 / d. p To represent; in the fifth composite point cloud, d p The larger the value of d, the smaller the relative density ρ of the point cloud; p The smaller the value, the greater the relative density ρ of the point cloud, such as... Figure 12 As shown in the figure, the color intensity of each point indicates the relative density of each point in the fifth composite point cloud.

[0111] In calculating the third point cloud, the fourth point cloud, and the fifth composite point cloud, the motion input point is substituted into the calculation. The k-nearest neighbor algorithm is used to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively.

[0112] The real-time motion coordinates of the end effector of the rehabilitation robot are sampled to obtain the three-dimensional coordinates of the motion input point, which is S. 输入 (x s ,y s ,z s Using the k-nearest neighbor algorithm, the first, second, and third nearest neighbors of the motion input point in the third, fourth, and fifth composite point clouds are found respectively. Within a point cloud, the nearest neighbors of the motion input point are found, and their spatial relationships are as follows: Figure 13 As shown. The first relative density ρ is obtained based on the calculated relative density. S,limb Second relative density ρ S,robot and the third relative density ρ S,combine Among them, since the density of the third point cloud and the fourth point cloud are equal, ρ S,limb= ρ S,robot .

[0113] Determine whether the motion input point is within the human-machine coordination space based on the first relative density, the second relative density, and the third relative density.

[0114] If the third relative density is greater than the second relative density, i.e., ρ S,combine >ρ S,robot If the value is positive, it means the motion input point is within the human-machine coordination space; otherwise, it is determined that the motion input point is not within the human-machine coordination space.

[0115] This patent provides a method for determining the human-machine coordination space based on the density of nearest neighbors in a point cloud. It directly utilizes the changing characteristics of point cloud density in the human-machine coordination space to quickly determine whether a motion input point is within the human-machine coordination space based on the relative density value. This avoids the need for complex algorithms in three-dimensional space for position analysis and judgment. The entire judgment process involves low computational load, fast judgment speed, and high positioning accuracy. Furthermore, this invention can establish corresponding upper limb models for rehabilitation patients of different body types, thereby obtaining accurate corresponding human-machine coordination spaces, improving the system's adaptability and providing a kinematic basis for the safe control of rehabilitation robots.

[0116] Embodiment 2 of the present invention also provides a human-machine coordination spatial judgment device based on the density of nearest neighbors in a point cloud, such as... Figure 14 As shown, it includes:

[0117] The modeling module is used to create human upper limb models and rehabilitation robot models based on depth cameras.

[0118] The point cloud generation module is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace based on the human upper limb model and the rehabilitation robot model using the Monte Carlo method.

[0119] The filtering module is used to filter the first and second point clouds to obtain a third and fourth point cloud with uniform density and consistent size.

[0120] The synthesis module is used to import the third and fourth point clouds into the intermediate coordinate system according to the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot, so as to obtain the fifth synthesized point cloud with the change of point cloud density gradient.

[0121] The density calculation module is used to calculate the relative density of each point in the third point cloud, the fourth point cloud, and the fifth composite point cloud based on the k-nearest neighbor algorithm.

[0122] The search and judgment module is used to substitute the motion input point into the calculation of the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively, and use the k-nearest neighbor algorithm to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively. Based on the first relative density, the second relative density, and the third relative density, it is determined whether the motion input point is within the human-machine coordination space.

[0123] It is not difficult to see that this embodiment is a device embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0124] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0125] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud, characterized in that, The human-computer coordination spatial judgment method based on the density of nearest neighbors in point clouds specifically includes: A human upper limb model was created using a depth camera, and a rehabilitation robot model was also created. Based on the human upper limb model and the rehabilitation robot model, the Monte Carlo method is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace. Filtering the first and second point clouds yields a third and fourth point cloud with uniform density and size. Based on the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot, the third point cloud and the fourth point cloud are mixed to obtain the fifth synthetic point cloud with changes in point cloud density gradient. Traverse each point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, search for the neighboring points of the current point according to the k-nearest neighbor algorithm, and use the reciprocal of the distance between the current point and the nearest neighbor as the relative density of the current point; In calculating the third point cloud, the fourth point cloud, and the fifth composite point cloud, the motion input point is substituted into the calculation. The k-nearest neighbor algorithm is used to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively. Determine whether the motion input point is within the human-machine coordination space based on the first relative density, the second relative density, and the third relative density; Specifically, determining whether the motion input point is within the human-machine coordination space based on the first relative density, the second relative density, and the third relative density includes: If the third relative density is greater than the second relative density, it means that the motion input point is within the human-machine coordination space; otherwise, it is determined that the motion input point is not within the human-machine coordination space.

2. The human-computer coordination spatial judgment method based on the density of nearest neighbors in point clouds according to claim 1, characterized in that, For a point p in the point cloud to be computed, p represents the current point, q represents any other point in the point cloud where p is located, N is the amount of data in the point cloud to be computed, dis(p,q) represents the distance between point p and point q, and d represents the distance between point p and point q. p Let p represent the minimum distance between point p and other points in the point cloud to be calculated. Then, the relative density of the current point is: 。 3. The human-computer coordination spatial judgment method based on the density of nearest neighbors in point clouds according to claim 1, characterized in that, The first and second point clouds are filtered using a voxel center filter, specifically including: Calculate the minimum bounding box side length based on the first point cloud and the second point cloud respectively; Based on the octree structure, three-dimensional voxel grids of the first point cloud and the second point cloud are constructed respectively. The non-empty voxels in the first and second point clouds are traversed separately. The center of the non-empty voxels is calculated based on the number of rows and columns of the non-empty voxels, resulting in the third and fourth point clouds with uniform density and consistent size.

4. The human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud, as described in claim 3, is characterized in that... The calculation of the center of a non-empty voxel based on the number of rows and columns of the non-empty voxel specifically includes: The center point of each voxel is used to approximate all other points within the voxel. For any point within a non-empty voxel, the minimum coordinate x of the voxel is calculated based on its row, column, layer, and the minimum coordinate x of its voxel. min y min z min The center of the voxel is calculated to be C. center (x center ,y center ,z center )for: ； Where r is the set voxel side length.

5. The human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud as described in claim 1, characterized in that, The generation of the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot's robotic arm workspace specifically includes: The forward kinematic equations of the human upper limb are obtained by solving the human upper limb model, which is specifically the human upper limb DH parameter. The forward kinematic equations of the rehabilitation robot are obtained by solving the model of the rehabilitation robot, and the model of the rehabilitation robot is the DH parameter of the rehabilitation robot. Random values ​​are randomly selected from the upper and lower limits of each joint in the human upper limb and the rehabilitation robot for traversal; By substituting the random values ​​of the human upper limb joints into the forward kinematics equations of the human upper limb, and by substituting the random values ​​of the rehabilitation robot joints into the forward kinematics equations of the rehabilitation robot, the set of position coordinate points of the upper limb and the set of position coordinate points of the rehabilitation robot end effector are obtained.

6. The human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud, as described in claim 5, is characterized in that... The step of randomly selecting values ​​from the upper and lower limits of each joint in the human upper limb and rehabilitation robot for traversal specifically includes: ； g p,limb Let g be a random value obtained for the angle of the p-th joint of the human upper limb. q,robot Let g be a random value obtained for the q-th joint angle of the rehabilitation robot. p,min Let g represent the minimum angle of the p-th joint. q,min Let g represent the minimum angle of the q-th joint. p,max G represents the maximum angle of the p-th joint. q,max This represents the maximum angle of the q-th joint, where RAND is a random number between 0 and 1.

7. The human-computer coordination spatial judgment method based on the density of nearest neighbors in a point cloud as described in claim 1, characterized in that, The process of creating a human upper limb model based on a depth camera specifically includes: Hand-eye calibration was performed on the depth camera and the rehabilitation robot to obtain the coordinate transformation relationship between the pixel coordinate system and the rehabilitation robot coordinate system. The depth camera was used to capture the joint coordinates of the human upper limb. By using a depth camera to take multiple photos of the upper limbs of the rehabilitation patients, the joint data and three-dimensional coordinates of the upper limbs of the rehabilitation patients were obtained, and the lengths of the shoulder girdle, humerus and forearm of the upper limbs were calculated. The simplified kinetic chain of the upper limb of the rehabilitation patient was obtained. Based on the length of the scapular girdle, humerus and forearm of the human upper limb and the limit range of motion of the joints of the human upper limb, the simplified DH parameters of the human upper limb were obtained.

8. A human-machine coordinated spatial judgment device based on the density of nearest neighbors in a point cloud, characterized in that, The human-machine coordination space judgment device includes: The modeling module is used to create human upper limb models and rehabilitation robot models based on depth cameras. The point cloud generation module is used to generate the first point cloud of the human upper limb workspace and the second point cloud of the rehabilitation robot robotic arm workspace based on the human upper limb model and the rehabilitation robot model using the Monte Carlo method. The filtering module is used to filter the first and second point clouds to obtain a third and fourth point cloud with uniform density and consistent size. The synthesis module is used to combine the third and fourth point clouds to obtain a fifth synthesized point cloud with varying point cloud density gradients, based on the base coordinate system of the rehabilitation object and the spatial coordinate system of the rehabilitation robot. The density calculation module is used to traverse each point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, search for the neighboring points of the current point according to the k-nearest neighbor algorithm, and use the reciprocal of the distance between the current point and the nearest neighbor as the relative density of the current point. The search and judgment module is used to substitute the motion input point into the calculation of the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively, and use the k-nearest neighbor algorithm to find the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor of the motion input point in the third point cloud, the fourth point cloud, and the fifth composite point cloud, respectively. The relative densities of the first nearest neighbor, the second nearest neighbor, and the third nearest neighbor are obtained respectively. Based on the first relative density, the second relative density, and the third relative density, it is determined whether the motion input point is within the human-machine coordination space.

Images