Real-time pose acquisition method for modular robots based on deep learning

Abstract

The present invention discloses a real-time pose acquisition method for a modular robot based on deep learning. The method comprises labeling the robot's feature points to obtain point cloud information; downsampling the point cloud and removing discrete points; fitting the point cloud output after the point cloud information processing to obtain a fitted graph of the point cloud based on the point cloud information; and processing the coordinate information of the fitted graph to obtain the coordinate values ​​of the robot's feature points in the world coordinate system. The coordinate values ​​are then processed into a data structure that can be burned into the robot's main control chip, becoming a machine language that the robot can recognize. Upon receiving the recognizable coordinate information, the robot module will execute the corresponding action. The present invention uses a deep learning strategy to train the robot's feature module, giving the robot a strong learning ability and enabling precise positioning of the robot.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a method for acquiring real-time posture of a modular robot based on deep learning. Background Art

[0002] Modularity is a development trend in intelligent robots. Using basic modules with specific functions, robots suitable for specific tasks can be quickly assembled to adapt to different working conditions and mission requirements. For example, a robot can adopt a snake-like or caterpillar-like configuration to traverse narrow holes, transform into a quadruped robot to traverse rough terrain, and even form a ring-like configuration to achieve high-speed rolling motion on flat terrain. These robots have broad application prospects in fields such as medicine, marine, and aerospace. Modular robots have already achieved certain applications in industry and medical rehabilitation, and are gradually entering the market. They currently occupy a place in industrial grasping and rehabilitation robots, becoming a major direction of innovation in robotics technology.

[0003] However, in robot control, due to the uncertainty of the environment and the indeterminateness of the robot's shape, it is difficult to achieve precise control of the robot's various shapes and postures in actual control. Compared with traditional feature extraction and posture coordinate acquisition methods, deep learning can get rid of the dependence on human experience and does not require manual feature design. Through the training of multi-layer neural networks, it can automatically obtain the position and posture of the target object with very high accuracy.

[0004] Patent CN113362388A discloses a deep learning model for target localization and pose estimation. It first inputs a two-dimensional image and outputs the target object's category within the image, its minimum bounding matrix within the image, and the target's position and pose in the world coordinate system. However, this method only uses deep learning to achieve static pose estimation. For dynamic objects, whose morphology and features are highly uncertain, recognition of static objects does not necessarily translate to dynamic objects.

[0005] Patent CN114858158A discloses a deep learning-based mobile robot relocalization method. This method generates a motion path through the mobile robot's real-time data acquisition system, and uses a strategic experience system to evaluate features to achieve posture control and action execution. However, this method requires pre-generated motion paths and empirical data, making it impractical in unknown and complex environments and resulting in poor reuse.

[0006] Although many robots' posture acquisition is based on deep learning, and the position and posture of the target object are acquired through the training of multi-layer neural networks, there are still many problems in actual operation. The posture acquisition is not accurate enough, which affects the normal movement of the robot. Summary of the Invention

[0007] In order to solve the above problems, the present invention provides a real-time posture acquisition method for a modular robot based on deep learning. By performing deep learning training on the robot, the robot's posture can be acquired in real time and real-time control can be achieved.

[0008] To this end, the technical solution of the present invention is: a method for acquiring real-time posture of a modular robot based on deep learning, comprising the following steps:

[0009] 1) Acquisition of robot data features: Marking the robot’s feature points to obtain point cloud information;

[0010] 2) Feature point cloud information processing:

[0011] 2.1) Downsample the point cloud and combine random sampling with farthest point sampling to sample the target point. The specific steps are as follows:

[0012] 2.1.1) Randomly select k points from the target point cloud with a large number of points as the target point cloud, and randomly select a point from these k points as the starting sampling point and add it to the set S. At this time, the set S has a total of 1 point, and the target point cloud set contains k-1 points;

[0013] 2.1.2) Calculate the Euclidean distances from k-1 points in the target point cloud to the set of points S, select the point with the largest Euclidean distance as the new sampling point, and move the new sampling point into the set S. At this point, there are 2 points in the set S, and the target point cloud set contains k-2 points.

[0014] 2.1.3) Calculate the minimum Euclidean distance from the target point cloud set to all points in S, and select the farthest point as a new sampling point to add to the set S. At this point, there are 3 points in the set S, and the target point cloud set contains k-3 points;

[0015] 2.1.4) Repeat steps 2.1.1) to 2.1.3) until the number of points in the set S is the target point k, and the sampling is completed;

[0016] 2.2) Discrete point removal: Straight-through filtering is used to remove discrete points, that is, to filter out points whose values ​​in the specified dimensional direction are not within a given threshold;

[0017] 3) Terminal coordinate fitting: After the point cloud information is processed, the output result of the point cloud is fitted, and the fitting graph of the point cloud can be obtained based on the point cloud information;

[0018] 4) Coordinate information processing and execution of robot actions: After obtaining the fitting graph of the point cloud, the coordinate information of the fitting graph needs to be processed to obtain the coordinate values ​​of the robot's feature points in the world coordinate system. The coordinate values ​​are then processed into a data structure that can be burned into the robot's main control chip, becoming a machine language that the robot can recognize. After receiving the recognizable coordinate information, the robot module will perform the corresponding action.

[0019] On the basis of the above solution and as a preferred solution of the above solution: when marking the characteristic points of the robot in step 1), the following steps are included:

[0020] 1.1) Use a depth camera to obtain the depth values ​​and depth images of the robot's feature points;

[0021] 1.2) Use graphic annotation software to segment and annotate the feature points, and generate a Pyqt file after annotation;

[0022] 1.2.1) Enhance the data to generate similar but not identical samples and expand the dataset;

[0023] 1.2.2) Train the dataset and obtain point cloud information after training.

[0024] On the basis of the above scheme and as a preferred scheme of the above scheme: the enhancement processing of step 1.2.1) includes inverting, rotating, cropping, scaling, increasing color brightness, decreasing color brightness, adjusting saturation, denoising or removing Gaussian noise of the image.

[0025] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step 3), the target point cloud is a circular point cloud, the target point cloud is modeled, the spatial position and morphological parameters of the feature points fitted by the positioning ring point cloud are obtained, and the circular surface where the point cloud is located is fitted using a random sampling consistency algorithm.

[0026] On the basis of the above solution and as a preferred solution of the above solution: the step 3) comprises the following steps:

[0027] 3.1) Randomly select some points from the point cloud as internal fixed points and use the selected point cloud data to fit the model parameters;

[0028] 3.2) Substitute the remaining data into the fitted model, calculate the points that fit the model as inliers, and the points that do not fit the model as outliers;

[0029] 3.3) Calculate the ratio of the number of inliers in the total point cloud and determine whether it is greater than the set threshold. If it is, it is an ideal inlier. If not, continue to iterate until the maximum number of iterations is reached.

[0030] 3.4) Continue to randomly sample the point cloud data and calculate the number of inliers. Compare it with the previous inlier number. If it is greater than the previous inlier number, update the model. If it is less, continue iterating until the loop ends and the optimal model parameters are obtained.

[0031] 3.5) Fit the point cloud torus and bring the obtained feature point parameters and point cloud data into the random sampling consistency algorithm:

[0032] 3.5.1) Randomly select N points from the processed positioning ring point cloud. Based on the two properties of the positioning ring, namely, the distance from any point on the cylindrical surface to the axis is equal to the radius, and the distance from the center point of the cylinder axis to the two end faces is equal, a loss function is established. The least squares method is used to calculate the positioning ring rotation angle and the positioning ring displacement parameters.

[0033] 3.5.2) After obtaining the rotation angle and displacement parameters, construct a homogeneous transformation matrix and express the point cloud coordinates in the coordinate system of the positioning ring. Subtract the distance between the point cloud and the positioning ring axis from the positioning ring radius to calculate the error, set a threshold, and determine whether it is an inlier point.

[0034] 3.5.3) Calculate the percentage of inliers in the total number of points in the point cloud and determine whether to continue iterating. If so, compare the percentage of inliers between the previous and next iterations and update the positioning ring rotation and displacement parameters in real time until the iteration ends.

[0035] 3.5.4) Obtain a circular fitting graph.

[0036] On the basis of the above solution and as a preferred solution of the above solution: in step 3.3), the sampling probability and the number of sampling iterations are calculated as follows:

[0037] Assume that the proportion of inliers in the total number of point clouds is k, then

[0038]

[0039] Then, when calculating N points, the probability that there is at least one sampled outlier is:

[0040]

[0041] After h iterations, the probability of an interior point is:

[0042] P 内点 =1-(1-k N ) h

[0043] Taking the logarithm of the above formula, we can get the number of iterations:

[0044]

[0045] On the basis of the above solution and as a preferred solution of the above solution: in step 4), the coordinate information processing includes the following steps:

[0046] 4.1) Get the ring information of the fitted figure and output the coordinates of the center point of the ring. The coordinate information is a 4*4 matrix array;

[0047] 4.2) The coordinate values ​​obtained by this matrix are the actual coordinate values ​​of the feature points. In the algorithm, multiple loop calculations are performed and the time interval is set to 1s to obtain the dynamic coordinate values ​​of multiple sets of feature points. The zero point of the coordinate value in the world coordinate system is the zero point of the depth camera.

[0048] 4.3) Convert the coordinate values ​​of the robot feature points in the world coordinate system into octal, hexadecimal or higher system;

[0049] 4.4) Reassemble the converted data, select useful data segments, remove useless data segments, and reassemble them together to form a new data structure. This new data structure is then burned into the robot's main control chip, establishing communication with the robot module, and becoming machine language that the robot can recognize;

[0050] 4.5) Each processed coordinate value still corresponds to the original feature data, that is, to the original robot module. The new coordinate value is sent to the original robot module in real time. After receiving the recognizable coordinate information, the robot module will perform the corresponding action.

[0051] On the basis of the above scheme and as a preferred scheme of the above scheme: each feature point has a coordinate matrix, and the first three values ​​of the first row, the first three values ​​of the second row, and the first three values ​​of the third row in the coordinate array constitute a rotation angle matrix, that is, three rotation angle values ​​α, γ, and β. The first three values ​​of the fourth column are the coordinate values ​​x, y, and z of the center point of the ring, which are also the coordinate values ​​of the fitted feature points, and the fourth row is [0, 0, 0, 1].

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. Use deep learning strategies to train the robot's feature modules, giving the robot strong learning capabilities and enabling precise positioning of the robot. Deep learning will not affect recognition due to changes in external scenes, and can accurately identify already marked features.

[0054] 2. The combination of random sampling and farthest point sampling not only improves the sampling accuracy, but also reduces the sampling time to a certain extent, with higher efficiency; filtering the discrete points of different dimensions in the three-dimensional space is conducive to filtering out irrelevant points to a large extent, which can improve the accuracy of fitting;

[0055] 3. By training on a small amount of data, we can obtain experimental results similar to those obtained by training on massive amounts of data, thus achieving the acquisition of feature point coordinates and ensuring the accuracy of the real-time pose of the feature points.

[0056] 4. It can realize the real-time output of the pose coordinates of multiple and different feature points. It can not only identify the three-dimensional coordinates of the robot's feature points in the world coordinate system, but also accurately give the angle information of the rotation matrix. It has strong universality and plays a great role in subsequent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The following is a further detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

[0058] Figure 1 This is a block diagram of the principle of collaborative action of modular robots of the present invention;

[0059] Figure 2 This is a schematic diagram of the deep learning algorithm of the present invention;

[0060] Figure 3 Get an example graph of the coordinate information of a single robot feature point;

[0061] Figure 4 This is an example diagram of a modular robot achieving docking action through feature point target acquisition;

[0062] Figure 5 This is an example diagram of the modular robot's obstacle avoidance action;

[0063] Figure 6 An example diagram of feature point target acquisition when modular robots are combined to form a robotic arm;

[0064] Figure 7 Schematic diagram of point cloud filtering. DETAILED DESCRIPTION

[0065] like Figure 1As shown, the specific implementation principle of the modular robot collaborative action described in this embodiment is divided into three parts: the real-time pose acquisition method. The first part is the robot obtaining the weight value of the feature point (target point) through deep learning; the second part is the robot processing the acquired coordinate information and feeding it back to the robot end; the third part is converting the processed coordinate information into machine language that the robot can recognize, thereby obtaining real-time coordinate information; and finally, executing the collaborative action. In this schematic, the robot's characteristic joints are trained through deep learning, and the trained data is then converted into machine language that the robot can recognize, finally allowing the robot to complete the action.

[0066] Specifically, the method for acquiring the real-time posture of a modular robot described in this embodiment can be divided into four parts for description:

[0067] 1. Acquisition of robot data features:

[0068] The deep learning algorithm used in this embodiment mainly adopts the improved YOLACT algorithm, which is the same as the conventional deep learning algorithm. When performing the YOLACT algorithm, the feature points of the robot must first be marked. The realsense depth camera can be used to obtain the depth value of the robot's feature points, as well as depth pictures, and the feature points are segmented and labeled. When labeling, Labelme graphic labeling software can be used, and a Pyqt file will be generated after labeling.

[0069] During annotation, factors like lighting and size can significantly impact the robot's final feature point recognition. Therefore, data augmentation is essential. This process generates similar but not identical samples, thereby expanding the dataset, reducing the model's reliance on attributes, and improving its generalization capabilities. Data augmentation can be performed through various methods, including image inversion, rotation, cropping, scaling, color brightness adjustment, color saturation adjustment, and Gaussian noise removal.

[0070] After the data set is created, it can be trained. The improved YOLACT algorithm is mainly used for training, and point cloud information is obtained after training.

[0071] 2. Feature point cloud information processing:

[0072] Point cloud processing is mainly divided into point cloud pre-processing, post-processing, and final clustering processing of point clouds.

[0073] The specific steps include:

[0074] 2.1) During point cloud pre-processing, downsampling is performed on the point cloud. This process can reduce the number of point clouds and improve computer execution efficiency.

[0075] Common sampling methods include random sampling, voxel sampling, and farthest point sampling. Each sampling method has its own advantages and disadvantages. This embodiment combines random sampling with farthest point sampling to perform target point sampling. The specific solution is as follows:

[0076] 2.1.1) Randomly select k points from the target point cloud with a large number of points as the target point cloud, and randomly select a point from these k points as the starting sampling point and add it to the set S. At this time, the set S has a total of 1 point, and the target point cloud set becomes k-1 points;

[0077] 2.1.2) Calculate the Euclidean distances from k-1 points in the target point cloud to the set of points S, and select the point with the largest Euclidean distance as the new sampling point. Move the new sampling point into the set S. At this point, the set becomes 2 points, and the target point cloud set becomes k-2 points.

[0078] 2.1.3) Calculate the minimum Euclidean distance from the target point cloud set to all points in S, and select the farthest point as the new sampling point to add to the set S. At this time, the set S becomes 3 points;

[0079] 2.1.4) Repeat the above steps until the number of points in the set S reaches the target point k, and the sampling is complete. Combining these two sampling schemes avoids the situation in which every point cloud in the farthest point sampling is calculated. During the calculation of each point cloud, many points may not be the desired points, which may prevent the point cloud from fitting the desired point cloud array. On the other hand, it avoids the randomness and lack of clustering of sampling points in random sampling, and does not generate too many discrete points.

[0080] 2.2) After completing the pre-processing of the point cloud, it is necessary to remove discrete points. When performing discrete point processing, it is necessary to first perform point cloud denoising.

[0081] In this embodiment, straight-through filtering is used to remove discrete points, that is, to filter out points whose values ​​in the specified dimensional direction are not within a given threshold. That is, data in one directional dimension is selected and a certain range is set. Points within the range will be retained, and points outside the range will be kicked out.

[0082] This embodiment innovatively selects the deletion of points in multiple dimensions after continuous trials, that is, certain thresholds are set in the x, y, and z planes of different dimensions. Points exceeding the threshold will be deleted, and points within the threshold will be retained. Figure 7As shown in the figure, the left picture is the point cloud picture without filtering. It can be found that there are many noise points in the three-dimensional x, y, and z directions, and these noise points are useless for the experiment. The right picture is the figure after three-dimensional filtering. It can be found that the image can clearly display the fitted circular point cloud.

[0083] 3. End coordinate fitting: After completing the corresponding processing of the point cloud, the output result of the point cloud needs to be fitted, and the mask success rate of the point cloud can be obtained based on the point cloud information. The robot structure targeted by this embodiment is as follows: Figure 3 As shown, the two end surfaces of the robot unit are cylindrical structures, and the middle is a flexible structure that can be stretched and bent. Figure 3 The label in the figure is a circular point cloud. By modeling the target point cloud, the spatial position and morphological parameters of the feature points fitted by the positioning ring point cloud can be obtained.

[0084] In this embodiment, the random sampling consensus algorithm Ransac is used to fit the annular surface where the point cloud is located. The specific implementation scheme is as follows:

[0085] 3.1) Randomly select some points from the point cloud as internal fixed points and use the selected point cloud data to fit the model parameters;

[0086] 3.2) Substitute the remaining data into the fitted model, calculate the points that fit the model as inliers, and the points that do not fit the model as outliers;

[0087] 3.3) Calculate the ratio of the number of inliers in the total point cloud and determine whether it is greater than the set threshold. If it is, it is an ideal inlier. If not, continue to iterate until the maximum number of iterations is reached.

[0088] 3.4) Continue to randomly sample and calculate the number of inliers in the point cloud data, and compare it with the previous inlier number. If it is greater than the previous inlier number, update the model. If it is less than the previous inlier number, continue to iterate until the loop ends and the optimal model parameters are obtained. However, after the above iterative method, the number of iterations is limited. Therefore, when implementing the solution, let the proportion of inliers in the total number of point clouds be k, then

[0089]

[0090] Then, when calculating N points, the probability that there is at least one sampling outlier is

[0091] P 外点 =1-k N

[0092] After h iterations, the probability of an interior point is

[0093] P 内点 =1-(1-k N ) h

[0094] Taking the logarithm of the above formula, we can get the number of iterations as

[0095]

[0096] In summary, the sampling probability of the interior point and the number of sampling iterations can be calculated.

[0097] 3.5) Fit the point cloud torus and bring the obtained feature point parameters and point cloud data into the Ransac algorithm. The specific implementation plan is as follows:

[0098] 3.5.1) Randomly select N points from the processed positioning ring point cloud. Based on the two properties that the distance from any point on the cylindrical surface of the positioning ring to the axis is equal to the radius and that the distance from the center point of the cylinder axis to the two end faces is equal, establish a loss function. Use the least squares method to calculate the positioning ring rotation angle and positioning ring displacement parameters.

[0099] 3.5.2) After obtaining the rotation angle and displacement parameters, construct a homogeneous transformation matrix and express the point cloud coordinates in the coordinate system of the positioning ring. Subtract the distance between the point cloud and the positioning ring axis from the positioning ring radius to calculate the error, set a threshold, and determine whether it is an inlier point.

[0100] 3.5.3) Calculate the proportion of inliers in the total number of point clouds and determine whether to continue iterating. If it continues iterating, compare the proportion of inliers in the previous and next iterations and update the rotation and displacement parameters of the positioning ring in real time until the iteration ends. Finally, we can get the following Figure 3 The circular fitting graph in the middle.

[0101] 4. Process coordinate information and execute robot actions:

[0102] After obtaining the above-mentioned fitting graph, it is necessary to process the coordinate information of the fitting graph. The specific implementation scheme is: after obtaining the fitting ring information, the coordinates of the center point of the ring are output, such as Figure 3 Coordinate information in the right image.

[0103] The coordinate information is a 4*4 matrix array, each feature point has a coordinate, and the first three values ​​of the first row, the first three values ​​of the second row, and the first three values ​​of the third row in the coordinate array constitute the rotation angle matrix, that is, the three rotation angle values ​​α, γ, and β. The first three values ​​of the fourth column are the coordinate values ​​x, y, and z of the center point of the ring, which are also the coordinate values ​​of the fitted feature point, and the fourth row is [0,0,0,1].

[0104] The coordinate values ​​obtained by the matrix are the actual coordinate values ​​of the feature points. The algorithm uses multiple loop calculations and sets the time interval to 1s to obtain the dynamic coordinate values ​​of multiple sets of feature points. The zero point of the coordinate value in the world coordinate system is the zero point of the depth camera realsense. In this embodiment, the point in the upper left corner is the zero point. The coordinate value is not constant. When the feature point changes, the coordinate will change accordingly.

[0105] After the above algorithm, the coordinate values ​​of the robot's feature points in the world coordinate system can be obtained. When controlling the robot in real time, the coordinate values ​​need to be processed. The specific processing solution is as follows:

[0106] First, convert the coordinate value into octal or hexadecimal. If the number of digits is insufficient, you can convert it into a higher base.

[0107] Next, the converted base is reassembled, useful data segments are selected, useless data segments are removed, and they are reassembled together to form a new data structure. This new data structure is burned into the robot's main control chip, which can be STM32 or Arduino, etc., to establish communication with the robot module, and become a machine language that the robot can recognize;

[0108] Each processed coordinate value still corresponds to the original feature data, that is, the original robot module, and the new coordinate value is sent to the original robot module. After receiving the coordinate information that can be recognized, the robot module will perform the corresponding action, such as Figure 4 Different robot modules are used to connect the end faces. Figure 5 The robot performs obstacle avoidance and so on.

[0109] Figure 3 This is an example of obtaining coordinate information of feature points of a single robot. The left picture shows a single-module robot diagram and a mask diagram, the middle picture shows a fitted circular diagram, and the right picture shows the implemented coordinates.

[0110] Figure 4 This is an embodiment of a modular robot realizing docking action through feature point target acquisition, wherein the upper, middle and lower parts are respectively one feature point acquisition, two feature point acquisitions, and three feature point acquisitions, and the corresponding coordinate information is output.

[0111] Figure 5 This is an embodiment of a modular robot obstacle avoidance action. The figure shows that the robot's obstacle avoidance can be achieved by displaying real-time three-dimensional coordinate information.

[0112] Figure 6 This is an example of feature point target acquisition when modular robots are combined to form a robotic arm. It can be seen that the robotic arm composed of multiple modules can still achieve stable coordinate output.

[0113] Figure 7 This is an example of a point cloud filtering image. It can be seen that the unfiltered image has multiple noise points, while the noise in the filtered image is significantly reduced.

[0114] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A real-time pose acquisition method for modular robots based on deep learning, characterized by: The following steps are involved: 1) Acquisition of robot data features: Marking the robot’s feature points to obtain point cloud information; 2) Feature point cloud information processing: 2.1) Downsample the point cloud and combine random sampling with farthest point sampling to sample the target point. The specific steps are as follows: 2.1.1) Randomly select k points from the target point cloud with a large number of points as the target point cloud, and randomly select a point from these k points as the starting sampling point and add it to the set S. At this time, the set S has a total of 1 point, and the target point cloud set contains k-1 points; 2.1.2) Calculate the Euclidean distances from k-1 points in the target point cloud to the set of points S, select the point with the largest Euclidean distance as the new sampling point, and move the new sampling point into the set S. At this point, there are 2 points in the set S, and the target point cloud set contains k-2 points. 2.1.3) Calculate the minimum Euclidean distance from the target point cloud set to all points in S, and select the farthest point as a new sampling point to add to the set S. At this point, there are 3 points in the set S, and the target point cloud set contains k-3 points; 2.1.4) Repeat steps 2.1.1) to 2.1.3) until the number of points in the set S is the target point k, and the sampling is completed; 2.2) Discrete point removal: Straight-through filtering is used to remove discrete points, that is, to filter out points whose values ​​in the specified dimensional direction are not within a given threshold; 3) Terminal coordinate fitting: After the point cloud information is processed, the output result of the point cloud is fitted, and the fitting graph of the point cloud can be obtained based on the point cloud information; 4) Coordinate information processing and execution of robot actions: After obtaining the fitting graph of the point cloud, the coordinate information of the fitting graph needs to be processed to obtain the coordinate values ​​of the robot's feature points in the world coordinate system. The coordinate values ​​are then processed into a data structure that can be burned into the robot's main control chip, becoming a machine language that the robot can recognize. After receiving the recognizable coordinate information, the robot module will perform the corresponding action.

2. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 1, characterized in that: When marking the characteristic points of the robot in step 1), the following steps are included: 1.1) Use a depth camera to obtain the depth values ​​and depth images of the robot's feature points; 1.2) Use graphic annotation software to segment and annotate the feature points, and generate a Pyqt file after annotation; 1.2.1) Enhance the data to generate similar but not identical samples and expand the dataset; 1.2.2) Train the dataset and obtain point cloud information after training.

3. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 2, characterized in that: The enhancement processing in step 1.2.1) includes inverting, rotating, cropping, scaling, increasing color brightness, decreasing color brightness, adjusting saturation, or removing Gaussian noise of the image.

4. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 1, wherein: In step 3), the target point cloud is a circular point cloud, the target point cloud is modeled, the spatial position and morphological parameters of the feature points fitted by the positioning ring point cloud are obtained, and the circular surface where the point cloud is located is fitted using a random sampling consistency algorithm.

5. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 4, characterized in that: The step 3) comprises the following steps: 3.1) Randomly select some points from the point cloud as internal fixed points and use the selected point cloud data to fit the model parameters; 3.2) Substitute the remaining data into the fitted model, calculate the points that fit the model as inliers, and the points that do not fit the model as outliers; 3.3) Calculate the ratio of the number of inliers in the total point cloud and determine whether it is greater than the set threshold. If it is, it is an ideal inlier. If not, continue to iterate until the maximum number of iterations is reached. 3.4) Continue to randomly sample the point cloud data and calculate the number of inliers. Compare it with the previous inlier number. If it is greater than the previous inlier number, update the model. If it is less, continue iterating until the loop ends and the optimal model parameters are obtained. 3.5) Fit the point cloud torus and bring the obtained feature point parameters and point cloud data into the random sampling consistency algorithm: 3.5.1) Randomly select N points from the processed positioning ring point cloud. Based on the two properties of the positioning ring, namely, the distance from any point on the cylindrical surface to the axis is equal to the radius, and the distance from the center point of the cylinder axis to the two end faces is equal, a loss function is established. The least squares method is used to calculate the positioning ring rotation angle and the positioning ring displacement parameters. 3.5.2) After obtaining the rotation angle and displacement parameters, construct a homogeneous transformation matrix and express the point cloud coordinates in the coordinate system of the positioning ring. Subtract the distance between the point cloud and the positioning ring axis from the positioning ring radius to calculate the error, set a threshold, and determine whether it is an inlier point. 3.5.3) Calculate the percentage of inliers in the total number of points in the point cloud and determine whether to continue iterating. If so, compare the percentage of inliers between the previous and next iterations and update the positioning ring rotation and displacement parameters in real time until the iteration ends. 3.5.4) Obtain a circular fitting graph.

6. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 5, characterized in that: In step 3.3), the sampling probability and the number of sampling iterations are calculated as follows: Assume that the proportion of inliers in the total number of point clouds is k, then Then, when calculating N points, the probability that there is at least one sampled outlier is: P 外点 =1-k N After h iterations, the probability of an interior point is: P 内点 =1-(1-k N ) h Taking the logarithm of the above formula, we can get the number of iterations:

7. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 1, wherein: In step 4), the coordinate information processing includes the following steps: 4.1) Get the ring information of the fitted figure and output the coordinates of the center point of the ring. The coordinate information is a 4*4 matrix array; 4.2) The coordinate values ​​obtained by this matrix are the actual coordinate values ​​of the feature points. In the algorithm, multiple loop calculations are performed and the time interval is set to 1s to obtain the dynamic coordinate values ​​of multiple sets of feature points. The zero point of the coordinate value in the world coordinate system is the zero point of the depth camera. 4.3) Convert the coordinate values ​​of the robot feature points in the world coordinate system into octal, hexadecimal or higher system; 4.4) Reassemble the converted data, select useful data segments, remove useless data segments, and reassemble them together to form a new data structure. This new data structure is then burned into the robot's main control chip, establishing communication with the robot module, and becoming machine language that the robot can recognize; 4.5) Each processed coordinate value still corresponds to the original feature data, that is, to the original robot module. The new coordinate value is sent to the original robot module in real time. After receiving the recognizable coordinate information, the robot module will perform the corresponding action.

8. The method for real-time pose acquisition of a modular robot based on deep learning according to claim 7, characterized in that: Each feature point has a coordinate matrix, and the first three values ​​of the first row, the first three values ​​of the second row, and the first three values ​​of the third row in the coordinate array constitute the rotation angle matrix, that is, the three rotation angle values ​​α, γ, and β. The first three values ​​of the fourth column are the coordinate values ​​x, y, and z of the center point of the ring, which are also the coordinate values ​​of the fitted feature point, and the fourth row is [0, 0, 0, 1].

Images