Robot pose positioning evaluation method and device, terminal and storage medium

Abstract

The invention provides a robot pose positioning evaluation method and device, a terminal and a storage medium, and the method comprises the steps: calculating pose data when a robot moves to a target pose state in response to a control instruction according to the control instruction received by the robot, and obtaining to-be-evaluated pose data; acquiring actual test data, acquired by the plurality of sensors, of the robot in the target pose state; generating corresponding virtual test data based on the actual test data, and constructing a test data set based on the actual test data and the virtual test data; and training target evaluation data based on the test data set, and evaluating whether the pose positioning of the robot is accurate based on the target evaluation data and the pose data to be evaluated. According to the method, the actual measurement data and the virtual data are combined, and the data of the multiple sensors are fused to evaluate the pose positioning of the robot, so that the problem that the pose measurement accuracy and reliability are limited due to dependence on a single sensor in a traditional pose test method can be solved.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a robot posture positioning evaluation method, device, terminal and storage medium. Background Art

[0002] At present, traditional posture measurement methods often rely on a single sensor, such as a laser rangefinder, a visual sensor or an inertial measurement unit (IMU), and each of these sensors has significant shortcomings, which limits the accuracy and reliability of posture measurement.

[0003] For example, although laser rangefinders have the ability to measure long distances with high precision, their measurement range is limited and they are easily affected by environmental factors such as light, dust, and obstacles. In addition, laser rangefinders are expensive and are not suitable for all application scenarios. For another example, visual sensors can obtain the position and posture information of robots through image processing technology, but the accuracy of visual sensors is affected by factors such as light, shadows, occlusions, and image resolution. As a result, in complex or dynamic environments, the performance of visual sensors may be significantly reduced, thereby reducing the accuracy of position and posture information. For another example, inertial measurement units calculate the position and posture of robots by measuring acceleration and angular velocity, but IMU data has measurement deviations and noise, and may undergo integral drift over time, resulting in inaccurate positioning and navigation.

[0004] Therefore, the prior art has defects and needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a robot posture positioning evaluation method, device, terminal and storage medium are provided, aiming to solve the problem that traditional posture testing methods rely on a single sensor to limit the accuracy and reliability of posture measurement.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] A robot posture positioning evaluation method, wherein the method comprises:

[0008] Calculating the posture data of the robot when it moves to the target posture state in response to the control instruction according to the control instruction received by the robot, and obtaining the posture data to be evaluated;

[0009] Acquire actual test data of the robot in the target posture state collected by multiple sensors;

[0010] Generate corresponding virtual test data based on the actual test data, and construct a corresponding test data set based on the actual test data and the virtual test data;

[0011] Target evaluation data is trained based on the test data set, and whether the posture positioning of the robot is accurate is evaluated based on the target evaluation data and the posture data to be evaluated.

[0012] In one implementation, the generating corresponding virtual test data based on the actual test data includes:

[0013] Performing trend analysis on the actual test data to obtain corresponding trend analysis results;

[0014] Corresponding virtual test data is generated according to the trend analysis result.

[0015] In one implementation, the step of constructing a corresponding test data set based on the actual test data and the virtual test data includes:

[0016] The actual test data and the virtual test data are selected according to a preset data ratio to construct a corresponding test data set.

[0017] In one implementation, training target evaluation data based on the test data set includes:

[0018] Assigning corresponding initial weights to the test data in the test data set, and performing weighted fusion processing on the test data in the test data set to obtain fused evaluation data;

[0019] Calculating the dynamic deviation between the fused evaluation data and the actual test data, and determining whether the dynamic deviation meets a preset deviation condition;

[0020] When the dynamic deviation does not meet the preset deviation condition, the initial weight is adjusted based on the dynamic deviation to obtain an adjusted weight, and new actual test data collected by the multiple sensors when the robot responds to the control instruction and moves to the target posture state is reacquired;

[0021] Re-performing the step of generating corresponding virtual test data based on the actual test data based on the new actual test data, and constructing a corresponding test data set based on the actual test data and the virtual test data, to obtain a new test data set;

[0022] Re-performing the step of performing weighted fusion processing on the test data in the test data set to obtain fused evaluation data and subsequent steps thereof based on the adjusted weights and the new test data set, until the dynamic deviation meets the preset deviation condition;

[0023] When the dynamic deviation meets the preset deviation condition, the fused evaluation data is determined as the target evaluation data.

[0024] In one implementation, before assigning corresponding initial weights to the test data in the test data set, the method further includes:

[0025] A preprocessing operation is performed on the test data in the test data set.

[0026] In one implementation, the step of evaluating whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated includes:

[0027] Comparing the target evaluation data with the pose data to be evaluated to calculate a pose error between the target evaluation data and the pose data to be evaluated;

[0028] Determine whether the posture error meets a preset error range to obtain a corresponding determination result;

[0029] Based on the judgment result, it is evaluated whether the position and posture positioning of the robot is accurate.

[0030] In one implementation, the step of evaluating whether the posture positioning of the robot is accurate based on the judgment result includes:

[0031] When the judgment result shows that the posture error satisfies the preset error range, it is determined that the posture positioning of the robot is accurate;

[0032] When the judgment result indicates that the posture error does not satisfy the preset error range, it is determined that the posture positioning of the robot is inaccurate.

[0033] The present invention also discloses a robot posture positioning evaluation device, wherein the device comprises:

[0034] A posture data calculation module is used to calculate the posture data of the robot when it moves to a target posture state in response to the control instruction received by the robot, and obtain the posture data to be evaluated;

[0035] A measured data acquisition module, used to acquire actual test data of the robot in the target posture state collected by multiple sensors;

[0036] A virtual data generation module, used to generate corresponding virtual test data based on the actual test data;

[0037] A data set construction module, used to construct a corresponding test data set based on the actual test data and the virtual test data;

[0038] A data training module, used for training target evaluation data based on the test data set;

[0039] The posture positioning evaluation module is used to evaluate whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated.

[0040] The present invention also discloses a terminal, which includes: a memory, a processor, and a robot posture positioning evaluation program stored in the memory and executable on the processor, wherein the robot posture positioning evaluation program, when executed by the processor, implements the steps of the robot posture positioning evaluation method as described above.

[0041] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the robot posture positioning evaluation method as described above.

[0042] The present invention provides a robot posture positioning evaluation method, device, terminal and storage medium. The robot posture positioning evaluation method includes: calculating the posture data of the robot when it moves to a target posture state in response to the control instruction received by the robot, and obtaining the posture data to be evaluated; acquiring the actual test data of the robot in the target posture state collected by multiple sensors; generating corresponding virtual test data based on the actual test data, and constructing a corresponding test data set based on the actual test data and the virtual test data; training target evaluation data based on the test data set, and evaluating whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated. It can be seen that the present invention calculates the posture data to be evaluated when the robot responds to the control instruction and moves to the target posture state according to the control instruction, and collects the actual test data of the robot in the target posture state through multiple sensors, and then generates virtual test data based on the actual test data, and then constructs a test data set including actual test data and virtual test data, and trains target evaluation data based on the test data set, and finally evaluates whether the posture positioning of the robot is accurate based on the posture data to be evaluated and the target evaluation data. That is, the technical solution of the present application evaluates the posture positioning of the robot by combining actual measured data with virtual data, and fusing multiple sensor data, thereby overcoming the problem of relying on a single sensor to limit the accuracy and reliability of posture measurement in traditional posture testing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a preferred embodiment of the robot posture positioning evaluation method of the present invention;

[0044] Figure 2 It is a flowchart of a specific deviation adjustment and weight optimization method disclosed in the present invention;

[0045] Figure 3 It is a functional principle block diagram of a preferred embodiment of the robot posture positioning evaluation device of the present invention;

[0046] Figure 4 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] See also Figure 1 , Figure 1 : is a flow chart of the robot posture positioning evaluation method in the present invention. Figure 1 As shown, the robot posture positioning evaluation method described in the embodiment of the present invention includes:

[0049] Step S11, calculating the posture data of the robot when it moves to the target posture state in response to the control instruction according to the control instruction received by the robot, and obtaining the posture data to be evaluated.

[0050] In this embodiment, the posture data of the robot when it moves to the target posture state in response to the control instruction is calculated according to the control instruction, and the posture data is used as the posture data to be evaluated. It can be understood that the robot controller sends the corresponding control instruction to the robot, and the robot moves to the target posture state in response to the control instruction. When collecting the posture data of the robot in the target posture state, the posture data of the robot in the target posture state can be calculated according to the control instruction, and the posture data is used as the instruction data, that is, the posture data to be evaluated.

[0051] Step S12: Acquire actual test data of the robot in the target posture state collected by multiple sensors.

[0052] In this embodiment, when the robot responds to the control command and moves to the target posture state, the actual test data is obtained by collecting the posture data of the robot in the target posture state. It can be understood that the posture measurement results of the robot in each actual test process are measured by multiple different sensors. In other words, by fusing multiple sensors, or multiple sensor fusion methods, such as GPS (Global Positioning System) and IMU fusion positioning, laser radar and visual sensor fusion positioning, the data obtained can effectively reduce the errors that may be generated when a single sensor or a multi-sensor fusion method is used alone, thereby improving the measurement accuracy.

[0053] For example, for a part to be evaluated on the robot, such as a wrist or foot, when collecting the posture data to be evaluated and the actual test data corresponding to the part to be evaluated, first, a control instruction is issued through the robot controller to control the part to be evaluated of the robot to move to the target posture state, such as moving to a specified position, or maintaining a specified posture. According to the control instruction, the posture data of the part to be evaluated of the robot when it is at the specified position or maintains a specified posture can be calculated to obtain the data to be evaluated, and at the same time, the posture data of the part to be evaluated when it is at the specified position or maintains a specified posture is collected through multiple sensors to obtain the actual measurement data, that is, the actual test data.

[0054] It should be pointed out that the multiple sensors can be at least two groups of other sensors, such as a group of inertial sensors (IMU) and a group of depth cameras, wherein a group of inertial sensors can include multiple inertial sensors, and a group of depth cameras can also include multiple depth cameras, such as multiple three-dimensional cameras, and the multiple sensors can be installed on the robot or outside the robot. For example, the inertial sensor is installed on the part to be evaluated of the robot, and the depth camera is installed at a position where the part to be evaluated of the robot can be photographed without obstruction. Then, when the robot moves to the target posture state, at least two groups of other sensors are collected, or the posture data of the part to be evaluated of the robot is measured by other sensors. When it is at a specified position or maintains a specified posture, that is, the actual test data can include data collected by the inertial sensor when the part to be evaluated of the robot is at a specified position or maintains a specified posture, and data collected by the depth camera when the part to be evaluated of the robot is at a specified position or maintains a specified posture.

[0055] For example, in the process of multiple tests of the robot moving to the target posture state, each set of sensors collected corresponding actual test data. For example, in the process of controlling the robot to move from the original posture state to the target posture state for the first time, a set of actual test data including data collected by the inertia sensor and data collected by the depth camera is obtained. In the process of controlling the robot to move from the original posture state to the target posture state for the second time, another set of actual test data including data collected by the inertia sensor and data collected by the depth camera is obtained. And so on, in the process of multiple tests of the robot moving to the target posture state, multiple sets of actual test data are obtained.

[0056] It should also be pointed out that the actual test data can be determined according to the needs of the robot performance evaluation. For example, the robot performance evaluation may include but is not limited to robot posture accuracy evaluation, robot posture repeatability evaluation, robot multi-directional posture accuracy change evaluation, robot moving distance accuracy evaluation, robot moving distance repeatability evaluation, robot position stabilization time evaluation, and robot position overshoot evaluation, etc.

[0057] Step S13: Generate corresponding virtual test data based on the actual test data, and construct a corresponding test data set based on the actual test data and the virtual test data.

[0058] In this embodiment, after obtaining the actual test data of the robot in the target posture state collected by multiple sensors, the corresponding virtual test data can be generated based on the actual test data. Specifically, the actual test data is trend analyzed to obtain the corresponding trend analysis results; and the corresponding virtual test data is generated according to the trend analysis results. It can be understood that the predicted virtual test data is obtained based on the trend of the actual test data.

[0059] For example, first, multiple groups of actual test data corresponding to each sensor are obtained as the basis for trend analysis, and then trend analysis is performed on the actual test data corresponding to each sensor to determine the trend of each sensor test data. For example, multiple groups of actual test data 1 corresponding to the inertial sensor and multiple groups of actual test data 2 corresponding to the depth camera are obtained as the basis for trend analysis, trend analysis is performed on the multiple groups of actual test data 1 corresponding to the inertial sensor to determine the trend of the inertial sensor test data, and trend analysis is performed on the multiple groups of actual test data 2 corresponding to the depth camera to determine the trend of the depth camera test data. Then, according to the trend analysis results corresponding to each sensor, virtual test data corresponding to each sensor is predicted and generated.

[0060] It should be pointed out that in the process of trend analysis, the factors or parameters that play a major role in the changes in the test data of each sensor are analyzed mainly based on the actual test data corresponding to the sensor, so that the test data of each sensor forms a function expression based on the factor or parameter, so as to predict and generate virtual test data based on the function expression.

[0061] In this embodiment, after obtaining the actual test data and the virtual test data predicted based on the actual test data, a corresponding test data set can be constructed based on the actual test data and the virtual test data. Specifically, the actual test data and the virtual test data are selected according to a preset data ratio to construct the corresponding test data set. It can be understood that the data set combining virtual and real data is constructed based on the actual test data and the virtual test data, that is, the actual test data and the virtual test data of multiple sensors are mixed in a certain ratio to form a data set mixed with virtual and real data.

[0062] Step S14: training target evaluation data based on the test data set, and evaluating whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated.

[0063] In this embodiment, after constructing a test data set including measured data and virtual data, target evaluation data is trained based on the test data set. It can be understood that data training is performed by combining measured data with virtual data, wherein virtual test data is predicted based on the trend of actual test data, and actual test data and virtual test data are mixed and iterated for training, thereby providing more training data, helping to improve the generalization ability of the model, effectively reducing the actual test workload, and speeding up the training iteration speed.

[0064] In this embodiment, see Figure 2 As shown, the specific process of training target evaluation data based on the test data set may include the following steps:

[0065] Step S141: assign corresponding initial weights to the test data in the test data set, and perform weighted fusion processing on the test data in the test data set to obtain fused evaluation data.

[0066] In this embodiment, initial weights are assigned to the test data in the test data set. Specifically, different initial weights are assigned to the test data corresponding to different sensors (actual test data and virtual test data) in the test data set. For example, a first initial weight is assigned to the test data corresponding to the inertial sensor, and a second initial weight is assigned to the test data corresponding to the depth camera, and the sum of the first initial weight and the second initial weight is not greater than 1.

[0067] In this embodiment, a preset data fusion algorithm is used to perform weighted fusion on the test data after weights are assigned in the test data set to obtain fused evaluation data. For example, a weighted averaging method is used to perform weighted fusion on the test data after weights are assigned in the test data set to obtain fused evaluation data.

[0068] Step S142: Calculate the dynamic deviation between the fused evaluation data and the actual test data, and determine whether the dynamic deviation meets a preset deviation condition.

[0069] In this embodiment, the test data in the test data set is weightedly fused to obtain fused evaluation data, and the dynamic deviation between the fused evaluation data and the actual test data collected by the sensor can be calculated to determine whether the dynamic deviation meets the preset deviation condition, such as determining whether the dynamic deviation meets the preset deviation threshold.

[0070] Step S143: When the dynamic deviation does not meet the preset deviation condition, the initial weight is adjusted based on the dynamic deviation to obtain the adjusted weight, and new actual test data collected by multiple sensors when the robot responds to the control instruction and moves to the target posture state is re-acquired.

[0071] In this embodiment, when the dynamic deviation does not meet the preset deviation condition, the initial weight is adjusted based on the dynamic deviation to obtain the adjusted weight, and new actual test data collected by multiple sensors when the robot responds to the control instruction and moves to the target posture state is obtained again. In other words, the robot is controlled to perform multiple test processes from the original posture state to the target posture state, and the test data collected by multiple different sensors is obtained to obtain multiple sets of new actual test data.

[0072] Step S144 , re-performing the step of generating corresponding virtual test data based on the actual test data based on the new actual test data, and constructing a corresponding test data set based on the actual test data and the virtual test data, to obtain a new test data set.

[0073] In this embodiment, the above step S13 is re-executed based on the new actual test data to obtain a new test data set, that is, new virtual test data is generated based on the new actual test data, and then a new test data set is constructed based on the new actual test data and the new virtual test data.

[0074] Step S145, re-execute the step of performing weighted fusion processing on the test data in the test data set to obtain fused evaluation data based on the adjusted weights and the new test data set, and its subsequent steps, until the dynamic deviation meets the preset deviation condition.

[0075] In this embodiment, after the weights are adjusted based on the dynamic deviation to obtain the adjusted weights and a new test data set is constructed, the test data in the new test data set is further weighted fused based on the adjusted weights and the new test data set to obtain new fused evaluation data, and the dynamic deviation between the new fused evaluation data and the new actual test data is continued to be calculated, and then it is determined whether the dynamic deviation meets the preset deviation condition. When the calculated dynamic deviation still does not meet the preset deviation condition, the above steps S143, S144, and S145 are re-executed, that is, the new actual test data is re-acquired, the new test data set is re-constructed, and then the test data in the new test data set is weighted fused again, and the dynamic deviation between the fused evaluation data at this time and the new actual test data is continued to be calculated until the dynamic deviation meets the preset deviation condition.

[0076] Step S146: When the dynamic deviation satisfies the preset deviation condition, the fused evaluation data is determined as the target evaluation data.

[0077] In this embodiment, when the dynamic deviation meets the preset deviation condition, the above-mentioned iterative training process of the post-fusion evaluation data is stopped, and then the corresponding post-fusion evaluation data when the dynamic deviation meets the preset deviation condition is determined as the target evaluation data, that is, the process of training the post-fusion evaluation data is repeated to perform data collection to build a test data set, data weighted fusion and weight adjustment until the post-fusion evaluation data reaches the training termination condition, that is, the target evaluation data is dynamically adjusted and weighted to ensure the stability and accuracy of the fusion result. It can be seen that the dynamic deviation adjustment and weight optimization mechanism can adjust the weight of each sensor in the fusion algorithm in real time to adapt to different environments and operating conditions and enhance the robustness of the measurement results.

[0078] It should be pointed out that the termination condition for training the post-fusion evaluation data can also be to determine whether the post-fusion evaluation data has reached a convergence state. If it has reached a convergence state, the iterative training process is stopped, and the converged post-fusion evaluation data is determined as the target evaluation data. If it has not reached a convergence state, the iterative training continues, or when the maximum number of iterations is reached, the iterative training is stopped, and further optimized. In the process of iterative training, several sets of prediction data can be generated according to the initial iterative test results, and iterations are continued according to the prediction data. After the iteration of the prediction data is completed, several sets of data are actually tested, and then the trend law is determined according to the new measured data and the prediction data is iterated until the post-training fusion error is less than the threshold or the fusion result converges. Or, when the preset number of iterations is not reached, measured data or virtual data can be randomly extracted from the test data set for training iterations. After the preset number of iterations is reached, virtual test data is continued to be generated according to the new measured data, and training iterations are continued until the post-training fusion error is less than the threshold or the fusion result converges.

[0079] In this embodiment, before assigning corresponding initial weights to the test data in the test data set, the method may further specifically include: performing a preprocessing operation on the test data in the test data set; the preprocessing operation is a unified operation of the data timestamp, format and reference coordinate system. It is understandable that during training, the test data in the test data set is preprocessed to unify its timestamp, format and reference coordinate system to ensure data consistency, and data processing operations such as denoising and filtering may also be performed on the test data in the test data set, thereby improving the accuracy of the data.

[0080] In this embodiment, after training the fused and stable target evaluation data based on the test data set combined with virtual and real data, the robot's posture positioning can be evaluated based on the target evaluation data and the posture data to be evaluated to determine whether it is accurate. Specifically, the target evaluation data is compared with the posture data to be evaluated to calculate the posture error between the target evaluation data and the posture data to be evaluated; the posture error is determined to satisfy the preset error range to obtain a corresponding judgment result; based on the judgment result, the robot's posture positioning is evaluated to determine whether it is accurate, and when the judgment result shows that the posture error satisfies the preset error range, the robot's posture positioning is determined to be accurate; when the judgment result shows that the posture error does not satisfy the preset error range, the robot's posture positioning is determined to be inaccurate. It can be understood that the target evaluation data after training is compared with the posture data to be evaluated to calculate the posture error between the two sets of data, and the robot's posture is evaluated according to the size and distribution of the posture error. If the posture error is within the acceptable error range, it is considered that the robot's posture measurement is accurate; if the posture error is not within the acceptable error range, it is necessary to calibrate or adjust the sensor or the data fusion algorithm used in the weighted fusion processing.

[0081] It can be seen that in the embodiment of the present invention, the posture data to be evaluated when the robot responds to the control instruction and moves to the target posture state is calculated according to the control instruction, and the actual test data of the robot in the target posture state is collected through multiple sensors, and then virtual test data is generated based on the actual test data, and then a test data set including actual test data and virtual test data is constructed, and target evaluation data is trained based on the test data set, and finally, whether the posture positioning of the robot is accurate is evaluated based on the posture data to be evaluated and the target evaluation data. That is, the technical solution of the present application evaluates the posture positioning of the robot by combining actual measured data with virtual data and fusing multiple sensor data, thereby overcoming the problem of relying on a single sensor to limit the accuracy and reliability of posture measurement in traditional posture testing methods.

[0082] It should be pointed out that an inertia sensor is installed on the part to be evaluated of the humanoid robot, such as the wrist joint, and a depth camera is installed on the outside of the humanoid robot. The installation position of the depth camera should enable the depth camera to capture the part to be evaluated of the robot without obstruction, that is, the wrist of the robot to be evaluated, so as to estimate its position.

[0083] For example, in the specific application scenario of evaluating the position accuracy of a humanoid robot, the humanoid robot commands the wrist joint to move to a specified position according to the control instruction, and calculates the three-dimensional position data N to be evaluated when the wrist joint of the humanoid robot moves to the specified position according to the control instruction. ctrl , and at the same time, the actual three-dimensional position data of the humanoid robot wrist when it moves to the specified position is collected through the inertial sensor and the depth camera. The two sets of position data collected by the inertial sensor and the depth camera are fused to evaluate the position accuracy of the robot controlled by the robot controller. The specific evaluation process is:

[0084] Perform n actual position tests on the robot wrist joint position to obtain multiple sets of actually measured three-dimensional position coordinates, that is, obtain the actual three-dimensional position data collected by the IMU and depth camera during each actual position test, that is, N imu and N cam .

[0085] Among them, N imu It represents the three-dimensional position coordinates actually measured when the robot wrist joint moves to the specified position obtained by IMU through position integration, N cam Indicates the three-dimensional position coordinates actually measured when the robot wrist moves to the specified position obtained by the depth camera.

[0086] Then, virtual 3D position data is generated based on the actual 3D position data, that is, virtual test data corresponding to each sensor is predicted and generated according to the trend analysis results corresponding to each sensor. It should be noted that the trend analysis uses the most recent actual test data of the sensor for analysis. The initial trend analysis uses the actual test data collected by the sensor during the initial iteration, and the subsequent trend analysis in the iterative training uses the most recently obtained actual test data.

[0087] For example, in the robot position test, it is assumed that the features that affect the prediction error mainly include the timestamp t, environmental factors such as the ambient temperature TE, and the robot motion features SP, such as the speed or acceleration when the wrist joint moves to the specified position. Then, the principal component analysis (PCA) is used to calculate the features that have the greatest impact on the prediction error. The specific calculation process is as follows: Assume that one of the test sensors, such as IMU, has n actual test data points during the initial iteration process. Each actual test data point and the prediction error are related to the timestamp t. i , Ambient temperature TE i and robot motion characteristics SP i Related, then the timestamp t i , Ambient temperature TE i and robot motion characteristics SP i They are standardized to make their mean 0 and standard deviation 1, and the standardized eigenvalues ​​are obtained. The standardized eigenvalues ​​are combined into the feature matrix X, that is:

[0088] X=[t i -Mean(t),TE i -Mean(TE),SP i -Mean(SP)];

[0089] Among them, Mean(t), Mean(TE), and Mean(SP) represent the timestamp t in multiple tests respectively. i , Ambient temperature TE i and robot motion characteristics SP i The mean of .

[0090] Next, calculate the covariance matrix Z of the feature matrix X, that is:

[0091] Z = X'X;

[0092] Where X' represents the transposed matrix of X.

[0093] Then perform eigenvalue decomposition on the covariance matrix Z to obtain the eigenvalue λ 1 , 2 , 3 and the corresponding eigenvector V 1 、V2 、V 3 .

[0094] If we determine λ 1 >λ 2 >λ 3 , then λ 1 and its corresponding V 1 as the principal component eigenvalues ​​and their corresponding eigenvectors.

[0095] Then check the eigenvector V 1 with timestamp t i , Ambient temperature TE i and robot motion characteristics SP i The corresponding element, for example, when processing the i When the corresponding elements are extracted, the eigenvector V 1 The element V corresponding to the timestamp ti 1 ,t i , and use the linear regression model to predict the value of the first principal component score PPC1, where the formula of the linear regression model is:

[0096] PPC1 = β 0 +β 1 t;

[0097] Among them, β 0 and β 1 represents the parameters of the linear regression model, and t represents the timestamp.

[0098] Use the least squares method to estimate the parameters β of the linear regression model 0 and β 1 :

[0099]

[0100] in, represents the average value of timestamp t, represents the mean value of PPC1.

[0101] Using the estimated parameters β 0 and β 1 To predict the value of PPC1 and compare it with the actual value of PPC1, if the difference between the predicted value and the actual value is small, it is considered that there is a good linear relationship between PPC1 and timestamp t.

[0102] After further deduction, using the above method of deriving PPC1 and timestamp t, it can be obtained that there is also a linear relationship between PPC1 and the actual test data collected by the IMU, and finally the linear relationship between the actual test data collected by the IMU and the timestamp is obtained, that is:

[0103] N imu =γ 0 +γ 1 t;

[0104] Among them, γ 0 and γ 1 represents the parameters of the linear regression model, and t represents the timestamp. This formula can be used to predict the test data of the IMU, thereby obtaining the virtual test data corresponding to the IMU.

[0105] Furthermore, the linear relationship between the actual test data collected by the IMU and the timestamp is further refined, namely:

[0106] N imu =γ 0 +γ 1 t+σN(0,1);

[0107] Where σ represents the standard deviation of Gaussian noise, and N(0,1) represents a Gaussian distributed random number with a mean of 0 and a standard deviation of 1.

[0108] Correspondingly, the data prediction formula corresponding to the depth camera can be derived by using a derivation method similar to the data prediction formula corresponding to the above-mentioned IMU.

[0109] Then, based on the derived data prediction formula corresponding to the IMU and the data prediction formula corresponding to the depth camera, multiple sets of virtual 3D position data can be generated according to the needs. and in, It represents the virtual three-dimensional position data predicted based on the derived IMU corresponding data prediction formula. Represents the virtual three-dimensional position data predicted based on the derived data prediction formula corresponding to the depth camera.

[0110] Secondly, construct a data set that combines virtual and real data and preprocess the data in the data set, that is, obtain the actual three-dimensional position data N of multiple sensors. imu and N cam and virtual 3D position data and Afterwards, the actual 3D position data and the virtual 3D position data can be mixed in a certain proportion to form a test data set that combines virtual and real data. For example, 50% of the actual 3D position data and 50% of the virtual 3D position data are mixed to form a test data set, and then the N imu and N cam ,as well as and Perform preprocessing such as denoising, filtering, unifying timestamps, formats (dimensions), and reference coordinate systems to improve data accuracy and ensure data consistency. For example, the data after preprocessing is as follows:

[0111] The actual three-dimensional position data N measured by the IMU sensor imu =[0.9,2.1,2.8];

[0112] The actual 3D position data N measured by the depth camera cam =[1.2,1.8,3.2];

[0113] Furthermore, the three-dimensional position data N to be evaluated can also be ctrl Perform preprocessing, namely:

[0114] N ctrl =[1.0,2.0,3.0].

[0115] Next, in the process of weighted fusion of the data in the test data set, different initial weights are assigned to the test data (actual test data and virtual test data) corresponding to different sensors in the test data set. For example, assuming that the initial weights are all 0.5, that is:

[0116] The initial weight w corresponding to the IMU imu , and w imu =0.5;

[0117] The initial w corresponding to the depth camera cam , and w cam =0.5;

[0118] Then, calculate the fused position data N fuse ,Right now:

[0119]

[0120] In addition, in the process of iterative training of the fused position data, the virtual and real data are used for training to adjust the dynamic deviation and optimize the weight. The specific iterative training process is described above, and the data processing involved in the iterative training process is as follows:

[0121] First, set the learning rate α and the regularization parameter λ, that is:

[0122] α=0.01;λ=0.1;

[0123] Among them, the learning rate α is used to control the step size of weight update, and the regularization parameter λ is used to prevent weight overfitting.

[0124] Calculate the deviation between the fused position data and the actual 3D position data, that is:

[0125]

[0126] Among them, e represents the dynamic deviation, that is, the fusion evaluation data P fuse The actual measurement data N collected by the sensor imu With N cam The difference between imu It represents the deviation between the actual 3D position data measured by IMU and the fused position data, e cam Represents the deviation between the actual 3D position data measured by the depth camera and the fused position data.

[0127] N imu,i Represents the i-th element of the actual three-dimensional position data measured, N cam,i Represents the i-th element of the actual 3D position data measured by the depth camera, N fuse,i Represents the i-th element of the fused position data.

[0128] When updating the weights, the corresponding gradient is calculated based on the deviation calculated above, that is:

[0129]

[0130] in, Represents the IMU weight gradient, that is, the weight w imu The orientation and size should be adjusted to minimize the deviation; Represents the depth camera weight gradient, that is, weight w cam The orientation and size should be adjusted to minimize the deviation.

[0131] Then, the weights are updated based on the gradient, that is:

[0132]

[0133] The weights can also be normalized to ensure that the sum of the weights of different sensors is no greater than 1, that is:

[0134] w imu,i =w imu / (w imu +w cam );

[0135] w cam,i =1-w imu,i ;

[0136] Among them, w imu,i Represents the normalized weight corresponding to the IMU, w cam,i Represents the normalized weight corresponding to the depth camera.

[0137] During the kth iteration of training, the root mean square of the dynamic deviation between the fused evaluation data and the actual measurement data corresponding to the IMU and depth camera is compared. When the root mean square of the dynamic deviation of the two is less than the preset root mean square threshold ∈rmse, the iteration stops, or when the maximum number of iterations T is reached, the iteration stops, otherwise, the iteration continues, that is, after updating the weights, the fused position is recalculated, and the training process of dynamic deviation adjustment and weight optimization is carried out.

[0138] For example, when and When , the iteration stops, otherwise the iteration continues, where The calculation method of is as follows:

[0139]

[0140] Among them, after the iteration stops, the target evaluation data after fusion stabilization is obtained:

[0141] It should be noted that the pose accuracy represents the deviation between the command pose and the average value of the actual pose when approaching the command pose from the same direction. In the current specific application scenario of evaluating the position accuracy of humanoid robots, the position accuracy refers to the difference between the position of the command pose and the center of gravity of the cluster of multiple evaluation positions.

[0142] For example, determine the coordinates of the cluster center of each point after the same posture is repeatedly responded m times That is, the average of the training stable target evaluation data obtained by multiple position evaluations of the posture, and then calculate the coordinates and the three-dimensional position data N to be evaluated ctrl , the Euclidean distance, where N ctrl The spatial coordinates of ctrl ,y ctrl ,z ctrl ), and then evaluate the robot position accuracy based on the Euclidean distance, the relevant calculation formula is:

[0143]

[0144] Among them, AP p Indicates location accuracy, AP x 、AP y 、AP z Respectively represent the position accuracy of the x, y, and z axes, represents the coordinates of the cluster centers of each point obtained after repeated response to the same posture m times, (x fuse,j ,y fuse,j ,z fuse,j) represents the coordinates of the fused evaluation data of the jth part to be evaluated after the training is stable.

[0145] For another example, in the specific application scenario of evaluating the posture accuracy of a humanoid robot, the humanoid robot commands the wrist joint to form a specified posture according to the control instruction, and calculates the three-dimensional posture data q to be evaluated when the wrist joint of the humanoid robot forms the specified posture according to the control instruction. cmd , which can be expressed in quaternions, and the actual 3D posture data of the humanoid robot wrist joint when it forms a specified posture is collected through the inertial sensor and the depth camera. The two sets of actual 3D posture data collected by the inertial sensor and the depth camera are fused to evaluate the posture accuracy of the robot controlled by the robot controller. The specific evaluation process is:

[0146] Refer to the data set construction method and data preprocessing method described above to construct the test data set and preprocess the actual 3D posture data and virtual 3D posture data in the data set, and then initialize the weight w imu and w cam , where weights can be assigned according to sensor reliability. For example, if the IMU is more reliable, then w imu >w cam , and then perform weighted fusion processing on the test data in the test data set to obtain the fused posture data, namely:

[0147]

[0148] Among them, q imu It represents the actual three-dimensional posture data of the robot's part to be evaluated obtained by IMU through position integration, expressed in quaternion; q cam The actual three-dimensional posture data of the robot's part to be evaluated obtained by the depth camera is expressed in quaternion; It represents the virtual 3D posture data predicted based on the derived IMU corresponding data prediction formula. Represents the virtual 3D posture data predicted based on the derived data prediction formula corresponding to the depth camera.

[0149] Then the fused posture data is iteratively trained, namely:

[0150] First, the fused posture data The actual 3D posture data q collected by IMU and depth camera imu and q cam , are converted to Euler angles, namely:

[0151] θ fuse =(θ fuse,yaw ,θ fuse,pitch ,θ fuse,roll);

[0152] θ imu =(θ imu,yaw ,θ imu,pitch ,θ imu,roll );

[0153] θ cam =(θ cam,yaw ,θ cam,pitch ,θ cam,rill );

[0154] Secondly, the deviation between the actual 3D posture data measured by the IMU and the depth camera and the fused posture data is calculated respectively, that is:

[0155]

[0156] Among them, E imu It represents the deviation between the actual 3D posture data measured by IMU and the fused posture data, E cam Represents the deviation between the actual 3D pose data measured by the depth camera and the fused pose data.

[0157] Next, the Euclidean norm, i.e., the L2 norm, can be used to calculate the norm of the deviation vector to obtain the deviation vector and its norm of each sensor, which can be used for subsequent weight updates and posture evaluation, i.e.:

[0158]

[0159] Among them, ||E imu || represents the norm of the actual 3D posture data measured by IMU and the fused posture data, ||E cam || represents the norm of the actual 3D posture data measured by the depth camera and the fused posture data.

[0160] Then, the deviation is compared with the preset deviation threshold ∈. If ||E imu ||<∈ and ||E cam ||<∈, or the maximum number of iterations T is reached, the iteration is stopped, otherwise the aforementioned gradient descent method is used to update the weights. After the weights are updated, the fused posture data is recalculated, and the training process of dynamic deviation adjustment and weight optimization is carried out.

[0161] That is, when ||E imu ||<∈ and ||E cam When ||<∈, the iteration stops, and the posture evaluation data after the current data fusion is stable is:

[0162]

[0163] Finally, the posture accuracy is calculated to evaluate the posture accuracy. The posture accuracy can be the difference between the posture data to be evaluated and the average value of the fused and stabilized posture evaluation data obtained when evaluating the posture multiple times, that is:

[0164]

[0165] in, It represents the average value of posture evaluation data obtained by repeatedly responding to the same posture n times after fusion is stabilized.

[0166] In one embodiment, if Figure 3 As shown, based on the above robot posture positioning evaluation method, the present invention also provides a robot posture positioning evaluation device, including:

[0167] A posture data calculation module 11 is used to calculate the posture data of the robot when it moves to a target posture state in response to the control instruction received by the robot, and obtain the posture data to be evaluated;

[0168] The measured data acquisition module 12 is used to acquire the actual test data of the robot in the target posture state collected by multiple sensors;

[0169] A virtual data generating module 13, used to generate corresponding virtual test data based on the actual test data;

[0170] A data set construction module 14, configured to construct a corresponding test data set based on the actual test data and the virtual test data;

[0171] A data training module 15 is used to train target evaluation data based on the test data set;

[0172] The posture positioning evaluation module 16 is used to evaluate whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated.

[0173] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0174] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0175] When the processor 502 executes the program, the robot posture positioning evaluation method provided in the above embodiment is implemented.

[0176] Furthermore, the terminal further includes:

[0177] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0178] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0179] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0180] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0181] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0182] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0183] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the robot posture positioning evaluation method as described above is implemented.

[0184] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0185] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0186] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, in which the order shown or discussed may not be followed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0187] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can read instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary and then storing it in a computer memory.

[0188] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0189] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0190] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0191] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0192] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A robot posture positioning evaluation method, characterized in that: The method comprises: Calculating the posture data of the robot when it moves to the target posture state in response to the control instruction according to the control instruction received by the robot, and obtaining the posture data to be evaluated; Acquire actual test data of the robot in the target posture state collected by multiple sensors; Generate corresponding virtual test data based on the actual test data, and construct a corresponding test data set based on the actual test data and the virtual test data; Target evaluation data is trained based on the test data set, and whether the posture positioning of the robot is accurate is evaluated based on the target evaluation data and the posture data to be evaluated.

2. The robot posture positioning evaluation method according to claim 1, characterized in that: The generating corresponding virtual test data based on the actual test data includes: Performing trend analysis on the actual test data to obtain corresponding trend analysis results; Corresponding virtual test data is generated according to the trend analysis result.

3. The robot posture positioning evaluation method according to claim 1, characterized in that: The constructing a corresponding test data set based on the actual test data and the virtual test data includes: The actual test data and the virtual test data are selected according to a preset data ratio to construct a corresponding test data set.

4. The robot posture positioning evaluation method according to claim 1, characterized in that: The step of training target evaluation data based on the test data set includes: Assigning corresponding initial weights to the test data in the test data set, and performing weighted fusion processing on the test data in the test data set to obtain fused evaluation data; Calculating the dynamic deviation between the fused evaluation data and the actual test data, and determining whether the dynamic deviation meets a preset deviation condition; When the dynamic deviation does not meet the preset deviation condition, the initial weight is adjusted based on the dynamic deviation to obtain an adjusted weight, and new actual test data collected by the multiple sensors when the robot responds to the control instruction and moves to the target posture state is reacquired; Re-performing the step of generating corresponding virtual test data based on the actual test data based on the new actual test data, and constructing a corresponding test data set based on the actual test data and the virtual test data, to obtain a new test data set; Re-performing the step of performing weighted fusion processing on the test data in the test data set to obtain fused evaluation data and subsequent steps thereof based on the adjusted weights and the new test data set, until the dynamic deviation meets the preset deviation condition; When the dynamic deviation meets the preset deviation condition, the fused evaluation data is determined as the target evaluation data.

5. The robot posture positioning evaluation method according to claim 4, characterized in that: Before allocating corresponding initial weights to the test data in the test data set, the method further includes: A preprocessing operation is performed on the test data in the test data set.

6. The robot posture positioning evaluation method according to any one of claims 1 to 5, characterized in that: The step of evaluating whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated includes: Comparing the target evaluation data with the pose data to be evaluated to calculate a pose error between the target evaluation data and the pose data to be evaluated; Determine whether the posture error meets a preset error range to obtain a corresponding determination result; Based on the judgment result, it is evaluated whether the position and posture positioning of the robot is accurate.

7. The robot posture positioning evaluation method according to claim 6, characterized in that: The step of evaluating whether the position and posture positioning of the robot is accurate based on the judgment result includes: When the judgment result shows that the posture error satisfies the preset error range, it is determined that the posture positioning of the robot is accurate; When the judgment result indicates that the posture error does not satisfy the preset error range, it is determined that the posture positioning of the robot is inaccurate.

8. A robot posture positioning evaluation device, characterized in that: The device comprises: A posture data calculation module is used to calculate the posture data of the robot when it moves to a target posture state in response to the control instruction received by the robot, and obtain the posture data to be evaluated; A measured data acquisition module, used to acquire actual test data of the robot in the target posture state collected by multiple sensors; A virtual data generation module, used to generate corresponding virtual test data based on the actual test data; A data set construction module, used to construct a corresponding test data set based on the actual test data and the virtual test data; A data training module, used for training target evaluation data based on the test data set; The posture positioning evaluation module is used to evaluate whether the posture positioning of the robot is accurate based on the target evaluation data and the posture data to be evaluated.

9. A terminal, characterized in that: include: A memory, a processor, and a robot posture positioning evaluation program stored in the memory and executable on the processor, wherein the robot posture positioning evaluation program, when executed by the processor, implements the steps of the robot posture positioning evaluation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the robot posture positioning evaluation method according to any one of claims 1 to 7.

Images