Robot space positioning system based on space-time synchronization and positioning method thereof

Abstract

The application is suitable for the field of robot control and automation technology, and provides a robot space positioning system based on space-time synchronization and a positioning method thereof, which comprises a data acquisition module, a time alignment module, a world model construction module, a world state prediction module and a decision and execution module, wherein the data acquisition module is used for acquiring initial data of multiple sensors; the time alignment module performs time alignment on original data of multiple sensors; the world model construction module is used for constructing a world model of external dynamics of the robot; the world state prediction module is used for calculating a delay duration of execution of the robot, and predicting a space state of the robot at an execution time; and the decision and execution module is used for acquiring an execution instruction of the robot and driving the robot to work.The asynchronous data collected by multiple sensors is time-aligned, the accuracy of robot space positioning is improved, the compensation for execution delay of the robot is realized, and the positioning precision of the robot in a dynamic environment is improved.

Description

Technical Field

[0001] This invention relates to the field of robot control and automation technology, and more specifically, to a robot spatial positioning system and positioning method based on spatiotemporal synchronization. Background Technology

[0002] Robot spatial positioning technology refers to the technology that allows a robot to perceive environmental information, combine it with its own posture, and guide its end effector to complete the task. Robot spatial positioning technology is the foundation for robots to perform tasks such as grasping and assembly in automated work processes, and its accuracy determines the quality and efficiency of the robot's work.

[0003] In practical robot systems, in order to achieve more accurate spatial positioning, a method of fusing measurement data from multiple sensors is used for decision-making. However, the clock and acquisition cycle of each sensor are different, which makes the multi-sensor data for decision-making asynchronous. Directly using these time-displaced data will lead to spatiotemporal inconsistencies. At the same time, there is also an execution delay between the robot issuing control commands and executing control commands, which makes it difficult to meet the high-speed and high-precision working requirements of robots in dynamic environments. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a robot spatial positioning system and positioning method based on spatiotemporal synchronization.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a robot spatial localization method based on spatiotemporal synchronization, comprising: S1. Acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor data, and establish a multi-sensor database; S2. Based on the acquisition timestamp, preprocess the initial data of the multi-sensor system to obtain time-aligned multi-sensor data; S3. Construct a world model based on the time-aligned multi-sensor data; S4. Obtain the execution delay duration of the robot, and based on the execution delay duration, predict the spatial state of the robot at the execution moment using the world model; S5. Based on the predicted spatial state, obtain the robot's execution instructions, and drive the robot to work according to the execution instructions.

[0006] The present invention is further configured such that: the multiple sensors in S1 include external sensors and internal sensors, each sensor is set with a collection period, and the data collected by the multiple sensors is automatically uploaded to the multiple sensor database and the multiple sensor database is updated in real time.

[0007] The present invention further describes that S2 is further comprising: S201, Set the fusion time Where k = 0, 1, 2, ... This represents the k-th fusion time. S202. For each sensor, match the distance fusion time in the multi-sensor database. The two most recent valid data points are marked with their collection timestamps. and ,satisfy < < ; Obtain the collection timestamp Corresponding measurement value and the collection timestamp Corresponding measurement value The This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time, the This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time.

[0008] The present invention further illustrates that S2 includes: S203. Select the interpolation model based on the fusion time. Collection timestamp Collection timestamp Measured values and measured values Calculate the fusion time of the i-th sensor. Measured values ; S204, Obtain the fusion time Sensor measurement vector ,in, Indicates the time of fusion of the i-th sensor. The measured value, where N represents the total number of sensors.

[0009] The present invention further describes that S4 further includes: S401. The state is estimated through the world model, and combined with the robot's motion planning, the first execution time T_st of the robot is determined, and the execution delay T_delay is calculated. The second execution time T_nd = T_st + T_delay of the robot is calculated through the first execution time T_st and the execution delay T_delay. S402. Based on the second execution time T_nd, call the world model again to predict the world state at the second execution time T_nd.

[0010] The present invention further describes that S401 further includes: S4011. Perform state estimation through the world model to obtain the optimal state estimate at the current moment. Based on the optimal state estimate at the current moment and combined with the robot's motion planning, determine the robot's first execution time T_st, where the first execution time T_st is the moment when the robot plans to execute the instruction. S4012. Obtain the execution delay duration T_delay, where the execution delay includes the communication system transmission delay T_tx, the control system response delay T_rx, and the mechanical system motion delay T_mot; calculate the execution delay duration. ; S4013. Based on the first execution time T_st and the execution delay duration T_delay, calculate the robot's second execution time T_nd = T_st + T_delay, where the second execution time T_nd represents the expected execution time for completing robot execution delay compensation.

[0011] The present invention further explains that the prediction of the world state at the second execution time T_nd in S402 specifically involves: The world model is invoked based on the second execution time T_nd, and the state is estimated again through the world model to obtain the optimal state estimate of the second execution time T_nd, and the world state of the second execution time T_nd is predicted.

[0012] The present invention further describes that S5 is further as follows: The robot's own posture is obtained by calling the internal sensor data in the multi-sensor database. Based on the world state at the second execution time T_nd, the robot's motion sequence is calculated to obtain the robot's execution instructions. The robot is then driven to execute according to the execution instructions.

[0013] A spatiotemporal synchronization-based robot spatial localization system, used to implement the spatiotemporal synchronization-based robot spatial localization method described above, includes a data acquisition module, a time alignment module, a world model construction module, a world state prediction module, and a decision-making and execution module. The data acquisition module is used to acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor data, and establish a multi-sensor database. The time alignment module is used to preprocess the raw data of the multi-sensor according to the acquisition timestamp to obtain time-aligned multi-sensor data. The world model building module is used to construct a world model of the robot's external dynamics based on time-aligned multi-sensor data. The world state prediction module is used to calculate the robot's execution delay duration, calculate the robot's execution time based on the execution delay duration, and predict the robot's spatial state at the execution time using the world model; The decision-making and execution module is used to obtain the robot's execution instructions based on the predicted spatial state at the execution time, and drive the robot to work according to the execution instructions.

[0014] By adopting the above technical solution, this application includes at least one of the following beneficial technical effects: 1. Time-align asynchronous data collected by multiple sensors, unify timestamps, reduce data errors in decision-making, and improve the accuracy of robot spatial positioning.

[0015] 2. Considering the execution delay between the robot issuing control commands and executing them, the execution delay is decomposed into communication system transmission delay, control system response delay, and mechanical system motion delay. Based on the execution delay, the world state at the future execution moment is predicted, thereby compensating for the robot's execution delay and improving the robot's positioning accuracy in dynamic environments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] It should be noted that, unless otherwise specified, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0019] Please see Figures 1 to 2 The present invention provides the following technical solutions: Example 1, see Figure 1 A robot spatial localization method based on spatiotemporal synchronization includes the following steps: S1. Acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor data, and establish a multi-sensor database; S2. Based on the acquisition timestamp, preprocess the initial data from multiple sensors to obtain time-aligned multi-sensor data; S3. Construct a world model based on time-aligned multi-sensor data; S4. Obtain the robot's execution delay duration, and based on the execution delay duration, predict the spatial state of the robot at the execution moment using the world model; S5. Based on the predicted spatial state, obtain the robot's execution instructions and drive the robot to execute according to the execution instructions.

[0020] Furthermore, when acquiring initial data from multiple sensors, the specific steps are as follows: Multiple sensors include external sensors and internal sensors. External sensors specifically include vision sensors, lidar, etc., which acquire data on the robot's external environment to build a world model. Internal sensors specifically include joint encoders, inertial measurement sensors, etc., which acquire the robot's own posture data to generate control commands. In addition, each sensor has a different acquisition period. Generally, the acquisition period of external sensors is longer and the acquisition period of internal sensors is shorter. For example, the acquisition period of the vision sensor is 30ms and the acquisition period of the joint encoder is 1ms.

[0021] Furthermore, S2 specifically refers to: S201, Set the fusion time Where k = 0, 1, 2, ... Let k represent the k-th fusion time. The interval between fusion times is set to match the robot's control cycle. Preferably, the interval between fusion times is set to the robot's control cycle, so that the robot's entire multi-sensor system works around the same time axis, reducing the computational complexity of the system and improving computational efficiency.

[0022] S202. For each sensor, match the distance fusion time in the multi-sensor database. The two most recent valid data points are marked with their collection timestamps. and ,satisfy < < ; Get collection timestamp Corresponding measurement value and collection timestamp Corresponding measurement value , This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time.

[0023] S203. Select an interpolation model. The interpolation model can be selected based on the robot's motion scenario, such as a linear interpolation model, a polynomial interpolation model, etc., according to the fusion time. Collection timestamp Collection timestamp Measured values and measured values Calculate the fusion time of the i-th sensor. Measured values .

[0024] S204, Obtain the fusion time Sensor measurement vector ,in, Indicates the time of fusion of the i-th sensor. The measured value, where N represents the number of sensors.

[0025] Furthermore, S4 specifically refers to: S401. Estimate the state using the world model, combine it with the robot's motion planning, determine the robot's first execution time T_st, and calculate the execution delay T_delay. Using the first execution time T_st and the execution delay T_delay, calculate the robot's second execution time T_nd = T_st + T_delay.

[0026] Specifically, the second execution time T_nd = T_st + T_delay of the computational robot is as follows: S4011. Perform state estimation through the world model to obtain the optimal state estimate at the current moment. Based on the optimal state estimate at the current moment and combined with the robot's motion planning, determine the robot's first execution time T_st. The first execution time T_st is the moment when the robot plans to execute the instruction, that is, the execution time without considering the execution delay.

[0027] S4012. Obtain the execution delay duration T_delay. The execution delay includes the communication system transmission delay T_tx, the control system response delay T_rx, and the mechanical system motion delay T_mot. Calculate the execution delay duration. Among them, the transmission delay T_tx of the communication system, the response delay T_rx of the control system, and the motion delay T_mot of the mechanical system depend on the system performance of the robot and can be evaluated through experimental calibration and other methods.

[0028] S4013. Based on the first execution time T_st and the execution delay duration T_delay, calculate the robot's second execution time T_nd = T_st + T_delay. The second execution time T_nd represents the expected execution time for completing robot execution delay compensation.

[0029] S402. Based on the second execution time T_nd, call the world model again to predict the world state of the second execution time T_nd.

[0030] Specifically, the predicted world state at the second execution time T_nd is as follows: Based on the world model being invoked at the second execution time T_nd, the state is estimated again through the world model to obtain the optimal state estimate of the second execution time T_nd and predict the world state of the second execution time T_nd.

[0031] Furthermore, S5 specifically refers to: The robot's own posture is obtained by calling the internal sensor data in the multi-sensor database. Based on the world state at the second execution time T_nd, the robot's motion sequence is calculated to obtain the robot's execution instructions. The robot is then driven to execute according to the execution instructions.

[0032] Example 2, see Figure 2 A spatiotemporal synchronization-based robot spatial localization system, used to implement the aforementioned spatiotemporal synchronization-based robot spatial localization method, includes a data acquisition module, a time alignment module, a world model construction module, a world state prediction module, and a decision-making and execution module. The data acquisition module is used to acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor's data, and establish a multi-sensor database. The time alignment module is used to preprocess the raw data from multiple sensors based on the acquisition timestamps to obtain time-aligned multi-sensor data. The world model building module is used to construct a world model of the robot's external dynamics based on time-aligned multi-sensor data; The world state prediction module is used to calculate the robot's execution delay, calculate the robot's execution time based on the execution delay, and predict the robot's spatial state at the execution time through the world model; The decision-making and execution module is used to obtain the robot's execution instructions based on the predicted spatial state at the execution time, and drive the robot to work according to the execution instructions.

[0033] Obviously, the embodiments described above are merely some, not all, embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

Claims

1. A robot spatial localization method based on spatiotemporal synchronization, characterized in that: S1. Acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor data, and establish a multi-sensor database; S2. Based on the acquisition timestamp, preprocess the initial data of the multi-sensor system to obtain time-aligned multi-sensor data; S3. Construct a world model based on the time-aligned multi-sensor data; S4. Obtain the execution delay duration of the robot, and based on the execution delay duration, predict the spatial state of the robot at the execution moment using the world model; S5. Based on the predicted spatial state, obtain the robot's execution instructions, and drive the robot to work according to the execution instructions.

2. The robot spatial localization method based on spatiotemporal synchronization according to claim 1, characterized in that: The multi-sensor in S1 includes external sensors and internal sensors. Each sensor is set with a collection period. After the multi-sensor collects data, it is automatically uploaded to the multi-sensor database and the multi-sensor database is updated in real time.

3. The robot spatial localization method based on spatiotemporal synchronization according to claim 2, characterized in that: S2 is further defined as follows: S201, Set the fusion time Where k = 0, 1, 2, ... This represents the k-th fusion time. S202. For each sensor, match the distance fusion time in the multi-sensor database. The two most recent valid data points are marked with their collection timestamps. and ,satisfy < < ; Obtain the collection timestamp Corresponding measurement value and the collection timestamp Corresponding measurement value The This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time, the This indicates that the i-th sensor was at the data acquisition timestamp of... The measured value at that time.

4. The robot spatial localization method based on spatiotemporal synchronization according to claim 3, characterized in that: S2 further includes: S203. Select the interpolation model based on the fusion time. Collection timestamp Collection timestamp Measured values and measured values Calculate the fusion time of the i-th sensor. Measured values ; S204, Obtain the fusion time Sensor measurement vector ,in, Indicates the time of fusion of the i-th sensor. The measured value, where N represents the total number of sensors.

5. The robot spatial localization method based on spatiotemporal synchronization according to claim 4, characterized in that: S4 further includes: S401. The state is estimated through the world model, and combined with the robot's motion planning, the first execution time T_st of the robot is determined, and the execution delay T_delay is calculated. The second execution time T_nd = T_st + T_delay of the robot is calculated through the first execution time T_st and the execution delay T_delay. S402. Based on the second execution time T_nd, call the world model again to predict the world state at the second execution time T_nd.

6. The robot spatial localization method based on spatiotemporal synchronization according to claim 5, characterized in that: S401 further includes: S4011. Perform state estimation through the world model to obtain the optimal state estimate at the current moment. Based on the optimal state estimate at the current moment and combined with the robot's motion planning, determine the robot's first execution time T_st, where the first execution time T_st is the moment when the robot plans to execute the instruction. S4012. Obtain the execution delay duration T_delay, where the execution delay includes the communication system transmission delay T_tx, the control system response delay T_rx, and the mechanical system motion delay T_mot; calculate the execution delay duration. ; S4013. Based on the first execution time T_st and the execution delay duration T_delay, calculate the robot's second execution time T_nd = T_st + T_delay, where the second execution time T_nd represents the expected execution time for completing robot execution delay compensation.

7. The robot spatial localization method based on spatiotemporal synchronization according to claim 6, characterized in that: The prediction of the world state at the second execution time T_nd in S402 is specifically as follows: The world model is invoked based on the second execution time T_nd, and the state is estimated again through the world model to obtain the optimal state estimate of the second execution time T_nd, and the world state of the second execution time T_nd is predicted.

8. The robot spatial localization method based on spatiotemporal synchronization according to claim 7, characterized in that: S5 further includes: The robot's own posture is obtained by calling the internal sensor data in the multi-sensor database. Based on the world state at the second execution time T_nd, the robot's motion sequence is calculated to obtain the robot's execution instructions. The robot is then driven to execute according to the execution instructions.

9. A robot spatial positioning system based on spatiotemporal synchronization, used to implement the robot spatial positioning method based on spatiotemporal synchronization as described in claim 8, characterized in that: It includes a data acquisition module, a time alignment module, a world model construction module, a world state prediction module, and a decision-making and execution module. The data acquisition module is used to acquire initial data from multiple sensors, mark the acquisition timestamp for each sensor data, and establish a multi-sensor database. The time alignment module is used to preprocess the raw data of the multi-sensor according to the acquisition timestamp to obtain time-aligned multi-sensor data. The world model building module is used to construct a world model of the robot's external dynamics based on time-aligned multi-sensor data. The world state prediction module is used to calculate the robot's execution delay duration, calculate the robot's execution time based on the execution delay duration, and predict the robot's spatial state at the execution time using the world model; The decision-making and execution module is used to obtain the robot's execution instructions based on the predicted spatial state at the execution time, and drive the robot to work according to the execution instructions.

Images