Methods, devices, equipment, storage media, and program products for robot obstacle avoidance

Abstract

This application discloses a method, apparatus, device, storage medium, and program product for robot obstacle avoidance. The specific technical solution includes: collecting data from multiple sensors within a preset radius range of the robot; calculating the repulsive force between the robot and the dynamic obstacle when a dynamic obstacle is determined to appear in the running path based on the multiple sensor data; adjusting the robot's target running direction and speed according to the repulsive force; calculating the optimal path between the robot and the target position in real time based on a preset home grid map, which includes passageways and static obstacles in the home environment, while the robot runs in the target running direction at the specified speed; and ensuring that the optimal path and the predicted motion trajectory corresponding to the dynamic obstacle do not overlap, allowing the robot to run along the optimal path. This improves the response speed of the obstacle avoidance robot and prevents collisions.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a method, apparatus, device, storage medium and program product for robot obstacle avoidance. Background Technology

[0002] With the rapid development of technologies such as the Internet of Things, artificial intelligence, and big data, smart home products are gradually entering people's daily lives. As an important component of smart homes, automatic obstacle-avoidance home robots can autonomously complete various tasks such as patrolling and providing companionship, effectively reducing the burden on family members and improving their quality of life.

[0003] In a home environment, due to limited space, complex layout, and numerous dynamic and static obstacles, such as family members or pets, and static obstacles such as narrow passages and furniture, obstacles are often present.

[0004] Currently, autonomous movement and obstacle avoidance in home environments are achieved by integrating multiple sensors and navigation algorithms into automated obstacle avoidance home robots. However, in complex home scenarios, traditional automated obstacle avoidance home robots rely on deep learning models. The training and inference processes of deep learning models are slow, which affects the response speed of automated obstacle avoidance home robots. Summary of the Invention

[0005] This application provides a method, apparatus, device, storage medium, and program product for robot obstacle avoidance, which can improve the response speed of obstacle avoidance robots and prevent collisions.

[0006] In a first aspect, embodiments of this application provide a method for robot obstacle avoidance, including:

[0007] Collects data from multiple sensors along the robot's running path;

[0008] If, based on the data from the multiple sensors, a dynamic obstacle is determined to exist within a preset radius range of the robot, the repulsive force between the robot and the dynamic obstacle is calculated.

[0009] The robot's target direction and speed are adjusted according to the repulsive force.

[0010] As the robot moves toward the target direction at the specified speed, the optimal path between the robot and the target location is calculated in real time based on a preset home grid map, which includes passageways and static obstacles in the home environment.

[0011] If the optimal path and the predicted motion trajectory corresponding to the dynamic obstacle do not overlap, the robot runs according to the optimal path.

[0012] In one possible implementation, before calculating the optimal path between the robot and the target location in real time based on a preset home grid map during the robot's movement in the target direction and at the target speed, the method further includes:

[0013] Video image data, point cloud data, and ultra-wideband (UWB) data are collected using multiple sensors.

[0014] The video image data, the point cloud data, and the UWB data are time-synchronized and spatially aligned to obtain fused sensor data, which includes the position information of each pixel.

[0015] Construct a two-dimensional raster map according to the preset raster size;

[0016] The preset family grid map is obtained by determining the grid with obstacles based on the position information of each pixel.

[0017] In one possible implementation, the step of time-synchronizing and spatially aligning the video image data, the point cloud data, and the UWB data to obtain fused sensor data includes:

[0018] The video image data, the point cloud data, and the UWB data are time-synchronized to obtain synchronized video image data, synchronized point cloud data, and synchronized UWB data.

[0019] The transformation matrix from the synchronized point cloud data to the synchronized video image data, and the transformation matrix from the synchronized UWB data to the synchronized point cloud data, are used to construct the particles of the particle swarm optimization algorithm.

[0020] The particle swarm optimization algorithm is used to update the position of each particle, thus obtaining the updated particle position for each particle.

[0021] Based on the correspondence between the updated particle position and the transformation matrix, determine the update transformation matrix corresponding to the updated particle position;

[0022] The registration error between the synchronized point cloud data and the synchronized video image data, as well as the distance error between the synchronized UWB data and the synchronized point cloud data, are calculated using the updated transformation matrix.

[0023] The objective function is calculated using the configuration error and the distance error;

[0024] When the objective function satisfies the preset iteration stopping condition, the synchronized video image data, the synchronized point cloud data, and the synchronized UWB data are transformed according to the update transformation matrix corresponding to the objective function to obtain the fused sensor data.

[0025] In one possible implementation, determining the grid with obstacles based on the position information of each pixel to obtain the preset family grid map includes:

[0026] Based on the timestamps corresponding to the fused sensor data, determine the fused sensor data at different times;

[0027] The static obstacle and the dynamic obstacle are determined using the fused sensor data from different times.

[0028] Based on the position information of the corresponding pixels of the static obstacle, the target grid containing the static obstacle is determined;

[0029] Detect continuous grid cells in the two-dimensional grid map that are free of obstacles, and use these continuous grid cells as the channel;

[0030] The preset family grid map is constructed based on the target grid and the continuous grid.

[0031] One possible implementation also includes:

[0032] If a dynamic obstacle is determined to appear in the running path based on the data from the multiple sensors, the distance between the dynamic obstacle and the robot is determined in real time based on the data from the multiple sensors.

[0033] If the distance is less than a preset threshold, the robot will adjust its running speed to 0.

[0034] In one possible implementation, before the robot runs along the optimal path, the method further includes:

[0035] The speed and direction of the dynamic obstacle are obtained;

[0036] The predicted position of the dynamic obstacle at different times is determined based on its running speed and direction, thus obtaining the predicted motion trajectory.

[0037] Secondly, embodiments of this application provide a device for robot obstacle avoidance, comprising:

[0038] The data acquisition module is used to collect data from various sensors along the robot's running path.

[0039] The calculation module is used to calculate the repulsive force between the robot and the dynamic obstacle when it is determined, based on the data from the multiple sensors, that a dynamic obstacle appears within a preset radius of the robot.

[0040] An adjustment module is used to adjust the target running direction and speed of the robot according to the repulsive force;

[0041] The calculation module is also used to calculate the optimal path between the robot and the target location in real time based on a preset home grid map during the process of the robot running towards the target direction at the speed. The preset home grid map includes passages and static obstacles in the home environment.

[0042] The operation module is used to ensure that the robot runs along the optimal path when the predicted motion trajectories corresponding to the optimal path and the dynamic obstacles do not overlap.

[0043] Thirdly, embodiments of this application provide a terminal device, the device including: a processor and a memory storing computer program instructions;

[0044] The method for robot obstacle avoidance, as described in the first aspect, is implemented when the processor executes computer program instructions.

[0045] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the robot obstacle avoidance method as described in the first aspect is implemented.

[0046] In another aspect, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a robot obstacle avoidance method as described in the first aspect.

[0047] This application discloses a method, apparatus, device, storage medium, and program product for robot obstacle avoidance. During robot operation, it collects various sensor data along the robot's path. Based on this data, it determines whether dynamic obstacles exist within a preset radius. If dynamic obstacles are present, the robot's direction and speed are adjusted using the repulsive force between the robot and the obstacle, thus achieving emergency obstacle avoidance. While the robot operates at the adjusted target direction and speed, it calculates the optimal path between itself and the target location in real time based on a preset home grid map. This preset home grid map is a pre-constructed map containing passageways and static obstacles within the home environment. By utilizing reusable historical data from the preset home grid map, global path planning is possible for static obstacles. The robot operates along the optimal path provided the calculated optimal path does not overlap with the predicted trajectory of the dynamic obstacle. This simplifies the processing and reduces the robot's reliance on sensors and deep learning models, thereby improving system response speed. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a robot obstacle avoidance method provided in an embodiment of this application;

[0050] Figure 2 This is a flowchart illustrating a method for constructing a preset family grid map according to an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of the structure of a robot obstacle avoidance system provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a robot obstacle avoidance device provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0055] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0056] Currently, there are three methods for robot obstacle avoidance:

[0057] The first approach involves the robot capturing video images in real time and using a deep learning model to extract features from these images. By comparing these features with locally stored obstacle features, the robot identifies obstacles in the scene, thus achieving obstacle avoidance. However, this method requires significant computational resources, and the training and inference processes of the deep learning model are slow, limiting the system's response speed. Furthermore, this method relies heavily on the image quality of the video images; in low-light conditions, obstacle recognition can be affected, thus impacting obstacle avoidance functionality.

[0058] The second approach involves the robot acquiring environmental images using LiDAR, 3D Time-of-Flight (3D TOF), and 3D light structure technology. Image algorithms are then used to analyze obstacle contours, and based on obstacle positions and path relationships, the robot's running path is determined. However, this method requires complex image processing algorithms, increasing the computational burden and thus affecting system response speed.

[0059] The third approach involves the robot autonomously making obstacle avoidance decisions based on its current linear and angular velocities using a pre-trained deep learning model. However, this method relies heavily on deep learning models, resulting in high system costs and a lengthy training process. Furthermore, the long inference time of deep learning models during operation can lead to excessively high system response latency.

[0060] To address the problems existing in the prior art, embodiments of this application provide a method, apparatus, device, storage medium, and program product for robot obstacle avoidance. The following first describes the robot obstacle avoidance method provided in this application embodiment. Figure 1 As shown, the method includes:

[0061] S101. Collect data from various sensors along the robot's running path.

[0062] The robot integrates various sensors, specifically multiple sensors that can be integrated at the center of the top of the robot. For example, the sensors can be LiDAR sensors, depth cameras, and ultra-wideband (UWB) modules.

[0063] S102. When dynamic obstacles are found within the robot's preset radius range based on data from multiple sensors, calculate the repulsive force between the robot and the dynamic obstacles.

[0064] The method for determining whether dynamic obstacles appear within the robot's preset radius will be described in detail in subsequent embodiments.

[0065] Specifically, a pre-created potential field model is used, where obstacles generate a repulsive force, and the target position corresponding to the end of the motion path generates an attractive force. The repulsive force is used to prevent the robot from approaching the obstacle. The repulsive force can be calculated using Equation 1:

[0066] Formula 1

[0067] in, Indicates repulsive force. The repulsive force constant is a preset value. Used to indicate the range of influence of an obstacle, it is usually a fixed value. This represents the distance from the robot to the i-th obstacle. It is the unit vector from the obstacle to the robot.

[0068] S103. Adjust the robot's target running direction and speed according to the repulsive force.

[0069] In one example, by weighted summing the repulsive force and the current robot's direction vector, the resultant force corresponding to the current robot can be obtained. The direction and magnitude of this resultant force can then be used as the robot's target direction and velocity. It should be noted that the method described above for adjusting the robot's target direction and velocity using repulsive force is only an example. In actual implementations, the methods for calculating repulsive force and adjusting the target direction and velocity using repulsive force are not limited to this.

[0070] S104. During the process of the robot running in the target direction at the speed, the optimal path between the robot and the target position is calculated in real time based on the preset family grid map.

[0071] The preset home grid map includes passageways and static obstacles in the home environment.

[0072] It should be noted that a lightweight path planning algorithm can be preset in the robot. Whenever the robot deviates from the previously calculated running path, the optimal path between the robot's current position and the target position is calculated in real time according to the preset lightweight path planning algorithm.

[0073] In one example, the default lightweight path planning algorithm can be Dijkstra's algorithm.

[0074] S105. If the predicted motion trajectories corresponding to the optimal path and the dynamic obstacles do not overlap, the robot runs according to the optimal path.

[0075] The predicted motion trajectory is obtained based on data from multiple sensors.

[0076] Using the above method, various sensor data are collected along the robot's path during operation. Based on this data, it can be determined whether dynamic obstacles exist within a preset radius. If such obstacles are present, the repulsive force between the robot and the obstacle is used to adjust the robot's direction and speed, thus achieving emergency obstacle avoidance. Then, as the robot moves according to the adjusted target direction and speed, it calculates the optimal path between itself and the target location in real time based on a preset home grid map. This preset home grid map is a pre-constructed map containing passageways and static obstacles within the home environment. By utilizing reusable historical data from this map, global path planning can be performed for static obstacles. As long as the calculated optimal path does not overlap with the predicted trajectory of the dynamic obstacle, the robot follows the optimal path, simplifying the processing and reducing its reliance on sensors and deep learning models, thereby improving system response speed.

[0077] The following first receives the process of building a preset family grid map, such as Figure 2 As shown, before S104 above, during the process of the robot running in the target direction and at the target speed, and before calculating the optimal path between the robot and the target position in real time based on the preset home grid map, the method further includes:

[0078] S201: Collects video image data, point cloud data, and ultra-wideband (UWB) data through multiple sensors.

[0079] The video image data includes depth information for each pixel.

[0080] S202. Perform time synchronization and spatial alignment on video image data, point cloud data, and UWB data to obtain fused sensor data, which includes the position information of each pixel.

[0081] Specifically, the video image data, point cloud data, and ultra-wideband (UWB) data collected by the robot through various sensors will carry a timestamp corresponding to the time of data collection. Therefore, the time synchronization of video image data, point cloud data, and UWB data can be achieved through the timestamp.

[0082] Understandably, by synchronizing and aligning video image data, point cloud data, and UWB data in time and space, the positional information of each pixel in the video image data can be obtained, thereby constructing a three-dimensional spatial model corresponding to the home environment.

[0083] Specifically, time synchronization and spatial alignment can be performed by following these steps:

[0084] Step 1: Synchronize the video image data, point cloud data, and UWB data in time to obtain synchronized video image data, synchronized point cloud data, and synchronized UWB data.

[0085] Specifically, the time synchronization method is described in the relevant embodiments above, and will not be repeated here.

[0086] Step 2: Initialize the particle positions of the particle swarm optimization algorithm using the transformation matrix from synchronized point cloud data to synchronized video image data and the transformation matrix from synchronized UWB data to synchronized point cloud data.

[0087] Specifically, the transformation matrix from synchronized point cloud data to synchronized video image data and the transformation matrix from synchronized UWB data to synchronized point cloud data can be used as the particle's position coordinates, and can be expressed as: .in, This represents the transformation matrix from synchronized point cloud data to synchronized video image data. This represents the transformation matrix from synchronized UWB data to synchronized point cloud data.

[0088] Step 3: Use the particle swarm optimization algorithm to update the position of each particle to obtain the updated particle position for each particle.

[0089] Specifically, the particle's position can be updated from two dimensions: velocity and position. The particle's velocity can be updated according to Formula 2:

[0090] Formula 2

[0091] in, Indicates the updated speed. This represents the inertial weight, which controls the degree to which the particle's velocity is maintained, preventing the particle from leaving the current area too quickly; Represents the self-awareness factor, controlling the particle to its optimal position. The force of the movement; Representing social cognitive factors, it controls the particle's trajectory towards the globally optimal position. The force of the movement; and It is a random number, usually in the range [0, 1].

[0092] The particle's position can be updated according to Formula 3:

[0093] Formula 3

[0094] in, This indicates the updated position.

[0095] Step 4: Determine the update transformation matrix corresponding to the updated particle position according to the correspondence between the updated particle position and the transformation matrix.

[0096] Step 5: Calculate the registration error between synchronized point cloud data and synchronized video image data, and the distance error between synchronized UWB data and synchronized point cloud data using the update transformation matrix.

[0097] The registration error and distance error were calculated using formulas 4 and 5 respectively:

[0098] Formula 4

[0099] in, Indicates the registration error. This indicates the synchronization of point cloud data. This indicates synchronized video image data.

[0100] Formula 5

[0101] in, Indicates distance error. This indicates that UWB data is being synchronized.

[0102] Step 6: Calculate the objective function using configuration error and distance error.

[0103] The objective function can be defined as follows:

[0104] Formula 5

[0105] in, Let be the objective function. Preset weights.

[0106] Step 7: If the objective function satisfies the preset iteration stopping condition, transform the synchronized video image data, synchronized point cloud data, and synchronized UWB data according to the update transformation matrix corresponding to the objective function to obtain fused sensor data.

[0107] The preset iteration stopping condition can be reaching the maximum number of iterations or the convergence of the objective function value.

[0108] Thus, by continuously updating the position of each particle through the particle swarm optimization algorithm, the transformation matrix can be iteratively updated. After iteratively calculating the objective function using the updated transformation matrix, the registration error between point cloud data, video image data, and UWB data can be calculated. In order to minimize the registration error and improve the registration accuracy, when the objective function meets the preset iteration stopping condition, the synchronous video image data, synchronous point cloud data, and synchronous UWB data are transformed according to the transformation matrix corresponding to the current objective function, thereby ensuring the registration accuracy of the transformed data.

[0109] It should be noted that a Kalman filter is used to fuse point cloud data and video image data, using the data as different observation inputs to update the observation matrix H and the observation noise covariance matrix R for fusion.

[0110] S203. Construct a two-dimensional raster map according to the preset raster size.

[0111] S204. Determine the grid with obstacles based on the position information of each pixel to obtain a preset family grid map.

[0112] After constructing a two-dimensional grid map according to the preset grid size, the grid where the robot is located can be determined based on the robot's position. Then, based on the position information of each pixel in the video image data, the grid where each pixel is located can be determined, and the grid containing the pixel is identified as the grid containing the obstacle.

[0113] Specifically, you can build a preset family grid map by following these steps:

[0114] Step A: Determine the fused sensor data at different times based on the timestamps corresponding to the fused sensor data.

[0115] Step B: Use fused sensor data from different times to determine static and dynamic obstacles.

[0116] Understandably, for the same location, if the fused sensor data at different times is different, it indicates that the obstacle at that location is a dynamic obstacle.

[0117] Specifically, dynamic obstacles can be identified using image differencing or background modeling methods.

[0118] Step C: Determine the target grid containing static obstacles based on the position information of the corresponding pixels of the static obstacles.

[0119] Step D: Detect continuous grid cells in the 2D grid map that are free of obstacles, and use these continuous grid cells as channels.

[0120] Step E: Construct a preset family grid map based on the target grid and continuous grids.

[0121] Thus, by fusing sensor data at different times, static and dynamic obstacles can be identified. For static obstacles, their location information allows determination of the target grid cell they occupy in the 2D grid map. Furthermore, since the grid cells occupied by a passage are connected, the passage area can be determined by detecting continuous grid cells without obstacles in the 2D grid map. Based on this, the static obstacle area and passage area in the 2D grid map can be identified, facilitating subsequent path planning by the robot based on these areas.

[0122] It should be noted that during the robot's operation, the fused sensor data changes depending on the robot's location. Therefore, the robot can update the preset home grid map according to the fused sensor data determined at different times, ensuring the correctness of the preset home grid map and thus ensuring the accuracy of the subsequently generated optimal path.

[0123] Using the embodiments of this application, after acquiring video image data, point cloud data, and ultra-wideband (UWB) data, the depth and distance information corresponding to each pixel in the video image can be determined through time and spatial synchronization, thereby determining the location information of the pixel. Based on the location information, the grid in the two-dimensional grid map can be filled to identify grids containing obstacles. Thus, the two-dimensional grid map can be filled according to the real-time acquired sensor data to obtain a preset home grid map. Subsequently, the preset home grid map can be directly used for path planning, reducing computational resource consumption and improving path planning efficiency.

[0124] It should be noted that, since dynamic obstacles also exist in the home environment, in order to accurately determine whether dynamic obstacles affect the robot's operation, the trajectory of the dynamic obstacles can be determined using data from multiple sensors, thereby further determining whether a collision will occur between the dynamic obstacles and the robot. Based on this, in step S102 above, before calculating the repulsive force between the robot and the dynamic obstacle when dynamic obstacles are determined to appear in the running path based on multiple sensor data, the method further includes:

[0125] The speed and direction of movement of the dynamic obstacle are obtained; the predicted position of the dynamic obstacle at different times is determined based on the speed and direction of movement of the dynamic obstacle, and the predicted motion trajectory is obtained.

[0126] Using the method provided in this application, the running direction and speed of a dynamic obstacle can be determined through data from multiple sensors. Therefore, the predicted trajectory data of the dynamic obstacle can be predicted based on its current running speed and direction. Then, based on the predicted trajectory data and the robot's running speed, the robot can determine whether a collision with the dynamic obstacle will occur at its current speed. If a collision is predicted, the robot's running path will be considered to contain a dynamic obstacle. In this way, it is possible to predict in advance whether a collision with a dynamic obstacle will occur, thereby allowing for timely adjustments to the robot's direction and speed, achieving timely obstacle avoidance.

[0127] In some embodiments of this application, when a dynamic obstacle is determined to appear in the running path based on multiple sensor data, the robot can also determine the distance between the dynamic obstacle and the robot in real time based on multiple sensor data; if the distance is less than a preset threshold, the robot adjusts its running speed to 0. Thus, by monitoring the distance in real time, in unpredictable situations where the dynamic obstacle accelerates and the robot cannot adjust its direction in time, the robot can directly adjust its running speed to 0 and remain stationary, achieving emergency obstacle avoidance.

[0128] The following combination Figure 3 This application describes the complete process of the robot obstacle avoidance method provided in the embodiments, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a robot obstacle avoidance system provided in an embodiment of this application.

[0129] The system architecture includes an environment modeling module, a scene segmentation and obstacle classification module, a real-time obstacle avoidance strategy module, and a monitoring and feedback module.

[0130] The environment modeling module includes an environment perception unit and a data fusion unit. The environment perception unit acquires different sensor data through various sensors integrated on the robot, while the data fusion unit performs time synchronization and spatial alignment of the various sensor data.

[0131] The scene segmentation and obstacle classification module includes a home scene segmentation unit and an obstacle detection unit. The home scene segmentation unit identifies narrow passages, changing areas, and complex furniture layouts based on a preset home grid map. The obstacle detection unit distinguishes between static and dynamic obstacles within the home scene.

[0132] The real-time obstacle avoidance strategy module includes a path generation unit, a local adjustment unit, and an emergency adjustment unit. The path generation unit generates the optimal path based on a path generation algorithm. The local adjustment unit adjusts the robot's direction and speed based on the trajectory of the dynamic obstacle. The emergency adjustment unit performs emergency braking on the robot if a collision with the dynamic obstacle is predicted.

[0133] The monitoring and feedback module includes a real-time performance monitoring unit, a feedback adjustment unit, a computing resource unit, and a success rate calculation unit. The real-time performance monitoring unit monitors the latency and computational cost of the robot obstacle avoidance system, with system latency including data acquisition, processing, and path planning time. The computing resource unit assesses computing resource usage to ensure it remains within acceptable limits. The success rate calculation unit calculates and records obstacle avoidance successes and failures. The feedback adjustment unit analyzes system performance based on the records and identifies areas for improvement. Based on the identified results, the obstacle avoidance algorithm and environmental modeling are adjusted to improve system performance and stability.

[0134] Furthermore, by modularizing the aforementioned robot obstacle avoidance system, and by enabling multi-language compatibility between different modules in the Robot Operating System (ROS), a one-click update interface can be provided on the robot's user interface. After developers publish updates to the cloud, the system software and algorithms can be updated, vulnerabilities fixed, and performance improved via this interface.

[0135] Based on the same concept, embodiments of this application also provide a device for robot obstacle avoidance, such as... Figure 4 As shown, the device includes:

[0136] The acquisition module 401 is used to acquire various sensor data corresponding to the robot's running path;

[0137] The calculation module 402 is used to calculate the repulsive force between the robot and the dynamic obstacle when it is determined, based on the data from the multiple sensors, that a dynamic obstacle appears within a preset radius range of the robot.

[0138] Adjustment module 403 is used to adjust the target running direction and speed of the robot according to the repulsive force;

[0139] The calculation module 402 is also used to calculate the optimal path between the robot and the target location in real time based on a preset home grid map while the robot is running towards the target direction at the speed. The preset home grid map includes passages and static obstacles in the home environment.

[0140] The operation module 404 is used to ensure that the robot runs along the optimal path when the predicted motion trajectory corresponding to the optimal path and the dynamic obstacle does not overlap.

[0141] In one possible implementation, the device further includes:

[0142] The acquisition module 401 is also used to acquire video image data, point cloud data and ultra-wideband (UWB) data through multiple sensors;

[0143] An alignment module is used to perform time synchronization and spatial alignment on the video image data, the point cloud data, and the UWB data to obtain fused sensor data, which includes the position information of each pixel.

[0144] The building module is used to construct a two-dimensional raster map according to a preset raster size;

[0145] The determination module is used to determine the grid cells containing obstacles based on the position information of each pixel, thereby obtaining the preset family grid map.

[0146] In one possible implementation, the alignment module is specifically used for:

[0147] The video image data, the point cloud data, and the UWB data are time-synchronized to obtain synchronized video image data, synchronized point cloud data, and synchronized UWB data.

[0148] The transformation matrix from the synchronized point cloud data to the synchronized video image data, and the transformation matrix from the synchronized UWB data to the synchronized point cloud data, are used to construct the particles of the particle swarm optimization algorithm.

[0149] The particle swarm optimization algorithm is used to update the position of each particle, thus obtaining the updated particle position for each particle.

[0150] Based on the correspondence between the updated particle position and the transformation matrix, determine the update transformation matrix corresponding to the updated particle position;

[0151] The registration error between the synchronized point cloud data and the synchronized video image data, as well as the distance error between the synchronized UWB data and the synchronized point cloud data, are calculated using the updated transformation matrix.

[0152] The objective function is calculated using the configuration error and the distance error;

[0153] When the objective function satisfies the preset iteration stopping condition, the synchronized video image data, the synchronized point cloud data, and the synchronized UWB data are transformed according to the update transformation matrix corresponding to the objective function to obtain the fused sensor data.

[0154] In one possible implementation, a module is defined, specifically for:

[0155] Based on the timestamps corresponding to the fused sensor data, determine the fused sensor data at different times;

[0156] The static obstacle and the dynamic obstacle are determined using the fused sensor data from different times.

[0157] Based on the position information of the corresponding pixels of the static obstacle, the target grid containing the static obstacle is determined;

[0158] Detect continuous grid cells in the two-dimensional grid map that are free of obstacles, and use these continuous grid cells as the channel;

[0159] The preset family grid map is constructed based on the target grid and the continuous grid.

[0160] In one possible implementation, the device further includes:

[0161] The monitoring module is used to determine the distance between the dynamic obstacle and the robot in real time based on the multiple sensor data when it is determined that a dynamic obstacle appears in the running path.

[0162] An adjustment module is used to adjust the robot's operating speed to 0 when the distance is less than a preset threshold.

[0163] In one possible implementation, the device further includes:

[0164] The acquisition module is used to acquire the running speed and running direction of the dynamic obstacle;

[0165] The prediction module is used to determine the predicted position of the dynamic obstacle at different times based on its running speed and direction, and to obtain the predicted motion trajectory.

[0166] It should be noted that the obstacle avoidance device for the robot is the same as the obstacle avoidance method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0167] Figure 5 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0168] The electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0169] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0170] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0171] In a particular embodiment, memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0172] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any of the robot obstacle avoidance methods in the above embodiments.

[0173] In one example, the electronic device may also include a communication interface 503 and a bus 504. For example, Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 504 and complete communication with each other.

[0174] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0175] Bus 504 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Super Transmission (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 504 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0176] Furthermore, in conjunction with the robot obstacle avoidance methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the robot obstacle avoidance methods in the above embodiments.

[0177] This application also provides a computer program product, including a computer program that, when executed, implements any of the robot obstacle avoidance methods described in the above embodiments.

[0178] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0179] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on machine-readable media or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0180] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0181] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0182] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for robot obstacle avoidance, characterized in that, include: Collect data from various sensors along the robot's running path; If, based on the data from the multiple sensors, a dynamic obstacle is determined to exist within a preset radius range of the robot, the repulsive force between the robot and the dynamic obstacle is calculated. The robot's target direction and speed are adjusted according to the repulsive force. As the robot moves toward the target direction at the specified speed, the optimal path between the robot and the target location is calculated in real time based on a preset home grid map, which includes passageways and static obstacles in the home environment. If the optimal path and the predicted motion trajectory corresponding to the dynamic obstacle do not overlap, the robot runs according to the optimal path; Before calculating the optimal path between the robot and the target location in real time based on a preset home grid map during the robot's movement in the target direction and at the target speed, the method further includes: Video image data, point cloud data, and ultra-wideband (UWB) data are collected using multiple sensors. The video image data, the point cloud data, and the UWB data are time-synchronized and spatially aligned to obtain fused sensor data, which includes the position information of each pixel. Construct a two-dimensional raster map according to the preset raster size; The grid with obstacles is determined according to the position information of each pixel to obtain the preset family grid map; The step of performing time synchronization and spatial alignment on the video image data, the point cloud data, and the UWB data to obtain fused sensor data includes: The video image data, the point cloud data, and the UWB data are time-synchronized to obtain synchronized video image data, synchronized point cloud data, and synchronized UWB data. The particle positions of the particle swarm optimization algorithm are initialized using the transformation matrix from the synchronized point cloud data to the synchronized video image data and the transformation matrix from the synchronized UWB data to the synchronized point cloud data. The particle swarm optimization algorithm is used to update the position of each particle, thus obtaining the updated particle position for each particle. Based on the correspondence between the updated particle position and the transformation matrix, determine the update transformation matrix corresponding to the updated particle position; The registration error between the synchronized point cloud data and the synchronized video image data, as well as the distance error between the synchronized UWB data and the synchronized point cloud data, are calculated using the updated transformation matrix. The objective function is calculated using the registration error and the distance error; When the objective function satisfies the preset iteration stopping condition, the synchronized video image data, the synchronized point cloud data, and the synchronized UWB data are transformed according to the update transformation matrix corresponding to the objective function to obtain the fused sensor data.

2. The method according to claim 1, characterized in that, The step of determining the grid with obstacles based on the position information of each pixel to obtain the preset family grid map includes: Based on the timestamps corresponding to the fused sensor data, determine the fused sensor data at different times; The static obstacle and the dynamic obstacle are determined using the fused sensor data from different times. Based on the position information of the corresponding pixels of the static obstacle, the target grid containing the static obstacle is determined; Detect continuous grid cells in the two-dimensional grid map that are free of obstacles, and use these continuous grid cells as the channel; The preset family grid map is constructed based on the target grid and the continuous grid.

3. The method according to claim 1, characterized in that, Also includes: If a dynamic obstacle is determined to appear in the running path based on the data from the multiple sensors, the distance between the dynamic obstacle and the robot is determined based on the data from the multiple sensors. If the distance is less than a preset threshold, the robot will adjust its running speed to 0.

4. The method according to claim 1, characterized in that, Before the robot runs along the optimal path, the method further includes: The speed and direction of the dynamic obstacle are obtained; The predicted position of the dynamic obstacle at different times is determined based on its running speed and direction, thus obtaining the predicted motion trajectory.

5. A device for obstacle avoidance in robots, characterized in that, include: The data acquisition module is used to collect data from various sensors along the robot's running path. The calculation module is used to calculate the repulsive force between the robot and the dynamic obstacle when it is determined, based on the data from the multiple sensors, that a dynamic obstacle appears within a preset radius range of the robot. An adjustment module is used to adjust the target running direction and speed of the robot according to the repulsive force; The calculation module is also used to calculate the optimal path between the robot and the target location in real time based on a preset home grid map during the process of the robot running towards the target direction at the speed. The preset home grid map includes passages and static obstacles in the home environment. The operation module is configured to allow the robot to run along the optimal path when the optimal path and the predicted motion trajectory corresponding to the dynamic obstacle do not overlap. The device further includes: The acquisition module is also used to acquire video image data, point cloud data, and ultra-wideband (UWB) data through multiple sensors; The fusion module is used to perform time synchronization and spatial alignment on the video image data, the point cloud data and the UWB data to obtain fused sensor data, which includes the position information of each pixel. The building module is used to construct a two-dimensional raster map according to a preset raster size; The determination module is used to determine the grid cells containing obstacles according to the position information of each pixel, and to obtain the preset family grid map; The fusion module is specifically used for: The video image data, the point cloud data, and the UWB data are time-synchronized to obtain synchronized video image data, synchronized point cloud data, and synchronized UWB data. The particle positions of the particle swarm optimization algorithm are initialized using the transformation matrix from the synchronized point cloud data to the synchronized video image data and the transformation matrix from the synchronized UWB data to the synchronized point cloud data. The particle swarm optimization algorithm is used to update the position of each particle, thus obtaining the updated particle position for each particle. Based on the correspondence between the updated particle position and the transformation matrix, determine the update transformation matrix corresponding to the updated particle position; The registration error between the synchronized point cloud data and the synchronized video image data, as well as the distance error between the synchronized UWB data and the synchronized point cloud data, are calculated using the updated transformation matrix. The objective function is calculated using the registration error and the distance error; When the objective function satisfies the preset iteration stopping condition, the synchronized video image data, the synchronized point cloud data, and the synchronized UWB data are transformed according to the update transformation matrix corresponding to the objective function to obtain the fused sensor data.

6. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the robot obstacle avoidance method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the robot obstacle avoidance method as described in any one of claims 1-4.

8. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the robot obstacle avoidance method as described in any one of claims 1-4.

Images