Obstacle avoidance method of robot, robot and storage medium

Abstract

The embodiment of the invention relates to the technical field of robots, and discloses an obstacle avoidance method of a robot, the robot and a storage medium, and the method comprises the steps: firstly obtaining a historical track of a dynamic obstacle, then inputting the historical track into an LSTM prediction model, generating a prediction track of the dynamic obstacle and a corresponding confidence coefficient, and based on the confidence coefficient and the prediction track, obtaining a prediction result of the dynamic obstacle; and adopting a DWA algorithm to generate an avoidance path, and controlling the robot to walk according to the avoidance path so as to avoid the obstacle. According to the obstacle avoidance method, trajectory prediction is performed on the dynamic obstacle, the motion trend of the dynamic obstacle is sensed in advance, and the corresponding avoidance path is adaptively generated based on the motion trend, so that the avoidance path is more reasonable, collision with the dynamic obstacle is effectively avoided, the obstacle avoidance performance of the robot is improved, and the robot experience is improved. And therefore, the safety and the working efficiency of the robot are improved, the robot better adapts to the dynamically changing environment, and the adaptability of the robot is enhanced.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of robots, and in particular, to an obstacle avoidance method, a robot, and a storage medium for a robot. Background Art

[0002] With the rapid development of technology, mobile robots are applied in various fields. When a robot executes related tasks, such as a cleaning task, the environment it is in is relatively complex and may be disturbed by dynamic obstacles such as animals, pedestrians, and other moving objects. The robot needs to avoid these dynamic obstacles to prevent accidents such as collisions.

[0003] In traditional obstacle avoidance methods, obstacles are generally detected by lidar, and simple collision detection is used to avoid obstacles. For dynamic obstacles, existing obstacle avoidance schemes often have a single obstacle avoidance path, poor adaptability, are prone to collisions, and have low obstacle avoidance performance, reducing the working efficiency and safety of the robot. Summary of the Invention

[0004] Embodiments of the present application aim to provide an obstacle avoidance method, a robot, and a storage medium for a robot, which can plan a more reasonable adaptive avoidance path, improve the obstacle avoidance performance of the robot, and further improve the working efficiency and safety of the robot.

[0005] In a first aspect, some embodiments of the present application provide an obstacle avoidance method for a robot, including:

[0006] Obtaining the historical trajectory of a dynamic obstacle;

[0007] Inputting the historical trajectory into an LSTM prediction model to generate a predicted trajectory of the dynamic obstacle and a corresponding confidence level, where the confidence level represents the reliability of the predicted trajectory;

[0008] Based on the confidence level and the predicted trajectory, using the DWA algorithm to generate an avoidance path;

[0009] Controlling the robot to walk along the avoidance path to avoid the dynamic obstacle.

[0010] In some embodiments, the obtaining the historical trajectory of the dynamic obstacle includes:

[0011] Determining the state prediction data corresponding to N consecutive time instants as the historical trajectory, where N is a positive integer greater than 1, and the state prediction data represents the predicted state of the dynamic obstacle at the current time instant.

[0012] In some embodiments, before the obtaining the historical trajectory of the dynamic obstacle, the method further includes:

[0013] Obtain the observation data of the dynamic obstacle;

[0014] Based on the observation data, use the Kalman filtering algorithm to generate the state prediction data at the current moment.

[0015] In some embodiments, the step of using the Kalman filtering algorithm to generate the state prediction data at the current moment based on the observation data includes:

[0016] Obtain the state prediction data and the motion model at the previous moment;

[0017] Predict the original state prediction data at the current moment according to the state prediction data and the motion model at the previous moment;

[0018] Fuse the observation data and the original state prediction data to generate the state prediction data at the current moment.

[0019] In some embodiments, the step of fusing the observation data and the original state prediction data to generate the state prediction data at the current moment includes:

[0020] Calculate the difference between the observation data and the original state prediction data;

[0021] Calculate the product of the difference and the Kalman gain matrix;

[0022] Add the product to the observation data to generate the state prediction data at the current moment.

[0023] In some embodiments, before the step of using the DWA algorithm to generate an avoidance path based on the confidence level and the predicted trajectory, the method further includes:

[0024] If the confidence level is greater than or equal to the first preset threshold, start the step of using the DWA algorithm to generate an avoidance path based on the confidence level and the predicted trajectory;

[0025] If the confidence level is less than the first preset threshold, trigger the line laser sensor to scan and verify whether the obstacle is a dynamic obstacle. If it is verified that the obstacle is a dynamic obstacle, start the step of using the DWA algorithm to generate an avoidance path based on the confidence level and the predicted trajectory.

[0026] In some embodiments, before the step of triggering the line laser sensor to scan and verify whether the obstacle is a dynamic obstacle, the method further includes:

[0027] If the confidence level is less than the second preset threshold, control the speed of the robot to decrease to the first speed, where the second preset threshold is less than the first preset threshold.

[0028] In some embodiments, before generating an avoidance path using the DWA algorithm based on the confidence and the predicted trajectory, the method further includes:

[0029] Obtaining the speed data and altitude data in the state prediction data at the current moment;

[0030] If the speed data is greater than a first speed threshold or the altitude data is greater than a first altitude threshold, control the robot to stop and then walk a preset distance in a first direction, where the first direction is the opposite direction to the current walking direction of the robot;

[0031] If the speed data is less than a second speed threshold or the altitude data is less than a second altitude threshold, start the step of generating an avoidance path using the DWA algorithm based on the confidence and the predicted trajectory, where the second speed threshold is less than the first speed threshold, and the second altitude threshold is less than the first altitude threshold.

[0032] In some embodiments, generating an avoidance path using the DWA algorithm based on the confidence and the predicted trajectory includes:

[0033] Determining constraint conditions, where the constraint conditions include the current speed limit of the robot;

[0034] Determining a dynamic window based on the confidence and the constraint conditions, where the dynamic window includes the value range of the speed of the robot;

[0035] Uniformly sampling the speed of the robot within the dynamic window to obtain multiple sets of sampled speeds;

[0036] For each set of the sampled speeds, predicting the motion path of the robot at the sampled speed based on the motion model of the robot to obtain a first candidate path;

[0037] Eliminating the paths that intersect with the predicted trajectory from the first candidate paths to obtain second candidate paths;

[0038] Constructing an evaluation function based on the target direction, path smoothness, static obstacle distance, and dynamic safety term;

[0039] Evaluating the second candidate paths using the evaluation function to generate corresponding evaluation scores;

[0040] Determining the second candidate path with the highest evaluation score as the avoidance path.

[0041] In some embodiments, obtaining the observation data of the dynamic obstacle includes:

[0042] Multiple sensors are used to observe the dynamic obstacle, and visual data, point cloud data, and line laser data are collected;

[0043] A preset sliding window is determined, where the preset sliding window is a time range centered on a preset moment;

[0044] The target visual data, target point cloud data, and target line laser data under the preset sliding window are obtained;

[0045] The target visual data, the target point cloud data, and the target line laser data are converted into data in the robot coordinate system to obtain the observation data.

[0046] In some embodiments, after controlling the robot to walk along the avoidance path, the method further includes:

[0047] Monitoring the state of the dynamic obstacle to generate state information;

[0048] If the state information indicates that the dynamic obstacle is in a leaving state, controlling the robot to continue cleaning along the cleaning path;

[0049] If the state information indicates that the dynamic obstacle is in a continuous staying state, marking the area where the dynamic obstacle is located as a dynamic risk area, and after the cleaning task is completed, returning to the dynamic risk area for supplementary cleaning.

[0050] In a second aspect, some embodiments of the present application provide a robot, including a memory and a processor, the memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the robot implements the obstacle avoidance method of the robot as described above.

[0051] In a third aspect, some embodiments of the present application provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the obstacle avoidance method of the robot as described above.

[0052] The beneficial effects of the embodiments of the present application are as follows: Different from the prior art, the obstacle avoidance method provided by the embodiments of the present application includes: first obtaining the historical trajectory of a dynamic obstacle, then inputting the historical trajectory into an LSTM prediction model to generate a predicted trajectory of the dynamic obstacle and a corresponding confidence score, then using a DWA algorithm to generate an avoidance path based on the confidence score and the predicted trajectory, and controlling the robot to move along the avoidance path to avoid the obstacle. This obstacle avoidance method predicts the trajectory of a dynamic obstacle, perceives the movement trend of the dynamic obstacle in advance, and adaptively generates a corresponding avoidance path based on the movement trend, making the avoidance path more reasonable, effectively avoiding collisions with dynamic obstacles, improving the robot's obstacle avoidance performance, and thereby improving the robot's safety and work efficiency, and better adapting to dynamically changing environments, thereby enhancing the robot's adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0054] Figure 1 A schematic diagram of the application environment of one of the robot obstacle avoidance methods provided in this application;

[0055] Figure 2 A schematic diagram of the structure of one of the robots provided in this application;

[0056] Figure 3 A schematic diagram of a flow chart of one of the robot obstacle avoidance methods provided in this application;

[0057] Figure 4 A schematic diagram of a flow chart of one of the robot obstacle avoidance methods provided in this application;

[0058] Figure 5 Provided for this application Figure 4 Flow chart of step S50;

[0059] Figure 6 Provided for this application Figure 4 Flow diagram of step S60;

[0060] Figure 7 Provided for this application Figure 3 Schematic diagram of the process of step S30;

[0061] Figure 8 A schematic diagram of the flow chart of one of the robot obstacle avoidance methods provided in this application. DETAILED DESCRIPTION

[0062] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made. These all fall within the protection scope of the present application.

[0063] In order to make the purpose, technical solution and advantages of the present application more clear, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that if there is no conflict, the various features in the embodiments of the present application can be combined with each other, and all are within the protection scope of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the flowchart. In addition, the terms "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0065] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in this specification in the description of the present application are only for the purpose of describing specific embodiments and are not used to limit the present application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0066] In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0067] In the embodiments of the present application, the robot can be a mobile device capable of providing functional services, such as: cleaning robots, delivery robots, nursing robots, remote monitoring robots, etc. The following takes the cleaning robot as an example to illustrate the obstacle avoidance method of the robot provided in the embodiments of the present application. The cleaning robot includes but is not limited to vacuuming robots, mopping robots, or floor washing robots, etc.

[0068] Please refer to Figure 1 , Figure 1 which is a schematic diagram of an application environment provided by the embodiments of the present application. As Figure 1 shown, the cleaning robot 100 is located on the ground, and the ground can be the ground of places such as living rooms or offices. The place where the cleaning robot 100 is located includes objects such as a base station, a desk, a flower pot, a sofa, a bed, and a bedside table.

[0069] The free movement of the cleaning robot 100 is mainly completed by several modules including map building, positioning, navigation, and obstacle avoidance. It can be understood that after the cleaning robot 100 senses the surrounding environment, it plans one or more paths that have no collision with environmental obstacles and achieve full-area coverage according to a certain area cleaning sequence or planning method. These modules are implemented by sensors and corresponding control programs.

[0070] In some embodiments, a lidar and / or a vision sensor (such as a visible light camera) are installed on the cleaning robot 100. Among them, the lidar scans the surrounding environment where the cleaning robot 100 is located to obtain a laser point cloud. The vision sensor takes pictures of the surrounding environment where the cleaning robot 100 is located to acquire images. The lidar and the vision sensor are respectively communicatively connected to the controller, and send the laser point cloud and the image to the controller respectively. The controller calls a map building program pre-loaded in the memory of the cleaning robot 100 to build a cleaning environment map based on the laser point cloud and / or the image. Among them, the map building program may include a program corresponding to the SLAM algorithm (Simultaneous Localization and Mapping, SLAM), which will not be introduced in detail here. The map is saved in the memory of the cleaning robot 100. When the robot moves for operation, the controller calls the map as the basis for autonomous positioning, path planning, and obstacle avoidance.

[0071] It can be understood that the SLAM algorithm has both positioning and navigation functions. During the positioning process, the lidar is controlled to rotate at high speed to emit laser, measure the distance between the cleaning robot and the obstacle, and combine with the map to judge the relative position between the cleaning robot and the obstacle, so as to achieve positioning. In some embodiments, the cleaning robot 100 can perform visual positioning based on the vision sensor. During the navigation process, cleaning control is performed based on positioning and cleaning tasks, each area to be cleaned is cleaned one by one, and a full-coverage cleaning path is planned within each area to be cleaned to complete the corresponding cleaning tasks.

[0072] In some embodiments, the vision sensor also takes pictures of the surrounding environment where the cleaning robot 100 is located. For example, an RGB-D camera is used to collect the environmental RGB image and depth information, and the target objects in the image are detected in real time, and the shape, position, motion state, etc. of the objects are identified. In some embodiments, the cleaning robot 100 further includes a front collision laser (such as a line laser). The front collision laser sensor detects the height of the obstacle in the traveling direction of the cleaning robot 100. It is installed close to the ground at a certain inclination angle for measuring the height of dynamic obstacles. In some embodiments, the detection height range of the front collision laser sensor is 2 - 50 cm, and the accuracy is ±1 cm, which is mainly used to detect low dynamic obstacles.

[0073] The cleaning robot 100 can be configured in any suitable shape to achieve specific business function operations. For example, in some embodiments, the cleaning robot 100 can be a cleaning robot based on a SLAM system.

[0074] In some embodiments, the cleaning robot 100 can include a robot body, a lidar, a controller, and a traveling mechanism. The robot body is the main structure of the cleaning robot 100, and corresponding shape structures and manufacturing materials (such as hard plastic or metals such as aluminum and iron) can be selected according to the actual needs of the cleaning robot 100. For example, it can be set as a relatively flat cylindrical shape commonly used for cleaning robots 100.

[0075] The traveling mechanism is a structural device provided on the robot body to provide the cleaning robot 100 with the ability to move. Specifically, the traveling mechanism can be implemented using any type of mobile device, such as rollers, crawlers, etc.

[0076] The lidar is disposed on the body of the cleaning robot 100 and is used to sense the obstacles in the surrounding environment of the mobile cleaning robot 100, scan to obtain lidar point cloud data, and send it to the controller so that the controller can build a map based on the lidar point cloud data and perform functions such as walking obstacle avoidance. In some embodiments, the lidar includes radars such as pulsed lidar and continuous wave lidar.

[0077] The controller is an electronic computing core built into the robot body and is used to execute logical operation steps to achieve intelligent control of the cleaning robot. The controller is communicatively connected to the lidar and is used to build a map based on the lidar point cloud data and control the cleaning robot to move. It is also used for cleaning control based on the map and cleaning tasks. For example, it controls the cleaning robot to work in a brick joint cleaning mode.

[0078] It can be understood that the controller can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components. The controller can also be any traditional processor, controller, microcontroller, or state machine. The controller can also be implemented as a combination of computing devices. For example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration, or one or more combinations of a microcontroller unit (MCU), a field-programmable gate array (FPGA), and a system-on-chip (SoC).

[0079] It can be understood that the robot 10 in the embodiments of the present invention further includes a storage module, and the storage module includes, but is not limited to, one or more of devices such as Flash memory, NAND flash memory, vertical NAND flash memory (VNAND), NOR flash memory, resistive random access memory (RRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), spin transfer torque random access memory (STT-RAM), etc.

[0080] It can be understood that in some embodiments, according to the tasks to be completed, in addition to the above functional modules, one or more other different functional modules can also be carried on the robot body. For example, the robot body can further include a clean water tank, a sewage tank, a detergent box, a dust collection box, etc. The docking device of the robot can include interfaces of the clean water tank, the sewage tank, the detergent box, the dust collection box, or charging electrode plates, etc.

[0081] The above application environment is only for illustrative purposes. In actual applications, the obstacle avoidance method and related devices provided in the following embodiments of the present invention can be further extended to other suitable application environments, rather than being limited to Figure 1 the application environment shown in

[0082] When the robot performs related tasks, such as performing a cleaning task, it will walk along a cleaning path to achieve cleaning. During the cleaning process, obstacles are detected through sensors such as lidar, and simple collision detection (such as stopping or turning when encountering an obstacle) is used to avoid obstacles. For dynamic obstacles (such as people, pets, moving objects, etc.), existing obstacle avoidance solutions are often slow to respond, unable to make intelligent decisions quickly, and the obstacle avoidance paths are often single, with poor adaptability, prone to collisions, and affecting work efficiency and safety.

[0083] In view of the above problems, some embodiments of the present application provide an obstacle avoidance method for a robot. The method includes: first, obtaining the historical trajectory of a dynamic obstacle, then inputting the historical trajectory into an LSTM prediction model to generate a predicted trajectory of the dynamic obstacle and the corresponding confidence level, and based on the confidence level and the predicted trajectory, using the DWA algorithm to generate an avoidance path, and controlling the robot to walk along the avoidance path to avoid the obstacle.

[0084] This obstacle avoidance method predicts the trajectory of dynamic obstacles, perceives the movement trend of dynamic obstacles in advance, and adaptively generates corresponding avoidance paths based on this movement trend, making the avoidance paths more reasonable, effectively avoiding collisions with dynamic obstacles, improving the obstacle avoidance performance of the robot, thereby improving the safety and work efficiency of the robot, and better adapting to the dynamically changing environment, enhancing the adaptability of the robot.

[0085] Some embodiments of the present application provide a robot. Please refer to Figure 2 , the robot 100 includes at least one processor 102 and a memory 103 ( Figure 2 connected by a bus, taking one processor as an example).

[0086] It can be understood that the processor 102 is used to provide computing and control capabilities to control the robot to execute any one of the obstacle avoidance methods for robots provided in the following embodiments.

[0087] It can be understood that the processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0088] The memory 103, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the obstacle avoidance method of the robot in the embodiments of the present application. By running the non-transitory software programs, instructions, and modules stored in the memory 103, the processor 102 can implement any one of the obstacle avoidance methods for robots provided in the following embodiments. In some embodiments, the memory 103 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 103 may further include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0089] In some embodiments, the robot 100 may further include sensors such as lidar, cameras, gyroscopes, odometers, magnetometers, accelerometers, or speedometers. These sensors assist the robot 100 in sensing the environment, constructing an environmental map, or performing other controls. It can be understood that the structure illustrated in this embodiment does not limit the robot 100. In some implementations, the robot 100 may include more or fewer components than those shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0090] As can be understood from the above, the obstacle avoidance method for the robot provided in the embodiments of the present application may be implemented by the robot, for example, executed by one or more processors of the robot. In some embodiments, the obstacle avoidance method for the robot provided in the embodiments of the present application may also be implemented and executed by other devices with computing and processing capabilities, etc. Other devices with computing and processing capabilities may be intelligent devices communicatively connected to the robot, such as servers, etc.

[0091] The following describes the obstacle avoidance method for the robot provided in the embodiments of the present application in conjunction with the exemplary applications and implementations of the robot provided in the embodiments of the present application. Please refer to Figure 3 , Figure 3 which is a schematic flowchart of the obstacle avoidance method for the robot provided in the embodiments of the present application. It can be understood that the execution subject of this obstacle avoidance method for the robot may be one or more processors of the robot.

[0092] As Figure 3 shown, the method S100 includes but is not limited to the following steps:

[0093] S10: Obtain the historical trajectory of the dynamic obstacle;

[0094] The historical trajectory refers to historical trajectory data, which includes data such as the position, speed, and height of the dynamic obstacle.

[0095] In some embodiments, the historical trajectory is the true historical trajectory of the dynamic obstacle, that is, the sensor performs real-time detection on the dynamic obstacle, obtains the observation data of the dynamic obstacle, and uses the observation data as the historical trajectory.

[0096] Since the observation data is the original data detected by the sensor in real time, it has a large amount of noise and inaccurate data, which will affect the accuracy of subsequent trajectory prediction. Therefore, in some embodiments, the observation data is filtered to filter out the noise signals in the observation data, and the data obtained after filtering is used as the historical trajectory to improve the accuracy of subsequent dynamic obstacle trajectory prediction.

[0097] Specifically, the state prediction data corresponding to N consecutive moments is determined as the historical trajectory, where N is a positive integer greater than 1, and the state prediction data represents the predicted state of the dynamic obstacle at the current moment.

[0098] By predicting the state of the dynamic obstacle at the current moment, state prediction data can be obtained. The state prediction data includes various state prediction values of the dynamic obstacle at the current moment. For example, by predicting the state of the dynamic obstacle at the current moment, the height prediction value, width prediction value, speed prediction value, speed variance prediction value, distance prediction value, etc. of the dynamic obstacle at the current moment can be obtained. These prediction values together constitute the state prediction data at the current moment.

[0099] Each moment corresponds to a state prediction data. The state prediction data corresponding to N consecutive moments is determined as the historical trajectory. For example, if N is 3 and the current moment is the k-th moment, then the state prediction data at the k-th moment, the state prediction data at the k - 1-th moment, and the state prediction data at the k - 2-th moment are determined as the historical trajectory.

[0100] In some embodiments, the Kalman filter algorithm is used to obtain the state prediction data. The Kalman filter algorithm is an optimal state estimation method based on the assumptions of linear systems and Gaussian noise. It uses the state equation and observation equation of the system, and through two steps of prediction and update, continuously corrects the estimation of the system state according to new observation data to minimize the variance of the estimation error.

[0101] Specifically, as Figure 4 shown, the obstacle avoidance method S100 further includes:

[0102] S50: Obtain the observation data of the dynamic obstacle;

[0103] In the embodiments of the present application, multi-modal sensors are used for acquisition and fusion to obtain the observation data of the dynamic obstacle. Compared with single-sensor acquisition, multi-modal sensor acquisition and fusion can improve the detection accuracy, and further improve the accuracy of subsequent trajectory prediction.

[0104] As Figure 5 shown, the step S50 includes:

[0105] S51: Use multiple sensors to observe the dynamic obstacle and collect visual data, point cloud data, and line laser data;

[0106] For example, a vision sensor is used to observe dynamic obstacles, an RGB-D camera is used to collect environmental RGB images and depth information, and a YOLOv5 model is used to detect the target bounding boxes of dynamic obstacles in real time to obtain visual data. A lidar is used to observe dynamic obstacles to obtain 2D / 3D point cloud data, and the point cloud data includes data such as the positions and speeds of dynamic obstacles. A line laser sensor is used to observe dynamic obstacles to obtain line laser data, and the line laser data includes data such as the heights of dynamic obstacles. In some embodiments, each data can be preprocessed. For example, in combination with the visual data, the point cloud data regarding static obstacles in the point cloud data is removed to improve the accuracy of the observed data.

[0107] S52: Determine a preset sliding window, where the preset sliding window is a time range centered on a preset moment;

[0108] S53: Obtain target visual data, target point cloud data, and target line laser data under the preset sliding window;

[0109] Since the data collection frequencies of different sensors are inconsistent and the data generation rates are different, it is necessary to add hardware-level timestamps to each sensor data and align the sensor data at the same moment through a sliding window to solve the problem of time asynchrony of each data.

[0110] First, determine a preset sliding window, which is a time range centered on a preset moment. The preset moment is a timestamp. For example, in the embodiments of the present application, based on the point cloud data, the lidar timestamp is used as the preset moment, the allowable time range before and after the lidar timestamp is determined, the visual data within the time range is obtained to get the target visual data, the line laser data within the time range is obtained to get the target line laser data, and the point cloud data corresponding to the timestamp is used as the target point cloud data.

[0111] There can be one or multiple preset sliding windows. For example, obtain the point cloud data at the preset moment, then obtain the visual data under the first preset sliding window to get the target visual data, and then obtain the line laser data under the second preset sliding window to get the target line laser data. The first preset sliding window and the second preset sliding window are both centered on the preset moment but have different time ranges, so as to ensure that effective visual data and line laser data can be obtained near the lidar timestamp.

[0112] In some embodiments, the first preset sliding window is based on being able to select the visual data closest to the preset moment, which can reduce the time error. The first preset sliding window can be a little smaller to avoid introducing too much redundant data. The second preset sliding window is based on being able to include the data of multiple scans, which can improve the detection accuracy through multi-frame fusion.

[0113] For example, if the reference frequency of the lidar is 10 HZ (data is collected every 100 ms), the frequency of the vision sensor is 30 HZ (one frame every 33 ms), and the frequency of the line laser sensor is 40 HZ (one frame every 25 ms), then based on the point cloud data, within the allowable time range before and after its timestamp, the target vision data and the target line laser data are extracted respectively.

[0114] If the lidar timestamp T = 1000 ms, then centered on the lidar timestamp T = 1000 ms, expand it by ±5 ms before and after (i.e., the first preset sliding window is 995 ms to 1005 ms). Within this range, there may be only 1 frame of vision data (such as the frame at 997 ms or 1003 ms). Selecting the nearest frame can minimize the time error (such as selecting the frame at 997 ms, with a difference of only 3 ms from the lidar timestamp). Centered on the lidar timestamp T = 1000 ms, expand it by ±10 ms before and after (i.e., the second preset sliding window is 990 ms to 1010 ms). Within this range, there may be 3 line laser scan data (such as 990 ms, 1015 ms, etc.). Through multi-frame data verification, the noise interference of single scans can be excluded, and the reliability of dynamic obstacle detection can be improved.

[0115] S54: Convert the target vision data, the target point cloud data, and the target line laser data into data in the robot coordinate system to obtain the observation data.

[0116] There are differences in the coordinate systems of each data. To improve the accuracy of subsequent data fusion, first unify the coordinate systems and then perform data fusion.

[0117] Taking the forward direction of the robot as the X-axis and the center of the robot as the origin, establish the robot coordinate system. For the target vision data, convert the pixel coordinates into data in the robot coordinate system through the calibration matrix. For the target point cloud data, compensate the target point cloud coordinates through the installation position offset (such as the lidar is 10 cm in front of the sweeper) to obtain the point cloud data in the robot coordinate system. For the target line laser data, obtain the line laser data in the robot coordinate system according to the laser emission angle (such as horizontal ±15°) and the installation offset of the line laser sensor.

[0118] Finally, fuse the target vision data, the target point cloud data, and the target line laser data after coordinate transformation, and combine them into a feature vector as the observation data of dynamic obstacles.

[0119] Therefore, a multi-modal sensor is used to observe dynamic obstacles, obtain observation data, improve the accuracy and reliability of the observation data, and for different environmental conditions, more accurate observation data can be obtained, enhancing the environmental adaptability of data collection.

[0120] S60: Based on the observation data, the Kalman filtering algorithm is used to generate the state prediction data at the current moment.

[0121] The observation data is the data detected by the sensor in real time, and it has a large noise. Therefore, the Kalman filtering algorithm is used to filter the observation data to improve the accuracy of the data.

[0122] Specifically, as Figure 6 shown, this step S60 includes:

[0123] S61: Obtain the state prediction data and the motion model at the previous moment;

[0124] S62: Predict the original state prediction data at the current moment according to the state prediction data and the motion model at the previous moment;

[0125] S63: Fuse the observation data and the original state prediction data to generate the state prediction data at the current moment.

[0126] The Kalman filtering algorithm includes three stages, namely the definition stage, the prediction stage, and the update stage. In the definition stage, the state prediction data is defined as x = [x, y, vx, vy, w, h] T , where (x, y) is the position, (v x , v y ) is the velocity, w is the width, and h is the height.

[0127] In the prediction stage, according to the state prediction data and the motion model at the previous moment, the state at the current moment is predicted. Common motion models include the uniform motion model, the uniformly accelerated motion model, etc. In the embodiment of the present application, if the uniform motion model is adopted, and the current moment is the k-th moment, and the previous moment is the (k - 1)-th moment, then the original state prediction data at the current moment can be obtained through the following formula:

[0128]

[0129] where, x k is the original state prediction data at the k-th moment, x k-1 is the state prediction data at the (k - 1)-th moment, F is the state transition matrix, Δt is the sensor sampling interval, and in the embodiment of the present application, Δt takes 0.1.

[0130] Therefore, if the motion model is a uniform motion model, the original state prediction data at the current moment can be obtained based on the state prediction data at the previous moment and the state transfer matrix.

[0131] The original prediction data also needs to be corrected to obtain the final state prediction data at the current moment. Therefore, in the update phase, the observation data is fused with the original state prediction data to correct the original state prediction data and correct the prediction results.

[0132] Specifically, the difference between the observed data and the original state prediction data is first calculated, then the product between the difference and the Kalman gain matrix is calculated, and finally the product is added to the observed data to generate the state prediction data at the current moment.

[0133] By using the Kalman filter method in the above embodiment, state prediction data corresponding to N consecutive moments can be obtained. The state prediction data is the data after noise is eliminated, which will improve the accuracy of subsequent trajectory prediction.

[0134] S20: Inputting the historical trajectory into the LSTM prediction model to generate a predicted trajectory of the dynamic obstacle and a corresponding confidence level, wherein the confidence level represents the reliability of the predicted trajectory;

[0135] The LSTM prediction model, also known as the Long-Short Term Memory (LSTM) prediction model, is a special recurrent neural network that is mainly used to process and predict data with sequential properties. It is widely used in natural language processing, time series prediction and other fields.

[0136] Confidence indicates the reliability of the predicted trajectory. It is usually expressed as a probability or a numerical value, generally ranging from 0 to 1, such as a confidence level of 90% or 60%. A high confidence level indicates that the LSTM prediction model is highly confident that the prediction result is accurate, and the predicted trajectory is highly reliable. A low confidence level indicates that the LSTM prediction model is less certain about the prediction result, and the predicted trajectory is less reliable.

[0137] Input the historical trajectory into the LSTM prediction model to generate a predicted trajectory and the corresponding confidence level. For example, input 5 consecutive frames of trajectory from the historical trajectory into the LSTM prediction model to generate a trajectory for the next 0.5 seconds and the corresponding confidence level, such as 75%.

[0138] S30: Based on the confidence level and the predicted trajectory, generate an avoidance path using a DWA algorithm;

[0139] The DWA algorithm, namely the Dynamic Window Approach algorithm, is an algorithm for robot path planning and motion control. The DWA algorithm defines a dynamic speed window based on the current speed limit of the robot, searches for all possible speed combinations within this speed window, and evaluates each speed combination through an evaluation function to select the optimal speed, enabling the robot to reach the target position safely and efficiently while avoiding collisions with obstacles.

[0140] And different confidence levels result in different selected speed windows, and thus different generated avoidance paths. Specifically, as Figure 7 shown, step S30 includes:

[0141] S31: Determine the constraint conditions, where the constraint conditions include the current speed limit of the robot;

[0142] Due to factors such as the current environment limit of the robot and the driving motor, the robot needs to move within the constraint conditions. The current speed limit includes the current linear speed and acceleration limits, that is, the maximum linear speed, maximum acceleration that can be achieved, as well as the minimum linear speed, minimum acceleration that can be achieved, etc.

[0143] S32: Determine the dynamic window based on the confidence level and the constraint conditions, where the dynamic window includes the value range of the speed of the robot;

[0144] Different confidence levels result in different selected dynamic windows. If the confidence level is high, the selected dynamic window is also large, indicating a large value range of the speed. If the confidence level is low, the selected window is also small, indicating a small value range of the speed.

[0145] For example: The constraint condition is [vmin, vmax]. If the confidence level is greater than or equal to the first preset threshold, the complete speed space can be used as the dynamic window, and subsequently, the speed of the robot is uniformly sampled within [vmin, vmax]. If the confidence level is less than the first preset threshold and greater than the second preset threshold, the speed range is reduced, and the dynamic window is [v1, v2], where v1 is greater than vmin and v2 is less than vmax. Subsequently, the speed of the robot is uniformly sampled within [v1, v2]. If the confidence level is less than the second preset threshold, the speed range is further reduced, and the dynamic window is [v3, v4], where v3 is greater than v1 and v4 is less than v2. Subsequently, the speed of the robot is uniformly sampled within [v3, v4].

[0146] S33: Uniformly sample the speed of the robot within the dynamic window to obtain multiple groups of sampled speeds;

[0147] S34: For each set of the sampling speeds, based on the motion model of the robot, predict the motion path of the robot at the sampling speed to obtain a first candidate path;

[0148] The motion model of the robot takes into account information such as the current position, attitude, and speed of the robot. For each set of sampling speeds, based on the motion model of the robot, predict the motion trajectory of the robot in the future for a period of time at the sampling speed. Since there are multiple sets of sampling speeds, multiple motion paths, that is, multiple first candidate paths, can be obtained.

[0149] S35: From the first candidate paths, eliminate the paths that intersect with the predicted trajectory to obtain second candidate paths;

[0150] Among the multiple first candidate paths, there may be paths that intersect with the predicted trajectory of the dynamic obstacle. To avoid subsequent collisions, the paths that intersect with the predicted trajectory are eliminated, and the remaining first candidate paths are used as second candidate paths.

[0151] S36: Construct an evaluation function based on the target direction, path smoothness, static obstacle distance, and dynamic safety term;

[0152] The evaluation function comprehensively considers multiple factors. In the embodiment of the present application, four evaluation factors, namely the target direction, path smoothness, static obstacle distance, and dynamic safety term, are adopted, and the above four evaluation factors are weighted and summed according to certain weights to obtain the evaluation function.

[0153] Then the evaluation function is:

[0154] G(v,ω) = w1 * target direction + w2 * path smoothness + w3 * static obstacle distance + w4 * dynamic safety term (2)

[0155] Where, G(v,ω) is the evaluation function, w1 is the weight of the target direction, w2 is the weight of the path smoothness, w3 is the weight of the static obstacle distance, and w4 is the weight of the dynamic safety term.

[0156] The specific values of each weight can be set according to needs. In the embodiment of the present application, w1 is 0.4, w2 is 0.3, w3 is 0.2, and w4 is 0.1.

[0157] The target direction refers to the cosine value of the included angle between the current robot heading and the target point direction, which is obtained by the following formula:

[0158]

[0159] θgoal is the target point direction, θrobot is the current robot heading, and the larger the value of the target direction, the more the path is towards the target.

[0160] The path smoothness is calculated by curvature and obtained through the following formula:

[0161] Smoothness = 1 / (1 + |ω| * Δt) (4)

[0162] where ω is the angular velocity and Δt is the sensor time interval. The smaller the angular velocity, the smoother the path.

[0163] The static obstacle distance refers to the minimum distance between each trajectory point of the second candidate path and the static obstacle, which is normalized as the score.

[0164] The dynamic safety term refers to the collision probability between the second candidate path and the dynamic obstacle and is obtained through the following formula:

[0165] Dynamic safety term = 1 - t collision / threshold (5)

[0166] where t collision is the collision time and threshold is the collision time threshold. In the embodiments of the present application, threshold = 0.3s, and the shorter the collision time, the lower the score.

[0167] In some embodiments, if the collision time t collision is less than 0.3s, before determining the avoidance path, first control the robot to stop suddenly, and then retreat 0.5m to prevent collision with the dynamic obstacle.

[0168] S37: Evaluate the second candidate path using the evaluation function to generate a corresponding evaluation score;

[0169] S38: Determine the second candidate path with the highest evaluation score as the avoidance path.

[0170] Evaluate each second candidate path using the evaluation function to generate a corresponding evaluation score. The second candidate path with the highest evaluation score is the optimal path among all the second candidate paths, and it is determined as the avoidance path.

[0171] In some embodiments, before generating the avoidance path, in order to reduce the probability of collision with the dynamic obstacle, different avoidance strategies can also be selected based on the confidence level or the speed of the robot. After executing the avoidance strategy, then generate the avoidance path to complete the complete avoidance.

[0172] Specifically, if the confidence level is greater than or equal to the first preset threshold, then start the step of generating an avoidance path based on the confidence level and the predicted trajectory using the DWA algorithm. If the confidence level is less than the first preset threshold, then trigger the line laser sensor to scan and verify whether the obstacle is a dynamic obstacle. If it is verified that the obstacle is a dynamic obstacle, then start the step of generating an avoidance path based on the confidence level and the predicted trajectory using the DWA algorithm.

[0173] When the confidence level is greater than or equal to the first preset threshold, it indicates that the confidence level is relatively high and the reliability of the future trajectory predicted by the LSTM prediction model is relatively high. Then, the corresponding avoidance path can be directly generated.

[0174] When the confidence level is less than the first preset threshold, it indicates that the confidence level is not high, or even low. The reliability of the future trajectory predicted by the LSTM prediction model is relatively low, and the obstacle may be a static obstacle. Then, trigger the line laser sensor to further scan and verify at a high frequency. For example, perform a high-frequency scan at 40HZ, obtain the velocity variance of the obstacle, and determine whether the obstacle is a dynamic obstacle according to the velocity variance.

[0175] Through continuous high-frequency scanning, obtain the displacement change of the obstacle, and then calculate the velocity variance. When the velocity variance is greater than the velocity setting value (such as 0.5m / s2), it is determined that the obstacle is a dynamic obstacle; otherwise, it is marked as a static obstacle. After determining that it is a dynamic obstacle, then use the DWA algorithm to generate the corresponding avoidance path.

[0176] In some embodiments, if the confidence level is extremely low, in order to prevent the robot from colliding with the obstacle, the robot can be first controlled to execute a conservative avoidance strategy, and then further scanned and verified at a high frequency. That is, if the confidence level is less than the second preset threshold, then control the speed of the robot to decrease to the first speed, where the second preset threshold is less than the first preset threshold.

[0177] For example: the first preset threshold is 90%, and the second preset threshold is 60%. Then, if the confidence level ≥ 90%, directly start using the DWA algorithm to plan the avoidance path. If 60% ≤ C < 90%, then trigger the line laser sensor to perform a high-frequency scan and verification. If it is verified that the obstacle is a dynamic obstacle, then start using the DWA algorithm to plan the avoidance path. If the confidence level C < 60%, then start the conservative avoidance strategy, control the robot to decelerate to the first speed (such as 0.1m / s), and then start using the DWA algorithm to plan the avoidance path.

[0178] In the case of a low confidence level, using the line laser sensor for high-frequency scan and verification, or controlling the robot to decelerate and then using the line laser sensor for high-frequency scan and verification can effectively reduce the possibility of the robot colliding with dynamic obstacles and improve the safety of the robot.

[0179] If the predicted speed of a dynamic obstacle is high, the likelihood of a collision is high and the safety is low. Or if the height of a dynamic obstacle is high, the dynamic obstacle may be a human. If a human collides with the robot, the risk factor is high and the safety is low.

[0180] Therefore, in order to reduce the collision risk and improve safety, in some embodiments, before using the DWA algorithm to generate an avoidance path, the speed data in the state prediction data is used to further determine whether the dynamic obstacle is moving at high speed (such as running), and the height data in the state prediction data is used to further determine the type of the dynamic obstacle.

[0181] Specifically, the speed data and height data in the state prediction data are obtained. If the speed data is greater than the first speed threshold or the height data is greater than the first height threshold, the robot is controlled to stop and then walk a preset distance in the first direction, where the first direction is the opposite direction to the current walking direction of the robot. If the speed data is less than the second speed threshold or the height data is less than the second height threshold, the step of generating an avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory is started, where the second speed threshold is less than the first speed threshold and the second height threshold is less than the first height threshold.

[0182] For example: if the first speed threshold is 0.8 m / s and the second speed threshold is 0.3 m / s, then if the speed data is greater than 0.8 m / s, the dynamic obstacle may be a running obstacle, such as a running pet, and the robot is controlled to stop and then retreat a preset distance (such as retreating 0.5 m) to reduce the likelihood of a collision, and then the step of generating an avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory is started. If the speed data is less than 0.3 m / s, the dynamic obstacle is walking slowly and the likelihood of a collision is low, and then the step of directly generating an avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory is started, and the robot can avoid according to the avoidance path.

[0183] Another example: if the first height threshold is 1 m and the second height threshold is 0.5 m, then if the height data is greater than 1 m, the dynamic obstacle may be a human, and the robot is controlled to stop and then retreat a preset distance (such as retreating 0.5 m) to reduce the likelihood of a collision, and then the step of generating an avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory is started. If the height data is less than 0.5 m, the dynamic obstacle may be a pet, and then the step of directly generating an avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory is started, and the robot can avoid according to the avoidance path.

[0184] It should be noted that the first speed threshold, the second speed threshold, the first height threshold, and the second height threshold can all be set as needed, and the embodiments of the present application do not limit this. In other embodiments, the speed variance data in the state prediction data can also be used to select different avoidance strategies. When the speed variance data is large, it indicates that the possibility of the dynamic obstacle running is relatively high, so the robot is controlled to stop, then retreat a preset distance, and then start the step of planning an avoidance path. When the speed variance data is small, it indicates that the dynamic obstacle moves slowly, and the step of directly starting to plan an avoidance path can be performed.

[0185] S40: Control the robot to walk along the avoidance path to avoid the dynamic obstacle.

[0186] In summary, this obstacle avoidance method adaptively generates a corresponding avoidance path based on the future trajectory of the dynamic obstacle. The avoidance path is more reasonable, improving the obstacle avoidance performance of the robot, and thus enhancing the safety and adaptability of the robot.

[0187] In some embodiments, during the avoidance process, the state of the dynamic obstacle is also monitored to determine the next path and actions.

[0188] Specifically, as Figure 8 shown, this obstacle avoidance method S100 further includes:

[0189] S70: Monitor the state of the dynamic obstacle and generate state information;

[0190] S80: If the state information indicates that the dynamic obstacle is in a leaving state, control the robot to continue cleaning along the cleaning path;

[0191] S90: If the state information indicates that the dynamic obstacle is in a continuous staying state, mark the area where the dynamic obstacle is located as a dynamic risk area, and after the cleaning task is completed, return to the dynamic risk area for supplementary cleaning.

[0192] Use multiple sensors to continuously monitor the dynamic obstacle. If it is detected that the dynamic obstacle leaves, control the robot to end the avoidance and continue cleaning along the cleaning path to complete the cleaning task. If it is detected that the dynamic obstacle continuously stays, the possibility of collision is relatively high, so continue to avoid, mark the area where the dynamic obstacle is located as a dynamic risk area, do not perform the cleaning task on this dynamic risk area, and after the cleaning task is completed, return to the dynamic risk area for supplementary cleaning to reduce the missed cleaning rate and ensure the cleaning coverage rate.

[0193] Taking the first height threshold as 1m, the second height threshold as 50cm, the first speed threshold as 0.8m / s, the second speed threshold as 0.3m / s, the evaluation function as G(v, ω) = 0.4 * target direction + 0.3 * path smoothness + 0.2 * static obstacle distance + 0.1 * dynamic safety term, the first preset threshold as 90%, and the second preset threshold as 60% as an example, combined with a specific application scenario, the working process of this obstacle avoidance method can be described as follows:

[0194] Scenario 1: Avoidance of human bodies in a home scenario.

[0195] The robot cleans the living room along the cleaning path and detects a dynamic obstacle 1.2 meters ahead. The Kalman filter algorithm is used to generate state prediction data, which indicates that the dynamic obstacle moves horizontally at a speed of 0.5m / s, and the dynamic obstacle may be a human.

[0196] The historical trajectory of the dynamic obstacle is input into the LSTM prediction model to predict the prediction trajectory and confidence of the dynamic obstacle. If the LSTM prediction model predicts that the dynamic obstacle will enter the cleaning path after 0.3s with a confidence of 85%, the line laser sensor is triggered for high-frequency scanning verification. After confirming that the obstacle is indeed a dynamic obstacle, the DWA algorithm is started to generate an avoidance path.

[0197] With a confidence of 85%, the corresponding dynamic window is [v1, v2]. The speed of the robot is uniformly sampled within the dynamic window to obtain multiple groups of sampled speeds. For each group of sampled speeds, two first candidate paths are generated based on the motion model of the robot:

[0198] First candidate path 1: Turn right ω = 0.5rad / s to generate a semi-circular path that has no intersection with the predicted trajectory of the dynamic obstacle, and the evaluation function score Gnew = 85.

[0199] First candidate path 2: Decelerate to v = 0.2m / s and go straight.

[0200] There is a potential intersection between the first candidate path 2 and the predicted trajectory of the dynamic obstacle, and a collision may occur. Then, the first candidate path 2 is excluded, and the remaining first candidate path 1 is used as the second candidate path. Since there is only one second candidate path, this second candidate path is determined as the avoidance path.

[0201] Control the robot to walk along the avoidance path. The robot completes the detour within 1.2 seconds. During the detour, the dynamic obstacle is continuously monitored. After the dynamic obstacle leaves, control the robot to resume the original cleaning path and accelerate to 0.4m / s to continue completing the cleaning task.

[0202] Scenario 2: Emergency avoidance when a pet suddenly breaks in.

[0203] When the robot is cleaning the bedroom, a pet cat rushes into the cleaning area from the side at a speed of 1.2 m / s. The visual sensor captures the low moving target, and the Kalman filtering algorithm is used to obtain the state prediction data, which is used as the historical trajectory of the dynamic obstacle. The input LSTM prediction model generates the prediction trajectory and confidence level of the dynamic obstacle within the next 0.2 seconds.

[0204] Since the speed of the dynamic obstacle (1.2 m / s) is greater than 0.8 m / s, indicating that the speed of the dynamic obstacle is relatively fast, the robot can be controlled to execute an emergency avoidance strategy, that is, control the robot to stop suddenly, then retreat 0.5 m. Then, if the confidence level is 75%, the line laser sensor is triggered for high-frequency scanning verification. If the line laser sensor passes the scanning verification for 3 consecutive frames and confirms that the obstacle is a dynamic obstacle, an avoidance path (such as a left bypass path) is generated based on the DWA algorithm, otherwise it is marked as a static obstacle.

[0205] When the robot is walking along the avoidance path, it continuously monitors the dynamic obstacle. If the dynamic obstacle stays continuously, the area is marked as a dynamic risk area. The robot bypasses the dynamic risk area and continues cleaning. After the cleaning task is completed, it returns to the dynamic risk area for supplementary cleaning to prevent missed cleaning. If the dynamic obstacle leaves, the robot ends the avoidance and re-plans the cleaning path to complete the cleaning task.

[0206] Scenario 3: Avoidance of low obstacles under the bed.

[0207] During the cleaning process, the robot detects a 12 cm dynamic obstacle under the bed, and based on the state prediction data, it determines that the speed variance of the dynamic obstacle is relatively large (speed variance > 0.8 m / s²). Then, because the vision under the bed is easily blocked and it is in a dark environment, the possibility of collision is relatively high. Therefore, an avoidance path is directly generated based on the DWA algorithm, and the area is marked as a dynamic risk area. After the cleaning task is completed, it returns to the dynamic risk area for supplementary cleaning.

[0208] In summary, this obstacle avoidance method predicts the trajectory of dynamic obstacles, perceives the movement trend of dynamic obstacles in advance, and adaptively generates corresponding avoidance paths based on this movement trend, making the avoidance paths more reasonable, effectively avoiding collisions with dynamic obstacles, improving the obstacle avoidance performance of the robot, and thus improving the safety and working efficiency of the robot, and better adapting to the dynamically changing environment, enhancing the adaptability of the robot.

[0209] The embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions for causing an electronic device to execute the obstacle avoidance method of the robot provided by the embodiment of the present application. [[ID=۲۰]]

[0210] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0211] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0212] As an example, the executable instructions may or may not correspond to a file in the file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program under discussion, or stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or code portions).

[0213] As an example, the executable instructions may be deployed to execute on one computing device (including devices such as smart terminals and servers), or on multiple computing devices located at one location, or, on multiple computing devices distributed at multiple locations and interconnected through a communication network.

[0214] It should be noted that the device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. One can select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] Through the description of the above embodiments, those of ordinary skill in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course also by hardware. Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The said program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the said storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, and the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above. For the sake of brevity, they are not provided in detail; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An obstacle avoidance method for a robot, characterized in that, Including: Obtain the historical trajectory of the dynamic obstacle; Input the historical trajectory into the LSTM prediction model to generate the predicted trajectory of the dynamic obstacle and the corresponding confidence level, where the confidence level represents the reliability of the predicted trajectory; Based on the confidence level and the predicted trajectory, use the DWA algorithm to generate an avoidance path; Control the robot to walk along the avoidance path to avoid the dynamic obstacle.

2. The obstacle avoidance method according to claim 1, wherein The obtaining of the historical trajectory of the dynamic obstacle includes: Determine the state prediction data corresponding to N consecutive moments as the historical trajectory, where N is a positive integer greater than 1, and the state prediction data represents the predicted state of the dynamic obstacle at the current moment.

3. The obstacle avoidance method according to claim 2, wherein Before the obtaining of the historical trajectory of the dynamic obstacle, the method further includes: Obtain the observation data of the dynamic obstacle; Based on the observation data, use the Kalman filtering algorithm to generate the state prediction data at the current moment.

4. The obstacle avoidance method according to claim 3, wherein The generating of the state prediction data at the current moment based on the observation data using the Kalman filtering algorithm includes: Obtain the state prediction data and the motion model at the previous moment; Predict the original state prediction data at the current moment according to the state prediction data and the motion model at the previous moment; Fuse the observation data and the original state prediction data to generate the state prediction data at the current moment.

5. The obstacle avoidance method according to claim 4, wherein The fusing of the observation data and the original state prediction data to generate the state prediction data at the current moment includes: Calculate the difference between the observation data and the original state prediction data; Calculate the product of the difference and the Kalman gain matrix; Add the product to the observation data to generate the state prediction data at the current moment.

6. The obstacle avoidance method according to claim 1, wherein Before the generating of the avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory, the method further includes: If the confidence level is greater than or equal to the first preset threshold, start the step of generating the avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory; 7. The obstacle avoidance method according to claim 6, characterized in that, If the confidence level is less than the first preset threshold, trigger the line laser sensor to scan and verify whether the obstacle is a dynamic obstacle. If it is verified that the obstacle is the dynamic obstacle, start the step of generating the avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory. Before the triggering of the line laser sensor to scan and verify whether the obstacle is a dynamic obstacle, the method further includes:

8. The obstacle avoidance method according to claim 3, wherein If the confidence level is less than the second preset threshold, control the speed of the robot to decrease to the first speed, where the second preset threshold is less than the first preset threshold. Before the generating of the avoidance path using the DWA algorithm based on the confidence level and the predicted trajectory, the method further includes: Obtain the speed data and the height data in the state prediction data at the current moment; If the speed data is greater than the first speed threshold or the height data is greater than the first height threshold, control the robot to stop and then walk a preset distance in the first direction, where the first direction is the opposite direction to the current walking direction of the robot; If the speed data is less than the second speed threshold or the height data is less than the second height threshold, then start the step of generating an avoidance path based on the confidence and the predicted trajectory using the DWA algorithm, where the second speed threshold is less than the first speed threshold and the second height threshold is less than the first height threshold.

9. The obstacle avoidance method according to any one of claims 1-8, characterized in that, The generating of the avoidance path based on the confidence and the predicted trajectory using the DWA algorithm includes: Determine the constraint conditions, where the constraint conditions include the current speed limit of the robot; Determine a dynamic window based on the confidence and the constraint conditions, where the dynamic window includes the value range of the speed of the robot; Uniformly sample the speed of the robot within the dynamic window to obtain multiple sets of sampled speeds; For each set of the sampled speeds, predict the movement path of the robot at the sampled speed based on the movement model of the robot to obtain a first candidate path; From the first candidate paths, eliminate the paths that intersect with the predicted trajectory to obtain second candidate paths; Construct an evaluation function based on the target direction, path smoothness, static obstacle distance, and dynamic safety term; Evaluate the second candidate paths using the evaluation function to generate corresponding evaluation scores; Determine the avoidance path as the second candidate path with the highest evaluation score.

10. The obstacle avoidance method according to any one of claims 3-8, characterized in that, The obtaining of the observation data of the dynamic obstacle includes: Use multiple sensors to observe the dynamic obstacle and collect visual data, point cloud data, and line laser data; Determine a preset sliding window, where the preset sliding window is a time range centered on a preset moment; Obtain the target visual data, target point cloud data, and target line laser data under the preset sliding window; Convert the target visual data, the target point cloud data, and the target line laser data into data in the robot coordinate system to obtain the observation data.

11. The obstacle avoidance method according to any one of claims 1-8, characterized in that, After controlling the robot to walk along the avoidance path, the method further includes: Monitor the state of the dynamic obstacle and generate state information; If the state information indicates that the dynamic obstacle is in a leaving state, control the robot to continue cleaning along the cleaning path; If the state information indicates that the dynamic obstacle is in a continuous staying state, mark the area where the dynamic obstacle is located as a dynamic risk area, and after the cleaning task is completed, return to the dynamic risk area for supplementary cleaning.

12. A robot, characterized in that, Includes a memory and a processor, the memory is connected to the processor, the processor is configured to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the robot implements the obstacle avoidance method of the robot according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by the processor, the processor executes the obstacle avoidance method of the robot according to any one of claims 1-11.

Images